IMPORTANT NOTES This is not the correct article for you IF you only need to download the last snapshot to a USB or other storage for a bare metal restore. For such cases, please see LFTP4WIN: Linux FTP client for Windows (which will also allow you to download a full snapshot from the cloud vault) This is the correct article for you if minimum downtime is required and the ETA for a virtual disk export is not feasible. |
Your failover and restore scenario 1. A bare metal restore is needed for Direct-to-Cloud protected system(s) without Local Cache enabled or when the Local Cache is not usable. 2. The protected system(s) are running in Virtual Office and backups are continuing to the cloud. (Please note that Virtual Office setup is not covered by this knowledge base article.) 3. A temporary local BDR will be used to perform a vault recovery of a chosen snapshot for use in a bare metal restore of a protected system. |
TABLE OF CONTENTS To perform a D2C/ No Local Cache restore using a temporary BDR and incremental vault recovery, you must complete each of the six following processes for your failover and restore scenario: |
Process 1.
OS configuration of a temporary BDR so you can perform a vault recovery
A. Requirements
- Hardware must be compatible with Ubuntu Linux 20.0464-bit or newer
- 16GB ECC RAM
- Six (6) cores; Processor must support hardware-assisted virtualization.
- 256GB for the OS drive. NVMe is supported. Two additional storage drives large enough to hold the full snapshot are required. Note: SMR-based hard drives are not suitable as x360Recover storage devices.
- x360Recover uses a software RAID. If your BDR has a hardware RAID controller, it must be set to a RAID0 or JBOD configuration.
Virtual machine option
If you choose to create a temporary BDR as a virtual machine, click to choose one of the two types below to view the requirements.
IMPORTANT: Be aware that deploying x360Recover as a virtual appliance may have performance limitations when compared to deploying on bare metal.
Steps to use a VMware temporary BDR
VMware ESXi 5.5 and newer is supported for deploying Axcient x360Recover devices in a virtual environment.
- To operate as a virtual machine, an x360Recover appliance requires support for nested virtualization from the host hypervisor environment, to allow for recovery.
- x360Recover devices only support nested virtualization for Intel processors on VMware ESX 5.5 and newer.
- AMD-based servers are not supported on VMware by x360Recover for virtualizing.
NOTE: If you make changes to the virtual machine while it is running, you must reboot for the changes to be applied.
A. Minimum requirements to run as a guest on VMware:
1. 6 CPU cores assigned
2. 16GB RAM (preferably more)
3. Configured as either legacy MBR or UEFI BIOS
4. A system disk for the operating system of at least 200GB
5. One or more additional virtual disks for storage
B. Create a new virtual machine guest (See the Broadcom VMware installation instructions specific to your version)
1. Choose the most recent Ubuntu Linux operating system available as the target OS
2. The Virtual Machine version must be vmx-09 or higher
3. The Configuration Parameters should have the following lines.
If not, then click to add a row, line or parameter for each of the following:
a. hotadd = "FALSE"
b. featMask.vm.hv.capable = "Min:1"
c. vhv.enable = "TRUE"
d. disk.EnableUUID = "true"
4. Disable Enable CPU Hot
5. Expose hardware-assisted virtualization to the guest OS.
6. Enable virtualized CPU performance counters
Close this section and continue with Process 1-B Download the ISO’s and Utility
DeleteSteps to use a Hyper-V temporary BDR
The Hyper-V host must support nested virtualization.
This requires either:
- Windows Server 2016 or newer operating system
- Intel processor with VT-x and EPT technology
or
- Windows Server 2022
- AMD EPYC/Ryzen processor or later
1. 6 CPU cores assigned
2. 16GB RAM (preferably more)
3. Configured as Generation-2 (EFI-BIOS)
4. A system disk for the operating system of at least 200GB
5. One or more additional virtual disks for storage
6. We recommend that Secure Boot should be disabled for the Guest VM
B. Create a new guest virtual machine
1. Download the x360Recover installation media from the Axcient download page at https://help.axcient.com/General-Axcient-Information/downloads/
2. Save the x360Recover installation media in a location accessible from the Hyper-V host.
3. Ensure the guest is created as Generation 2 or later and name it x360Recover.
4. Do not use Dynamic Memory
5. Use the requirements in A-1 above to complete the VM creation.
C. Configure a new guest virtual machine
1. Hyper-V Manager
a. Disable Secure Boot
b. Enable Guest Services.
c. Create one or more virtual disks large enough for the full snapshot and subsequent incremental backups.
d. Enable Nested Virtualization via PowerShell with the following in the command line:
Set-VMProcessor -VMName x360Recover -ExposeVirtualizationExtensions $true
Close this section and continue with Process 1-B Download the ISO’s and Utility
DeleteB. Download the ISOs and the Utility
x360Recover runs on a custom Ubuntu 64-bit Linux operating system.
The x360Recover installation ISO will deploy the basic operating system, along with all x360Recover application components.
1. Open the ISO Terms - x360Recover (Replibit) downloads page.
At the bottom of the page, download these two files:
- ISO to USB Utility
and the
- Hardware Enablement HE ISO
Note on download times: It may take up to five minutes to complete the download of the installation ISO. (The typical download is approximately 1.5 GB.) You can check your View Downloads pop-up window for status.
2. Open the ISO Terms - x360Recover Bare Metal Restore downloads page
Scroll to the bottom of the page.
Click to agree to the terms to start the download for the Standard Bare Metal Restore ISO.
C. Create a bootable USB flash drive
1. Once you have downloaded the x360Recover Installation ISO and the Bare Metal Restore ISO, you must make individual bootable images on separate USB flash drives.
- If you prefer, an additional option is to burn the ISO’s onto separate DVD’s, (if the systems have a DVD reader.)
- The base operating system installation from USB or DVD typically takes about ten minutes.
- We recommend that you use our utility, as other ISO to USB utilities may not work as successfully with Axcient’s BMR process.
2. In your Downloads folder, right click the isotousb_setup.exe and run as administrator.
3. Your device should be automatically detected and selected.
4. If your device is not automatically detected and selected, choose the correct device from the USB device drop-down list.
5. Click Browse and select the x360Recover ISO image. Ensure that File system is set to FAT32 as the default or select FAT32 from the drop-down list.
6. Enter a New Volume label if desired.
7. Click Burn to begin the file copy process
D. Installation prerequisites
1. For multi-disk systems, it is advisable to disconnect the storage drives so that the desired OS drive is the only choice for installation.
2. To ensure that the installation can be completed successfully, the system should be connected to a network with DHCP and open internet access.
3. Power on the system and press the appropriate key to enter the BIOS setup menu. (Typically, this is <Del>, <F1> or <F2> on most white-box systems, or <F9> on HP servers)- x360Recover and the Linux operating system do not support booting from motherboard (fake) RAID controllers. (BIOS has an option to select RAID for the onboard SATA ports.) This onboard ‘RAID’ functionality is really a software-based RAID that requires Windows-only drivers that are not available on Linux. If your motherboard has such an option, ensure that the SATA ports are configured in SATA or AHCI mode for compatibility with Linux. Consult your system motherboard documentation for instructions on selecting between RAID and SATA/AHCI boot modes. If you are unable to do such, then set to JBOD and each drive as aRAID0.
- When booting from a USB device, it may be necessary to change the default boot order of the system.
- Locate the boot order management page within the BIOS and configure the system to enable booting from USB Flash drive.
- Ensure that the boot order places the USB device, or if used, a DVD drive, before the internal hard drives. Alternatively, press the Boot Menu key (Typically <F9>,<F10>, or <F11> on most systems) and directly select the USB or DVD as the boot device.
- Systems booting from NVMe media must be configured to boot using UEFI mode in the system BIOS.
- Disable the Secure Boot option. x360Recover uses dynamically generated kernel module drivers for Intel network adapters and ZFS that cannot be signed by a trusted certificate for every version of the Linux kernel that may be applied via updates.
E. Prepare to install
Once the BIOS configuration has completed:
1. Ensure that you have inserted the DVD or USB flash drive into the system.
2. After saving any settings changes, Exit the system’s BIOS.
3. Reminder: If you did not follow step D1 (above) earlier, it is advisable to shut down or power off the system and disconnect the storage drives so that the desired OS drive is the only choice for installation.
4. Reboot the machine.
F. Install the x360Recover OS
1. Booting the x360Recover OS should bring you to the following menu:
2. The initial Boot Menu defaults to the local hard drive. Choose the (Standard Install) unless you are using a single disk Nano BDR or have otherwise been instructed by support to choose an associated (Safe Mode) installer.
3. After you select an install type, the install will begin, showing the x360Recover splash screen.
G. The Axcient x360Recover installer menu
1. When the next popup windows opens, you must only select the following choice:
Installation type: Erase disk and Install Replibit
- Leave all other choices blank.
IMPORTANT NOTE: If any other operating system exists on the system, it will be reformatted and replaced with the x360Recover operating system.
2. Click Continue
3. Select the drive where you wish to install the x360Recover operating system (OS).
If the storage drives were disconnected earlier, as suggested in the Installation Prerequisites sections, the OS drive will be the only available choice. Typically, this choice will be (sda):
4. Click Install Now.
NOTE: This disk will be reformatted and erased during the installation.
5. Please wait for the install to fully complete. It may take several minutes, depending on the hardware and type of USB installation media being used.
6. When the install is complete, remove the USB drive or DVD from the system.
7. Then, click Restart Now.
NOTE: Depending on your system BIOS, if the reboot appears to be stalled, it may be necessary to press the <Space> bar once to continue.
8. When the system has finished rebooting, the login screen will be shown.
9. IMPORTANT: Copy the IP address now. You will need this IP address to log into the web interface in the next step.
Process 2. Back to Table of Contents
BDR provisioning
A. Provisioning
1. Ensure that the BDR has internet access.
Use the copied IP address (from above) to open a browser on a different machine within the same local network.
Make sure your BDR is turned on.
2. If prompted to update, click the Skip button.
3. In the Setup Wizard, read the End User License Agreement(EULA) and select the I Accept the terms in the license agreement radio button.
4. Click the I Agree button to continue.
5. Click the First machine in Cluster radio button.
6. Click the Next button to continue.
7. IMPORTANT: Before entering anything in the next screen:
Open a text editor on the local computer and write out the three (3) needed passwords.
Save the text file or passwords to your secure password repository.
Then, and only then, copy and paste each password for steps a, b and c, to configure each BDR password below:
a. The Linux Root User password, which is the operating system super-user. This user requires a highly secure password,
b. The User password, which is a limited access shell account, and
c. The Admin (Web GUI) password, used to access the Web Interface and all features
8. Click the Next button to continue.
9. Select the Appliance option to indicate this machine will serve as the BDR Appliance.
10. Click the Next button.
11.In the next steps, you will be redirected to a login page, where you will provide credentials for authentication with x360Recover.
12. Enter your x360Recover Manager (RMC) login credential and then click Continue.
13. Enter your reseller username and password.
Then enter the multi-factor authentication code and click Verify.
14. Next, click the Get Clients button.
The screen expands, prompting you to select the Client Username and Service Alias (Location) where this BDR will be deployed.
Then click the Next button.
15. Click the Customer Username drop-down menu, then select the appropriate Customer Username and Location.
16. Click the Select Time Zone drop-down menu to select the customer’s time zone.
17. Prior to clicking the Setup button to complete the configuration process. you must reboot. Be prepared, when the BDR reboots, to power off or unplug the BDR and re-attach the storage drives. Then power on the BDR to complete the reboot.
18. After completing the provisioning process, the new BDR will appear within x360Recover Manager under 'Services'', Clients' and 'Licenses'.
Note: No license is necessary to be able to perform a vault recovery.
Process 3. Back to Table of Contents
First time login and BDR setup
A. Log into the BDR Web UI
When the BDR has completed the setup and rebooted, use the IP address to access the BDR Web interface from any browser inside the local network.
1. When prompted, type admin for the Username and then type the Admin (Web GUI) password as set in Process 2-C, step 7c above.
Then, click the Login button.
B. Auto-enable Remote Assist
1. When you first log in, a popup window will prompt you to configure the Auto-Enable Remote Assist feature. We highly recommend that you select Yes so we can remotely help should you have any issues with Step D.
a. When you enable this feature, Remote Assist is automatically enabled for 7 days following an upgrade of the software.
b. Remote Assist allows Axcient Support to access your BDR devices without your intervention to perform troubleshooting tasks.
C. Add a storage pool
Unless you ordered a preconfigured BDR, you will be prompted to create the storage pool when you first log in to the device.
1. In the left hand navigation menu, click Manage Storage.
2. On the Manage Storage page, click to expand the Storage Pool section.
3. Next, select a RAID level.
For RAID-1, RAID-5 and RAID-6, one or more disks are reserved to store parity or redundancy information: |
RAID-1 requires a minimum of 2 disks. All disks in a RAID-1 set are mirror copies of the data, and the total usable data size is equivalent to the size of 1 disk. |
RAID-5 requires a minimum of 3 disks. In RAID-5, data is written across each disk in the set sequentially, with every 4th block being parity data. |
RAID-6 requires a minimum of 4 disks. In RAID-6, data is written across each disk in the set sequentially and parity data is stored using an advanced data guarding algorithm. |
4. Now, select the Compression Level
- For the purposes of a temporary BDR, the compression level for the Storage Pool should be set for ZSTD-3 mode.
- LZ mode has slightly lower compression but higher performance.
5. Expand the Unused Disks pane and select the desired disks to be used by the storage pool.
6. Click the Create Storage button to create the storage pool.
D. Configure key BDR settings
1. In the left-hand navigation menu, click the Settings tab.
2. Click the Time Settings section to confirm that your time settings are correct.
3. Click the Update Manager. For the purposes of a temporary BDR, turn off the automatic updates. The last thing you want is for the system to begin an update when you are about to start the vault recovery.
4. Do not click on or configure ANY other system settings. The “temporary” BDR setup process is now completed for the purpose of a vault recovery
Process 4. Back to Table of Contents
Incremental vault recovery
A. Summary
1. Incremental vault recovery allows you to perform the recovery of a protected system snapshot from a vault, back to a BDR, in multiple passes.
2. The first pass will copy the base snapshot of your choice to the BDR. We advise you to select the most recent (last known) clean snapshot.
3. If you have failed over to Virtual Office, backups will continue to run and new recovery points will continue to be created on the vault during this process.
4. Once the initial recovery pass is completed, you can perform additional passes to collect newly-created ‘incremental’ snapshots on the vault and copy them to the BDR.
B. Vault recovery steps
1. To begin the recovery process, click Recover from the Protected Systems page of the BDR.
2. This will open the Recover Protected System configuration dialog.
3. Enter the FQDN of your vault and click Test Connection.
4. When the connection test is successful, select the Customer Location and Protected Systems you wish to recover.
Note: Only customer locations and protected systems that belong to the same client as THIS BDR will be visible.
5. Select the desired snapshot to recover from the cloud vault. This should be the most recent and last known clean backup.
IMPORTANT: It is imperative that you read and understand the following three (3) points before moving forward.
- Do NOT check the Finalize Recovery box if you want incremental backups, from the protected system running in Virtual Office, to continue to the cloud vault during the recovery of the base snapshot to the BDR. Once the base snapshot has completed the recovery, you can run additional recovery passes and bring the base snapshot to the current state of the protected system.
- If you select the Finalize Recovery box at this time, this will be the one and only recovery pass. Further backups will be blocked, and additional recovery passes will not be possible afterwards.
- Should you decide at this time that no other backups are wanted and only one snapshot will be recovered, you can check the box and only run this one recovery pass. Choosing The Finalize Recovery box and clicking the Recover Button to begin the Final Recovery Pass ends the ‘incremental’ recovery process and will block further Direct-to-Cloud backups. For this choice skip down to Section C below. Otherwise continue to step 6.
6. Click the Recover button to begin the first pass.
IMPORTANT NOTE: Only one protected system at a time can be recovered during the process.
You cannot start a second recovery for any other protected system.
7. You may monitor progress on the Jobs -> System page for details:
8. Once the initial recover pass is completed, the protected system will be listed on the Protected Systems page.
Note: Icons in the STATUS column will indicate that backups are currently disabled on the BDR, and an incremental recovery process is in progress. Backups from Virtual Office to the vault will continue.
9. To perform additional incremental recovery passes for this protected system, navigate to the Protected Systems page and click on the system name.
This will open the Protected System Details page.
Now, click Recover.
Note that Recover is the only option available under the Actions section until the final recover pass is completed.
10. As shown below, all snapshot actions are also grayed out and unavailable until the final recover pass is completed:
11.Continue with Incremental passes until you are ready to perform the ‘Finalize Recovery’ pass.
C. Perform the Finalize Recovery pass
1. When you are ready to perform the Finalize Recovery pass:
a. Go to the cloud vault GUI >Jobs and make sure that there are no backup jobs running for the protected system.
b. Ensure that the protected system in Virtual Office is not in production use.
- Make sure that the client knows that you are ready for the final pass, and they will no longer have RDP access to the protected system.
- Edit the Virtual Office VM settings and remove the client’s RDP access. This will ensure that the client does not make any further changes to the protected system.
c. In the vault Protected System Details page, in the upper left under Actions, click the Schedule Now button. This will run a final backup. Once the backup is complete, use that last snapshot for the Finalize Recovery pass.
2. Return to the BDR and click the Recover button in the Protected System Details page.
Next, check the box for Finalize Recovery (to perform the final pass) then click Recover.
3. Once the Finalized Recovery pass has completed, all actions become available for the protected system on the BDR:
Process 5. Back to Table of Contents
Perform a Bare Metal Restore of the protected system
A. On the protected system
1. Boot the protected system using the USB with the x360Recover BMRISO, and then run GParted from the desktop.
1. Select each disk you intend to overwrite with the recovery from the top right dropdown menu.
2. You must remove any existing partitions, including locked partitions and groups on the destination disk.
a. To remove the chosen partitions, select each partition, one at a time, and click the red-circle-x icon on the toolbar.
b. To delete locked LVM partitions, first select the partition and click Partition-> Deactivate
c. To delete locked swap partitions, first select the partition and click Partition-> Swap Off
3. Then, after all desired partitions are removed, click the green check icon to save these changes.
B. Recovery Wizard
To recover a protected system from a local BDR, you'll use the Recovery Wizard in the x360Recover BMR ISO tools to select the desired options. The wizard will then make a direct iSCSI connection to the BDR, mounting the virtual disks to be copied onto the target system.
When performing a Bare Metal Restore from a BDR, the x360Recover recovery wizard will require admin (lowercase) user credentials for the BDR being used for recovery.
1. Click on Recovery Wizard to begin.
2. When the End User License Agreement (EULA) window opens, accept the EULA and select Next to choose your recovery path option.
3. From the Select Recovery Mode page, select Recover from Appliance and click Next to continue.
4. On the Replibit Server Connection page, complete the following fields:
a. IP address
b. User name - Enter admin in lowercase
c. Password (This is the admin password from Process 2-A, 7c that you were instructed to write down in a text editor)
5. From the list of protected systems available on the specified BDR, select the desired system and click Next to continue.
6. Select a snapshot to restore.
a. Select the most recent snapshot day (available days are in orange) then below, in the dropdown, choose the most recent snapshot time.
b. Leave the checkbox checked for Use Smart Recovery . The Smart Recovery tool reconstructs the disks exactly as they existed on the original protected system.
c. Once you have made your selections, click Next to continue.
7. The bare metal restore wizard will:
a. Contact the BDR
b. Start an iSCSI export of the selected protected system and snapshot
c. Mount the iSCSI volumes as locally attached disks to the BMR environment.
8. Once iSCSI creation has completed, click Next to continue.
9. When the Drive Mapping window opens, select the disks you wish to recover by clicking the appropriate check box.
You will see the physical disks present on the original protected system, along with the disk size and the volumes present on those original disks. Pic 5B-9
10. Make a selection from the Disk on local machine dropdown to specify where each protected system disk should be recovered, locally.
11. There are two available recovery modes:
a. The default Data Adaptive Mode is recommended. Recovery is performed partition by partition, using file system-specific cloning tools. Blank spaces will be skipped for efficiency and only in-use disk blocks will be copied.
b. Failsafe Mode is triggered if you de-select the default Data Adaptive Mode. Failsafe mode performs a raw disk dump of the entire drive in one pass. Failsafe mode may be slower than Data Adaptive Mode if substantial segments of the drive are empty, as these empty spaces will be copied and not skipped.
When would I choose Failsafe Mode?
If your protected system contains Windows dynamic disks, or you experience unrecoverable errors trying to use Data Adaptive Mode, you should select Failsafe mode for the recovery instead.
Depending on your data configuration, Failsafe Mode might be faster.
If your system has multiple disks, you can use Data Adaptive Mode for some and Failsafe mode for others where necessary.
Data adaptive mode
|
|
Failsafe mode
|
12. The recovery progress window will be displayed.
a. Recovery progress will continue to be displayed as each disk and volume is processed.
b. Progress, elapsed time, and estimated completion is displayed for each disk and/or volume being processed.
c. Once the recovery is completed, click Finish to close the wizard.
Process 6. Back to Table of Contents
Clean up the RMC and Virtual Office
A. RMC steps
1. In Virtual Office, shut down the virtual machines(s) that are no longer needed.This is also a good time to double check if you have other Virtual Office instances that are running in test or production mode that should not be. Charges are incurred whether in test or production mode after 30 days of use per protected system.
2. Remove the BDR license in the x360Recover Manager (RMC).
3. Put in a support request with Axcient support to discard this and any other production VMs by client name and full protected system name.
B. Appliance steps to avoid double billing
1. Factory reset the BDR, clearing all customer data.
Open a terminal as root, and then input each of the following commands, entering after each one.
flask
repenv
python resetFactory.py
2. Once the system shows up as offline on recover manager (RMC), the option to delete the appliance will be available from the (RMC) Appliances page.a. Click to change the display mode to Detail view (so that each appliance is shown on a single line.)
b. Click the appliance to expose the Actions menu then select Delete to remove the appliance.
Note: Delete is only present in the RMC when the Tunnel Status between the appliance and RMC is offline.
Congrats on a job well done! Wrap it up and head out.
SUPPORT | 720-204-4500 | 800-352-0248
- Contact Axcient Support at https://partner.axcient.com/login or call 800-352-0248
- Have you tried our Support chat for quick questions?
- Free certification courses are available in the Axcient x360Portal under Training
- Subscribe to Axcient Status page for updates and scheduled maintenance
1917