Perform a D2C/ No Local Cache restore using a temporary BDR and incremental vault recovery

IMPORTANT NOTES This is not the correct article for you IF you only need to download the last snapshot to a USB or other storage for a bare metal restore. For such cases, please see LFTP4WIN: Linux FTP client for Windows (which will also allow you to download a full snapshot from the cloud vault) This is the correct article for you if minimum downtime is required and the ETA for a virtual disk export is not feasible.

Your failover and restore scenario must meet the following three (3) requirements: 1. A bare metal restore is needed for Direct-to-Cloud protected system(s) without Local Cache enabled or when the Local Cache is not usable. 2. The protected system(s) are running in Virtual Office and backups are continuing to the cloud. (Please note that Virtual Office setup is not covered by this knowledge base article.) 3. A temporary local BDR will be used to perform a vault recovery of a chosen snapshot for use in a bare metal restore of a protected system.

CONTENTS

Process 1. OS Configuration of a temporary BDR so you can perform a vault recovery

Process 2. BDR provisioning

Process 3. First time login and BDR setup

Process 4. Incremental vault recovery  

Process 5. Perform a bare metal restore of the protected system

Process 6. Clean up 

Process 1. 

OS configuration of a temporary BDR so you can perform a vault recovery

A.    Requirements

1. Hardware must be compatible with Ubuntu Linux 20.0464-bit.
2. 16GB ECC RAM is recommended.
3. Six (6) cores; Processor must support hardware-assisted virtualization.
4. 256GB is recommended for the OS drive. NVMe is supported. Two additional storage drives large enough to hold the full snapshot are required. Note: SMR-based hard drives are not suitable as x360Recover storage devices.
5. x360Recover uses a software RAID. If your BDR has a hardware RAID controller,  it must be set to a RAID0 or JBOD configuration.

 

If you choose to create a temporary BDR as a virtual machine, click to choose one of the two types below to view the requirements. 

IMPORTANT:   Be aware that deploying x360Recover as a virtual appliance may have performance limitations when compared to deploying on bare metal.

B.      Download the ISOs and the Utility

x360Recover runs on a custom Ubuntu 64-bit Linux operating system.

The x360Recover installation ISO will deploy the basic operating system, along with all x360Recover application components.  

1. Open the ISO Terms - x360Recover (Replibit) downloads page.

At the bottom of the page, download the

• ISO to USB Utility

and the

•  Hardware Enablement HE ISO

Note on download times: It may take up to five minutes to complete the download of the installation ISO. (The typical download is approximately 1.5 GB.)  You can check your View Downloads pop-up window for status. 

2. Open the ISO Terms - x360Recover Bare Metal Restore downloads page

Scroll to the bottom of the page. 

Click to agree to the terms to start the download for the Standard Bare Metal Restore ISO.

 

C.      Create a bootable USB flash drive

1. Once you have downloaded the x360Recover Installation ISO and the Bare Metal Restore ISO, you must make individual bootable images on separate USB flash drives.

• If you prefer, an additional option is to burn the ISO’s onto separate DVD’s, (if the systems have a DVD reader.)
• The base operating system installation from USB or DVD typically takes about ten minutes. 
• We recommend that you use our utility, as other ISO to USB utilities may not work as successfully with Axcient’s BMR process.

2. In your Downloads folder, right click the isotousb_setup.exe and run as administrator.

3. Your device should be automatically detected and selected.

4. If your device is not automatically detected and selected, choose the correct device from the USB device drop-down list.

5. Click Browse and select the x360Recover ISO image. Ensure that File system is set to FAT32 as the default or select FAT32 from the drop-down list.

6. Enter a New Volume label if desired. 

7. Click Burn to begin the file copy process

 

D.    Installation prerequisites

1. For multi-disk systems, it is advisable to disconnect the storage drives so that the desired OS drive is the only choice for installation.

2. To ensure that the installation can be completed successfully, the system should be connected to a network with DHCP and open internet access.

3. Power on the system and press the appropriate key to enter the BIOS setup menu. (Typically, this is <Del>, <F1> or <F2> on most white-box systems, or <F9> on HP servers) 

1. x360Recover and the Linux operating system do not support booting from motherboard (fake) RAID controllers. (BIOS has an option to select RAID for the onboard SATA ports.) This onboard ‘RAID’ functionality is really a software-based RAID that requires Windows-only drivers that are not available on Linux. If your motherboard has such an option, ensure that the SATA ports are configured in SATA or AHCI mode for compatibility with Linux. Consult your system motherboard documentation for instructions on selecting between RAID and SATA/AHCI boot modes. If you are unable to do such, then set to JBOD and each drive as aRAID0.
   
   
2. When booting from a USB device, it may be necessary to change the default boot order of the system.
   
   
3. Locate the boot order management page within the BIOS and configure the system to enable booting from USB Flash drive.
   
   
4. Ensure that the boot order places the USB device, or if used, a DVD drive, before the internal hard drives. Alternatively, press the Boot Menu key (Typically <F9>,<F10>, or <F11> on most systems) and directly select the USB or DVD as the boot device.
5. Systems booting from NVMe media must be configured to boot using UEFI mode in the system BIOS.
   
   
6. Disable the Secure Boot option. x360Recover uses dynamically generated kernel module drivers for Intel network adapters and ZFS that cannot be signed by a trusted certificate for every version of the Linux kernel that may be applied via updates.

E.     Prepare to install

Once the BIOS configuration has completed:

1. Ensure that you have inserted the DVD or USB flash drive into the system.

2. After saving any settings changes, Exit the system’s BIOS.

3. Reminder: If you did not follow step D1 (above) earlier, it is advisable to shut down or power off the system and disconnect the storage drives so that the desired OS drive is the only choice for installation.

4. Reboot the machine.

F.     Install the x360Recover OS

1.  Booting the x360Recover OS should bring you to the following menu:

2. The initial Boot Menu defaults to the local hard drive. Use the cursor keys to scroll up and down to select one of the available installation options:

3. After you select an install type, the install will begin, showing the x360Recover splash screen.

 

 

G.    TheAxcient x360Recover installer menu

1.      In the next popup, only select installation type: Erase Disk and Install Replibit **Leaveall other choices blank. Pic 1G-1

** IMPORTANT NOTE: If any otheroperating system exists on the system, it will be reformatted and replacedwith the x360Recover operating system.

2.      Click Continue

 

 

3.      Select the drive to install thex360Recover operating system (OS). If the storage drives were disconnectedearlier, as suggested in the InstallationPrerequisites, theOS drive will be the only available choice. Typically, this choice will be (sda): Pic1G-3

4.      Click Install Now.  **NOTE: This disk will be reformatted anderased during the installation.

 

5.      Please wait for the install to fullycomplete. It may take several minutes, depending on the hardware and type ofUSB installation media being used. Pic 1G-5

 

 

6.      When the install is complete, remove theUSB drive or DVD from the system.

7.     Then, clickRestart Now. Pic 1G-7

 

 NOTE: Depending on yoursystem BIOS, if the reboot appears to be stalled, it may be necessary to pressthe <Space> bar once to continue.

8.      When the system has finished rebooting,the below login screen will be shown. Pic1G-8

9.      *Copy the IP address now. Youwill need the IP address to log into the web interface in the next step,Process 3 A. BDR setup andfirst-time login.

 

 

 

 

Process 2 . BDR Setupand first-time login

A.      First Time Login

1.      Ensure that the BDR hasinternet access. Use the copied IP address from above to open a browser on adifferent machine within the same local network. Make sure your BDR is turnedon.

2.      If prompted to update, clickthe Skip button.

3.      In the Setup Wizard, read the End User License Agreement(EULA) and select the I Accept the terms in the license agreement radio button.

4.      Click the I Agree button to continue.

5.      Click the First machine in Cluster radio button.

6.      Click the Next button to continue.

7.     Beforeentering anything in the next screen. Open a text editor on the local computerand write out the below 3 needed passwords. Save the text file or passwords toyour secure password repository. Then, and only then, copy and paste eachpassword for steps a, b and c, to configure each BDR password below. Pic 2A-7                                                   

a.     The Linux RootUser password,which is the operating system super-user; this user requires a highly securepassword,

b.     The User password, whichis a limited access shell account, and

c.      The Admin (Web GUI) password,which is used to access the Web Interface and all features

8.      Click the Next button to continue.

 

 
 

 

9.      Select the Appliance option to indicatethis machine will serve as the BDR Appliance. Pic 2A-9

10.     Click the Next button.

 

 
 

11.    In thenext steps, you will be redirected to a login page, where you will providecredentials for authentication with x360Recover. Pic 2A-8

 

 
 

12.    Enteryour x360Recover Manager (RMC) login credential and then click Continue. Pic 2A-12

 

 
 

13.     Enter your reseller username and password and then the multi-factorauthentication code and click Verify.

14.     Then click the Get Clients button. The screenexpands, prompting you to select the Client and Service Alias (Location) wherethis BDR will be deployed. Then click the next button. Pic 2A-14

15.    Clickthe Customer Username drop-down menu, selectthe appropriate Customer Username and Location.

 

 
 

 

16.     Click the Select Time Zone drop-down menu toselect the Customer’s time zone. Pic 2A-16

17.    Prior toclicking the Setup button to complete the configuration process and reboot. Beprepared, when the BDR reboots, to power off or unplug the BDR and re-attachthe storage drives. Then power on the BDR to complete the reboot.

18.     After completing the provisioningprocess, the new BDR will appear within x360Recover Manager under 'Services''Clients' and 'Licenses'.

 

 
 

 

 

 

Process3. Add a BDR license endpoint

A.    License Endpoint

1.     Loginto partner.axcient.com, Pic 3A-1

2.      Click Launch on the x360Recovertile

 

1.      

2.      

3.     Whenthe Dashboard opens, click Licenses near the bottom of the left menu, Pic 3A-3 

4.      Locate the new BDR. 

5.      Hover over the new BDR andselect the down arrow on the right side of the screen. 

6.      Select Edit Licenses.

 

1.      

2.      

3.      

4.      

5.      

6.      

7.     Clickthe plus sign to add additional server or workstation licenses, Pic 3A-7 

8.     Click Save

 

 

 

 

 

Process4. First time login and BDR Setup

A.    Log into the BDR Web UI 

Whenthe BDR has completed the setup and rebooted, again use the IP address toaccess the BDR Web interface from any browser inside the local network.

1.When prompted, type admin for the Username and the admin Password as set in Process 3-C,step 7c above and then click the Login button. Pic 4A-1

 

 

B.    Auto-enable Remote Assist 

1.      When you first log in, a pop-up window will prompt you to configurethe Auto-Enable Remote Assist feature. We highly recommend selecting Yes so wecan remotely help should you have any issues with Step D.

a.     When you enable thisfeature, Remote Assist is automatically enabled for 7 days following an upgradeof the software.

b.     Remote Assist allowsAxcient Support to access your BDR devices without your intervention to performtroubleshooting tasks.

c.     Axcient monitors all BDRdevices through Telemetry and attempts to proactively correct issues that mightoccur during updates.

 

 

C.     Add a storage pool 

Unlessyou ordered a preconfigured BDR, you will be prompted to create the storagepool when you first log in to the device.

1.      In the left-hand navigationmenu, click Manage Storage. Pic 4C-1

2.      In the Manage Storage page, click to expandthe Storage Pool section.

 

 

3.     Next,select a RAID level. Pic 4C-3

For RAID-1, RAID-5 and RAID-6, one or more disks are reserved to store parity or redundancy information.

RAID-1 requires a minimum of 2 disks. All disks in a RAID-1 set are mirror copies of the data, and the total usable data size is equivalent to the size of 1 disk.

RAID-5 requires a minimum of 3 disks. In RAID-5, data is written across each disk in the set sequentially, with every 4th block being parity data.

RAID-6 requires a minimum of 4 disks. In RAID-6, data is written across each disk in the set sequentially and parity data is stored using an advanced data guarding algorithm.

 

4.     Selectthe Compression level - For the purposes of a temporary BDR Compression Level for theStorage Pool should be set for ZSTD-3 mode. LZ mode has slightly lowercompression but higher performance.

 

 
 

5.      Expand the Unused Disks paneand select the desired disks to be used by the storage pool. Pic 4C-5

6.     Clickthe Create Storage button to create thestorage pool.

 

 
 

 

 

D.  Configure key BDR settings

1.      In the left-hand navigationmenu, click the Settings tab.

2.      In the Settings page, click to expand the License Details section to make sure thereis a license available for use. One endpoint license is required for theBDR to pull the snapshot from the vault to the BDR 

3.      If no licenses are displayed, return tothe License Details page and click the Sync button to force the BDR to resyncwith x360Recover Manager. Pic 3D-3

 

4.     Clickthe TimeSettings section to confirm that your time settings are correct.

5.     Clickthe Update Manager, for the purposes of atemporary BDR, turn off automatic updates. The last thing you want is for thesystem to begin an update when you are about to start the vault recovery.

8.     DoNotclick on or configure Any other system settings. The “temporary” BDRsetup process is now completed for the purpose of a vault recovery.

 

 

 
 

Process 4. Incremental Vault Recovery  

A.   Summary

1.      Incremental vaultrecovery allows you to perform the recovery ofprotected system snapshot from a vault back to a BDR in multiple passes.

2.      The first pass will copy the base snapshotof your choice to the BDR. We advise you to select the most recent, last knownclean snapshot.

3.      If you have failed over to VirtualOffice. Backups will continue to run and new recovery points will continue tobe created on the vault during this process. 

4.      Once the initial recovery pass iscompleted, you can perform additional  passes to collect newly-created‘incremental’ snapshots on the vault and copy them to the BDR.  

 

 

B.    Vault Recovery Steps

1.     To begin the recovery process,click Recover from the Protected Systems page of the BDR. Pic 4B-1

 

 
 

2.      
This will open the ‘RecoverProtected System’ configuration dialog.

3.      Enter the FQDN ofyour vault and click TestConnection. Pic 4B-3

4.      When the connection test is successful,select the CustomerLocation and Protected Systems you wish to recover. Pic 4B-4
Note:  Only customer locations and protectedsystems that belong to the same client as THIS BDR will be visible. If noprotected systems belonging to this client exist on the vault (that are notalready present on the BDR) then the selection list will be empty.

5.      Select the desired snapshot to recoverfrom the cloud vault.This should be the most recent and last known clean backup. Do Not check the Finalize Recovery box.

IMPORTANT: By default, the Finalize Recovery check box will be unselected to performan incremental recovery. This allows for incremental backups from the protectedsystem running in Virtual Office to continue to the cloud vault during therecovery of the base snapshot to the BDR. Once the base snapshot has completedthe recovery, additional recovery passes can then run and bring the basesnapshot to the current state of the protected system.

** Ifyou select the Finalize Recovery box At This Time, this will bethe One and Only recovery pass. No further recoverypasses will be possible afterwards.**

6.      Click the Recover button to begin the first pass.  **Note:Only one protected system at a time can be recovered during theprocess. You cannot start a second recovery for any otherprotected system. 

 

 

 

7.       You may monitor progress on the Jobs -> System page for details: Pic 4B-7

 

 
 

8.      Once the initial recover pass iscompleted, the protected system will be listed on the Protected Systems page. Pic4B-8

Note: Icons in the STATUS column will indicate that backups arecurrently disabled on the BDR, and an incremental recoveryprocess is in progress. Backups fromVirtual Office to the vault will continue. 

 

9.      To perform additional incrementalrecovery passes for this protected system, navigate to the Protected Systems page and click on the system name. Thiswill open the ProtectedSystem Details page.Now, click Recover.Pic4B-9

Notethat Recoveris the only option available under the Actions sectionuntil the final recover pass is completed. 

 

 
 

10.    As shown below, all snapshotactions are also grayed out and unavailable until the final recover pass iscompleted: Pic 4B-10

 

 
 

11.    Continue with Incremental passes untilyou are ready to perform the ‘Finalize Recovery’ pass.

12.  Whenyou are ready to perform the ‘Finalize Recovery’ pass:

a.    **Go to the cloud vault GUI >Jobs**and make sure that there are no backup jobs running for the protected system.

b.    **Ensure that the protected system in Virtual Office** is not in production use.

c.     ChoosingThe ‘FinalizeRecovery’ check box and clicking the Recover Button to begin the Final RecoveryPass, ends the ‘incremental’ recoveryprocess and will block further Direct-to-Cloud backups.

 

 

C.   Configure the Backup Agent for BDR Mode  

1.      Stop the following services on theprotected system in Virtual Office:

a.      Replibit agent service

b.      Replibit agent update service 

c.      Replibitmanagement service

2.     In the Windows search bar typeNotepad. Right click on Notepad and open as administrator. If you don’t open asan administrator, the changes in step 3 will not persist after saving. Pic 4C-2

 

 

3.     Open thearistos.cfg file in the Replibit installation directory. Typically, this fileis found at c:\program files (x86)\replibit.

4.     Change the field ‘SERVER=<vault FQDN >’ to SERVER=<IPaddress of BDR> .

5.     Save the file.

6.     Deletethe file Permid.cfg.

7.     Deleting thisfile will convert the agent back to BDR mode.

8.     Return to the BDRand click the Recover button in the Protected System Details page and check the box for Finalize Recovery (to perform the final pass) thenclick Recover. Pic 4C-8

 

 

9.     Once the Finalized Recovery passhas completed, all actions become available for the protected system on theBDR:

 

 

A.     On the protected system

Boot the protected system using the USB with the x360Recover BMRISO, and run GParted from the desktop. Pic 5A-1

 

 

1.     Select each disk you intend tooverwrite with the recovery from the top right dropdown menu.

2.     You must remove any existingpartitions, including locked partitions and groups on the destination disk.

a.      To remove the chosen partitions, select eachpartition, one at a time, and click the red-circle-x icon on the toolbar.  

b.     To delete locked LVM partitions,first select the partition and click Partition-> Deactivate

c.      To delete locked swap partitions,first select the partition and click Partition-> SwapOff 

3.      Then, after all desired partitions are removed, click the green check icon tosave these changes. Pic 5A-3

 

 

 

 

B.    RecoveryWizard

To recover a protected system from a local BDR, you'll use the Recovery Wizardin the x360Recover BMRISO toolsto select the desired options. The wizard will then make a direct iSCSIconnection to the BDR, mounting the virtual disks to be copied onto the targetsystem.  

When performing a Bare MetalRestore from an BDR, the x360Recover recovery wizard will require admin (lowercase) user credentials for the BDR being used for recovery.

 

1.      Click on RecoveryWizard to begin.

2.     When the EndUser License Agreement (EULA) window opens, accept the EULA and select Next to choose your recoverypath option.

3.     From the Select Recovery Mode page, select Recover from BDR and click Next to continue. Pic 5B-3

 

 

4.      Onthe Replibit Server Connection page, complete the following fields:

 

a.      IP address

b.     Username (admin)

c.      Password (thisis the admin password from Process 2-A, 7c that you were instructed to writedown in a text editor).Pic 5B-4

 

 

 

 

5.     From the list of protected systemsavailable on the specified BDR, select the desired system and click Next to continue. Pic 5B-5

 

 

6.     Select a snapshot torestore. Pic 5B-6

 

a.     Days with snapshotsavailable are backlit in light orange. Select a day, then select a specificrecovery point in the dropdown selector.

b.     Leave the checkboxchecked for Use SmartRecovery . The SmartRecovery tool reconstructs the disks exactly as theyexisted on the original protected system.

c.      Once you have madeyour selections, click Next to continue.

 

 

7.      The bare metal restorewizard will: Pic 5B-7

a.      Contact the BDR

b.      Start an iSCSI export of the selectedprotected system and snapshot

c.       Mount the iSCSI volumes as locallyattached disks to the BMR environment.  

 

8.     Once iSCSI creationhas completed, click Next to continue.

 

 

 

9.     When the Drive Mapping window opens, select which disks you wish to recover byclicking the appropriate check box. You will see the physical disks present on theoriginal protected system, along with the disk size and the volumes present onthose original disks. Pic 5B-9

 

 

 

10.    Makea selection from the Diskon local machine dropdown to specify where each protectedsystem disk should be recovered, locally. Pic 5B-10

 

 

 

11.    There are two available recovery modes: Pic 5B-11

a.     The default Data Adaptive Mode is the (recommended)recovery mode. Recovery is performed partition by partition, usingfilesystem-specific cloning tools. Blank spaces will be skipped for efficiencyand only in-use disk blocks will be copied.

b.     FailsafeModeis triggered if you de-select the default Data Adaptive Mode. Failsafe modeperforms a raw disk dump of the entire drive in one pass.   Failsafe modemay be slower than Data Adaptive Mode if substantial segments of the drive areempty, as these empty spaces will be copied and not skipped.

When would I choose Failsafe Mode?

If your protected system contains Windows dynamic disks, or you experience unrecoverable errors trying to use Data Adaptive Mode, you should select Failsafe mode for the recovery instead.

Depending on your data configuration, Failsafe Mode might be faster.

If your system has multiple disks, youcan use Data Adaptive Mode for some and Failsafe mode for others wherenecessary.

Data adaptive mode Copies each partition, one by one, using filesystem-specific copy tools in the most efficient manner.  Blank spaces are skipped  Only in-use disk blocks are copied

Failsafe mode  Copies the entire disk in a single pass, regardless of file system types.  Provides maximum reliability May be slower when the disk volume contains large, empty spaces. Avoids any file system-related errors

 

 

 

 

c.       

 

Once you have completed your drive mapping selections, click Next to continue.

 

 

 

12.  The recovery progress window will bedisplayed. Pic 5B-12

a.     Recoveryprogress will continue to be displayed as each disk and volume is processed. 

b.     Progress,elapsed time, and estimated completion is displayed for each disk and/or volumebeing processed.  

c.      Oncethe recovery is completed, click Finish to close the wizard.

 

 

 

 

 

Process 6. Convert the backup agent back toDirect-to-Cloud mode

A.     Perform the following steps

1.     Verify that nobackup is currently running on the appliance.

2.     Uninstall the agent from the protected system using Windows Add or Remove Programs.

**Important: Do NOT delete the agent installation directory,default is C:\Program Files (x86)\Replibit, or aristos.cfg file**

3.      On the cloud vault, under the Clients tab, to the right of the customer for this protected system, Download the Direct-to-Cloud Windows agent. Pic 6A-4

Important: The Direct-to-Cloud agent installation file contains a specially formatted name, including an installation token that identifies which client the agent is being installedfor. Do not rename the agent installation filew hen downloading it from the vault.

 

 

 

4.      From an administratorlogin on the protected system, run the downloaded Direct-to-Cloud agent installeron the protected system. Because the Replibit directory folder still exist, theagent installer will not present the parameter configuration screen duringinstallation. The configuration still remains in the Aristos.cfg file.

5.      Verify that the protectedsystem is registered in the Axcient Cloud: Pic 6A-5

A.     Openthe ProtectedSystems pageon the vault and review the Location column.

B.     If theprotected system is on Direct-to-Cloud, it will indicate Axcient Cloud under Location

Additionally, after an end point has been converted to D2C, it will display a Schedule section:

 

 

6.      Verify that the nextbackup executed is a full backup. (If this is not the case, click Schedule Now and select Full Backup from theProtected Systems Details page.)

 

 

 

Process7. Clean upVirtual Office

A.     RMC steps

1.     In the RMC and in Virtual Office shut down the virtual machines(s) thatare no longer needed. This is also a good time to double check if you haveother Virtual Office instances that are running in test or production mode thatshould not be. Charges are incurred whether in test or production mode after 30days of use per protected system.

2.      Remove the BDR license in thex360Recover Manager (RMC).

3.     Put in a ticket withAxcient support to discard the production VM notating the client and protectedsystem.

B.    Appliance steps to avoid double billing

1.      Factory reset the BDR,clearing all customer data. Open a terminal as root, input each of thefollowing commands, entering after each one. 

flask 

repenv 

pythonresetFactory.py 

Congrats on a job well done! Wrap it up andhead out.