Overview 

This purpose of this troubleshooting guide is to provide you with techniques, best practices, and “quick fixes” to resolve the most frequently encountered x360Recover questions and/or issues.  

This guide is not intended to be a comprehensive, step-by-step manual, but rather a troubleshooting resource for common x360Recover functions. For the full x360Recover User manual on how to install, deploy, and manage x360Recover’s Appliance software (including screenshots), please access the x360Recover Administration Guide.

The article will cover the following topics:

• How to download your initial x360Recover ISO Software
• How to troubleshoot failed backup jobs using the x360Recover Backup Analysis Tool
• How to troubleshoot x360Recover VSS issues 
• How to troubleshoot your CPU usage
• Appliance migration process
• How to replace a failed disk in the Backup Storage Pool

How to download your initial x360Recover ISO Software

To install x360Recover, you will need to download the x360Recover ISO file and burn the ISO to a bootable flash drive.

1. Before you begin your x360Recover software download, note that x360Recover may automatically start the installation without any prompts when using certain hardware; therefore, please use a machine that you do not need to preserve any data on, as x360Recover may delete your computer’s contents during the install process.

2. Download the latest version of the x360Recover using the Software downloads page.

Note: This downloading process may take up to five minutes (the typical download is approximately 1.5 GB); so please check your “View Downloads” – internet explorer pop-up window for a status if needed.

Troubleshoot common x360Recover installation functions

How to troubleshoot failed backup jobs using the x360Recover Backup Analysis Tool

Once you have installed x360Recover, we recommend that you check your backups.

Test your backups by:

• mounting,
• booting as a VM,
• exporting as a virtual disk, and as a Bare Metal Restore. 

if you experience any failed backup jobs, please troubleshoot using the following procedures:

1. Download the latest copy of the x360Recover Analysis Tool from this KB article:

The x360Recover Analysis Tool will analyze the health of a protected system, capture logs, and creates a text file with a synopsis of any issues it finds. 

 Notes

• You will need Administrator Privileges to run the x360Recover Analysis Tool.
• You will need to run the tool on the protected system from an elevated command prompt after stopping the x360Recover service.
• The tool will create a .zip file in the C:\x360RecoverAnalysisTool\x360Recover folder along with all other files.
• Please send Axcient only the .zip file when directed.
• When you run the tool, please click the Enter button when prompted to keep the data local.
• After the tool is finished, you can open up the text file it created (problems.txt file) and start troubleshooting the items it detected

Some of the x360Recover Analysis Tool integrity tests include: 

• Ensuring that x360Recover can traverse the directory structure,
• Numerating out the directories,
• Reading-down the Master File Table and looking for orphan objects,
• Checking for a bad pointer within the Master File Table,
• Checking for high fragmentation levels (which affects virtualization from Windows),
• Checking for any DLL mismatches,
• Running a real-time tracking mechanism that creates a hash file,
• Running a full differential in the last backup day (automatically) to compare every block in the backup set to that of the production set,
• Real-time resilvering of data,
• Tracking Window Boot DLL files’ expectation of virtualization (proper modifications to Boot DLL files greatly impacts stability and performance on Virtualization).

If any piece of the data set exceeds the tolerances, x360Recover will proactively fail the backup so you can troubleshoot.

How to Troubleshoot x360Recover VSS issues

When a x360Recover backup fails, the problem is almost always caused by the failure of the Microsoft VSS provider, or one of the application-specific VSS writers, and this article above will help you to diagnose these failures.

Typically, the top 10 reasons for backup failures associated with VSS include:

1. Snapshot creation failed due to Windows VSS failure.
2. There is not enough Shadow Copy storage space to create a backup.
3. A VSS writer is in a failed state.
4. A volume does not pass a CheckDisk (chkdsk).
5. The Master File Table is highly fragmented.
6. A volume is extremely fragmented.
7. Another application interrupted the snapshot creation.
8. An uninstalled backup application left behind its VSS provider.
9. The OS does not pass a System File Checker (SFC) scan.
10. A Windows Server Backup disk is online.

How to Troubleshoot your CPU usage

If you are experiencing performance issues, please follow these steps:

1. Look at the system resources.

2. CPU and RAM usage are easy to monitor by opening Task Manager.

Additionally, the disk sub-system can greatly affect VSS performance.

3. Monitor the CPU before and after the backup to set a baseline and to observe the change.

4. If the system is running an old or low-end CPU, you may already have a high CPU usage baseline, and running another service will exceed your CPU’s capacity.

5. If this is the case, please adjust the ‘Worker Threads’ or set the ‘Affinity’ to increase the backup duration using the following Knowledgebase article; the x360Recover agent defaults to backup all volumes and uses 4 worker threads per processor core. The KB article below demonstrates how to specify which volumes are backups and how many worker threads are utilized.

Modify the x360Recover agent: Set volumes and worker threads - aristos.cfg

Notes: Lowering the amount of worker threads will reduce the speed of the backup. If the backups start overlapping, you may need to increase the time between snapshots

As with the CPU, monitor the RAM and pagefile usage prior to, and during, a backup.

1. 1. If the protected system is already using all of the available RAM, the VSS service will end up using the slower pagefile.
   2. Adding more RAM is the best option but you can also look at reducing the amount of RAM other services are using.
   3. Exchange and SQL by default, will use all available RAM.
   4. A quick web search will guide you on setting the maximum RAM that different services can use.
   5. Run chkdsk and then defrag all the partitions being backed up.
   6. Remember that VSS needs free disk space to create the ShadowCopies.

If changing worker threads to as low as [1] still does not work, you can try changing the affinity on aristosagent to lower CPU usage while a backup is running by executing these two commands from a raised CMD or .BAT file:

wmic process where name="AristosAgent.exe" CALL setpriority "below normal" PowerShell "$Process = Get-Process AristosAgent; $Process.ProcessorAffinity=1"

NOTE: You can create a .BAT file with the two commands, then create a recurring scheduled task to run the .BAT file during or after a scheduled backup kicks off. In this way, you will know that AristosAgent.exe will be running.

Appliance migration process

1. Stop and disable the x360Recover service on the protected system.
2. Disable replication of the protected system on the original appliance.
3. Add 20 days to the Retention Policy on both the appliance and the vault so data will not be deleted during this process.
4. Migrate the protected system via USB or Network in the Details page of that protected system on the appliance.
5. Once the migration is complete, verify on the new Appliance that the data is there and is recoverable. Additionally, boot in Test 
6. Change the IP address of the appliance in cfg located in c:\program files (x86)\x360Recover to the new appliance's IP.
7. Enable the x360Recover service on the protected system.
8. After several backups have run, repeat the verification process by ensuring that the data is present and is recoverable.
9. Enable Replication on the new Appliance to the vault.
10. After the replication is back in sync, remove the 20 days on the retention policy.

How to Replace a Failed Disk in the Backup Storage Pool

If your pool has lost a disk but the RAID is still intact (you have not lost two disks in a RAID5 or three disks in a RAID6), your pool will be in a degraded state.

1. Use the Storage Pool drop-down to find the Failed Disk(s) In this section you should be able to see your failed disk. 

2. After physically replacing the failed disk:

3. Recreate the disk as a RAID0 (if the RAID controller is not passing the disks as JBOD) with a disk of equal or greater size. In this scenario you will need to reboot.

4. Once complete, use the Storage Pool drop-down to find the Unused Disk(s) In this section, you should find an unused disk.

5. Return to the Failed Disk(s) section, and click Replace next to the failed disk. Select the correct Unused Disk and click

6. You will see a popup notifying you that the disk is being replaced.

7. While the disk is being replaced, you will see the resilvering status under Storage Status.

8. When the resilvering process is complete, the pool will return to an Online Status.

SUPPORT | 720-204-4500 | 800-352-0248

• Contact Axcient Support at https://partner.axcient.com/login or call 800-352-0248
• Free certification courses are available in the Axcient x360Portal under Training
• To learn more about Axcient products, sign up for a free one-on-one training
• Subscribe to the Axcient Status page for updates and scheduled maintenance

1497