What is x360Recover AutoVerify?  

• AutoVerify data sheet

AutoVerify is an enhancement to x360Recover's nightly BootVM check feature which provides validation of the recoverability of backup snapshots. If enabled, AutoVerify runs solely on local BDR appliances.

• AutoVerify is now part of the BootVM check operation on all x360Recover appliances and is reported within x360Recover Manager and the Global Management Portal (GMP) on v8.3.0 or greater.  
• When nightly BootVM checks are enabled for each protected system on the local x360Recover appliance, each selected protected system will automatically boot the latest backup snapshot and perform AutoVerify operations to ensure that the recovered backup is healthy. 
• By design, BootVM checks capture a screenshot of the protected system for manual review after it has successfully booted.  
• AutoVerify extends the BootVM check functionality by automatically performing additional operations to ensure the recoverability of the snapshot. If the AutoVerify operations identify any data consistency issues within the snapshot , the operation will fail and the appliance will be notified. 

Under specific conditions, AutoVerify will raise an alert on the Alerts tab and attempt to ‘self-heal’ backup consistency issues by requesting a new full backup to synchronize the protected system with the appliance.  (Note: Self-healing full backups are disabled in the initial release.)

AutoVerify also provides details of protected system health status to the Axcient support team via telemetry, to enable us to proactively monitor and respond to support requests. 

• Interested in Cloud AutoVerify for Axcient-hosted vaults with Direct-to-Cloud endpoints? Learn about Cloud AutoVerify here:

AutoVerify operations

As of x360Recover v8.3.0, BootVM AutoVerify performs the following operations to validate each protected system snapshot status:

Heartbeat

The appliance verifies basic communications with the running virtual machine, and if not established, fails the check. This operation confirms that the system was able to fully boot the operating system, start services, and complete plug-n-play device discovery

Check Disk (chkdsk)

A Windows ‘chkdsk’ operation (using a Windows utility that checks the integrity of a file system) is performed on each volume within the protected system.

• ‘chkdsk’ may fail due to the following conditions:
  • Corrupted or incomplete backup snapshot on the BDR appliance.
    (A new full backup will likely resolve the issue)
  • File system corruption on the source protected system disk(s).
    (‘chkdsk’ should be run on the protected system to fix the disk problem)
• If chkdsk exits with specific errors codes that indicate possible data consistency issues, the appliance attempts to resolve the issue by performing a new full scan of the protected system and an alert is generated on the appliance.  Alerts generated by AutoVerify are automatically closed after the full scan completes. 
• Note: Full scans generated by AutoVerify will occur no more than once every 30 day

Note: ‘Self-heal’ full backups are disabled in the initial release

BootVM checks automatically allocate RAM dynamically to the virtual machine during testing.  One quarter of available RAM is allocated to the VM, with a minimum of 2GB and a maximum of 12GB, in order to ensure sufficient resources to complete the AutoVerify operations.

BootVM checks end automatically.  The appliance monitors the status of the AutoVerify operations and the virtual machine will be shut down as soon as all checks are complete.  Note: Maximum virtual machine run time is 90 minutes, and the AutoVerify operation will fail if not completed by then.

AutoVerify results

AutoVerify results, including complete heartbeat and ‘chkdsk’ details, are available on the protected systems Details page within the appliance.

Clicking on the AutoVerify results provides BootVM AutoVerify complete details on all operations:

AutoVerify Global Monitoring

In addition to appliance-level alerting, AutoVerify has been integrated into x360 Recover Manager reporting. This means you can easily see AutoVerify operations across all of your appliances and protected systems.
 
A summary of AutoVerify operations exists in two places: 

1. On the Devices page:

2. On the Backup summary report:

Benefits of AutoVerify

• The enhanced backup validation provided by AutoVerify operations allows for a high level of confidence in the integrity and recoverability of your backups. This will reduce the time spent manually fixing issues that might occur during a recovery fail over.
• Being alerted to possible issues with protected systems before they fail ensures that you have good backups and allows you to repair issues with bad data on the source system.
• You can lower the cost of ownership by automatically detecting, alerting and fixing backup consistency issues. Instead of manually reviewing BootVM screenshots on a daily, weekly or monthly basis, rely upon new automated validation and alerting.
• Reduce the time spent reviewing BootVM checks and provide faster identification of possible backup issues.
  • Prior to AutoVerify, BootVM checks ran for a fixed 10-minute interval and then captured a screenshot before moving on to the next system. This could lead to either wasted time spent waiting after the VM is up, or in some cases, grabbing a screenshot too soon and returning an inconclusive check.  AutoVerify proactively monitors the progress of the checks and will not exit while checks are running and the VM is responding to monitoring.  (A maximum of 90 minutes is allowed for all AutoVerify tests to complete.)  Once all checks are passed, AutoVerify will immediately exit and begin testing the next protected system.  This provides a much more efficient boot check process and returns definitive results as to the status of the protected pystem snapshot in most cases.
  • More memory may be used during Boot VM checks. Prior versions used a fixed 2GB of RAM for running checks.  BootVM checks now dynamically allocate 1/4 of available RAM (with a minimum of 2GB and a maximum of 12GB) to the virtual machine to ensure there are sufficient resources to complete all tests.

Troubleshooting AutoVerify

For directions on how to troubleshoot AutoVerify, please refer to the AutoVerify Troubleshooting Guide. 

 SUPPORT  | 720-204-4500 | 800-352-0248

• To learn more about any of our Axcient products,  sign up for free one-on-one training.
• Please contact your Partner Success Manager or Support if you have specific technical questions.
• Subscribe to the Axcient Status page for a list of status updates and scheduled maintenance.

 815 | 1041  |  1161  |  1407