Health status for protected systems - x360Recover

x360Recover and x360Recover Direct-to-Cloud (D2C)

Written By Tami Sutcliffe (Super Administrator)

Updated at October 19th, 2022

Overview 

This document explains how health status is calculated for x360Recover protected systems.

Which products does this apply to?

  • x360Recover D2C
  • x360Recover 

Direct-to-Cloud (D2C) health status


The health status of the vault does not depend on the health status of the D2C devices backed up on the vault.

Rather, the D2C protected systems calculate health based on the protected system health and the jobs running against it.  

There are currently four D2C protected system statuses:

D2C protected systems HEALTH STATUS

1. Parked

  • Device replication job is disabled

2. Normal

  • Device replication job has normal status (or job doesn't exist)
  • It's a newly added device and initial backup window is not opened yet

3. Warned

Device replication job has a warned status (disabled job is ignored)

4. Troubled

  • Device replication job has troubled status (disabled job is ignored)
  • If local cache is enabled for the machine and last successful check (having "OK" status) is older than the SLA threshold + 3 hours or not present at all.
Delete

As jobs run against the protected system (such as replication jobs), each job will have a status. There are three possible job statuses:

D2C jobs HEALTH STATUS

1. Normal

  • All cases that aren't related to troubled status
  • The health status of the job which has disabled thresholds should always be healthy

2. Warned

  • A replication job exists but has no RPs and the later of (a) the job creation time or (b) the start time of the first replication window is inside of threshold value.

3. Troubled

  • Replication job is out of threshold
  • Replication job exists, but has no RP and the later of (a) the job creation time or (b) the start time of the first replication window is out of threshold value
Delete

What alerts are driven on the status above?

 
Reason Status
OFFSITE_STATUS TROUBLED
OFFSITE_NEVER_RUN WARNED
None NORMAL

NOTES: 

  • If a job is disabled, the health status digest will be skipped. The health status before the job was disabled is retained.
  • Disabled jobs are ignored at the device level of the health status digest

x360Recover health status


For x360Recover, the health status calculations are available for four areas.

1. Appliances

2. Protected systems

3. Jobs

4. Vaults

Each area has it's own set of health statuses since they can potentially fail for each.  Let’s go over each of the health statuses for each area.


1. x360Recover appliance health status

There are currently four types of appliance health status settings:

Four types of APPLIANCE health status

1. Unprotected 

  • Appliance is registered but has no devices

2. Normal

  • All replication jobs statues have normal health status
  • Local backup job can be failed
  • Local backup job can be out of threshold

3. Warned

  • RMC did not receive the heartbeat from appliance for more than the amount of time specified by the network connectivity threshold
  • If latest replication job (of any device) failed (disabled jobs are ignored)
  • If at least one device is out of cloud/vault protection threshold (disabled jobs are ignored)
  • If the appliance is running an older version of software
  • If local storage is used for more than 80%, but less than 90%.
  • If least one device is troubled due to Local Cache SLA violation

4. Troubled

  • If local storage is used for more than 90%
Delete
  • If the latest replication job failed for the appliance, the appliance status is set to warned.  
  • If the latest replication job succeeded, the appliance status is set to normal.

What alerts are driven based on the status above?

 
Reason Status
FULL_90_PERCENT Troubled
APPLIANCE_OFFLINED Warned
OFFSITE_STATUS Warned
OFFSITE_HISTORY Warned
OUTDATED_SOFTWARE Warned
FULL_80_PERCENT Warned
LOCAL_CACHE_VERIFICATION_OUTDATED Warned
None Unprotected
None Normal

2. x360Recover protected system health status

There are currently four statuses for protected systems: 

Four types of PROTECTED SYSTEM health status

1. Parked

All jobs on device are disabled

  • Device is a cloud archive

2. Normal

  • Device has only normal jobs (or no jobs at all)
  • It's a newly added device and initial backup window is not opened yet

3. Warned

  • Device has at least one warned job and other jobs are in normal state (disabled jobs are ignored)

4. Troubled

  • Device has at least one troubled job (Disabled jobs are ignored)
  • If local cache is enabled for the machine and last successful check (having "OK" status) is older than the SLA threshold + 3 hours or not present at all
Delete

What alerts are driven based on the status above?

 
Reason Status
OFFSITE_NEVER_RUN Warned
SCHEDULE_DISABLED Parked
LOCAL_CACHE_VERIFICATION_OUTDATED Troubled
None Normal

3. x360Recover job health status

There are currently three statuses for x360Recover jobs.

Three types of JOB health status

1. Normal

  • This includes all cases that aren't related to warned or troubled statuses
  • The health status of the job which has disabled thresholds should always be healthy

2. Warned

  • Local backup/cloud/vault replication job exists but has no RPs and the later of (a) the job creation time or (b) the start time of the first replication window is inside of threshold value

3. Troubled

  • Local backup job is out of local threshold
  • Cloud/vault replication job is out of cloud threshold
  • Local backup/Cloud/Vault Job exists but has no RP and the later of (a) the job creation time or (b) the start time of the first replication window is out of threshold value
Delete

What alerts are driven based on the status above?  

  • There are two types of jobs that can be run against the protected system: local and replication jobs
 
Local Job Reasons Status
ONSITE_HISTORY Troubled
ONSITE_NEVER_RUN Warned
None Normal
 
Replication Job Reasons Status
OFFSITE_STATUS Troubled
VAULT_REPLICATION_NEVER_RUN Warned
CLOUD_REPLICATION_NEVER_RUN Warned
None Normal

4. x360Recover vault health status

Vault health status does not depend on health status of the protected systems backed up on the vault.  Client health status depends on the health status of vault-only protected systems.

There are currently three types of status for vaults:

Three types of VAULT health status

1. Normal 

  • If there are no reasons for warned status

2. Warned

  • If x360Manager did not receive the heartbeat from the vault for more than the amount of time specified by the network connectivity threshold
  • If a vault is online based on the network connectivity threshold and the vault is running an older version of software
  • If a vault is online based on the network connectivity threshold and vault storage loaded more than 80 percent and less than 90 percent

3. Troubled

  • If a vault is online based on the network connectivity threshold and vault storage loaded more than 90 percent
Delete

What alerts are driven based on the status above?

 
Reason Status
VAULT_OFFLINE Warned
FULL_90_PERCENT Troubled
OUTDATED_SOFTWARE Warned
FULL_80_PERCENT Warned
None Normal

 

 


 SUPPORT | 720-204-4500 | 800-352-0248

1241