Snapshot retention settings - x360Recover

Written By Tami Sutcliffe (Super Administrator)

Updated at May 31st, 2023

Customizing your backup snapshot retention lets you specify the length of time your recovery points are kept on a device before being automatically purged.

How should I configure my snapshot retention settings?

There are two methods available for determining how snapshot retention maintenance will be performedbasic versus tiered.

  • Tiered retention is the modern standard and we recommend it for all use cases and environments.

Basic retention snapshot cleanup


Tiered retention snapshot cleanup

  • Basic retention retains all snapshots less than a given number of days old. 
  • Snapshots older than the set retention period are discarded. 

Note: Basic retention is deprecated. We do not recommend the use of basic retention in any environment.
Basic retention does not use storage efficiently. 

This means significantly more storage is required for a given length of recovery retention using basic retention settings.

  • Tiered retention (sometimes referred to as Grandfather-Father-Son) is a process by which recovery points are removed differently in various time frames or tiers.
  • Recently-created granular recovery points are more frequently retained, while fewer older recovery points are kept.

Note: We recommend tiered retention for all use cases and environments.

Tiered retention involves granular recovery points with recent time frames (so fewer recovery points are retained for older backups.) 

Tiered retention also allows you to specify the number of daily, weekly, monthly, and even yearly recovery points for a given protected system, with intervening backups automatically removed and consolidated.


Default retention settings 

  • Tiered retention is the default retention policy when the operating system is installed. 

 A newly-registered protected system will default to tiered retention with the following settings:


Tiered retention:

  • Keep all snapshots for fourteen (14)  days

After fourteen (14) days, tiered retention uses the following settings:

  • Keep last snapshot of the day for thirty (30) days
  • Keep last snapshot of the week for ten (10) weeks
  • Keep last snapshot of the month for twelve (12)  months
  • Do not keep any yearly snapshots

How long does the x360Recover tiered retention mode keep my snapshots?

  • All snapshots are retained for some number of days (typically around 7-14 days.)  
This length of time provides granular recovery of changes made throughout each day for recent backups. As an example, you may need this when users request recovery of a file from a specific time in a recent workday.

  • One snapshot per day is retained for some number of days (typically around 30-45 day.) 
This time frame provides reasonable granularity of available backup snapshots. These snapshots can be used for recoveries spanning a medium time frame, such as when a user discovers a bad file but does not know specifically when the file was lost or whether it was corrupted recently.
Note: Only the last backup of the day is retained during this timeframe. All other backups are removed to conserve storage space on the BDR server.

  • One snapshot per week, one snapshot per month, and one snapshot per year may be retained for some number of weeks, months, or years.  
These tiers allow long term historical backups for compliance and SLA purposes, while efficiently managing the storage requirements of the BDR server.  
Note: Only the last backup of the calendar week, month, or year is retained during these timeframes and all other backups are removed to conserve space.


How does tiered retention work?

Think of tiered retention as a series of passes. Each pass "marks" a given snapshot to be retained. A snapshot that is "marked" by any pass must be retained.  

There are three types of snapshot retention passes available:

Pass to retain all snapshots

You can set this to retain a specific number of days. This must be a positive, non-zero number. If you select an invalid value, the default will automatically set to 14 days. The retention period is relative to the date/time of the newest snapshot, or the current date/time, whichever is older. All snapshots newer than the  retain-all-oldest-mark-date will be marked for retention.

Pass to retain a certain number of snapshots

There are four instances of this type of pass:

  • End of day
  • End of week
  • End of month
  • End of year

These passes are all independent of each other. (In other words, the "time periods" for each of these passes will overlap.)  These passes are all relative to the "retain-all-oldest-mark-date" and the times for each are not relative to each other.

In this way, each pass simply marks additional recovery points to be retained.

Changing the configuration of a different "end of period" pass does not change any other "end of period" pass.

AirGap

 Airgap is deployed by default to ensure recoverability of recent backups and retains a number of snapshots (regardless of dates). This ensures that some minimum number of recovery points is always available for recovery. For more information on AirGap, please review this article

Exactly which snapshots will be retained?   

With tiered retention, it is important to understand the selection method used for determining which snapshots will be retained vs those that will be deleted.

  • Both basic and tiered retention start by retaining all snapshots for some number of days. 
  • Retention of specific snapshots (to satisfy the designated tier retention periods) begins after the last day of this keep-all period.
  • The snapshot chosen for retention will be the last snapshot available within the given period. 

For example, when calculating daily snapshots, the snapshot chosen for retention will be the last backup taken during each day.

But remember to consider the details of those longer time periods:  The last backup during a weekly time frame might be on Friday, or Sunday, or even Tuesday afternoon, if backup frequency is intermittent.


More questions about your current retention settings? See How to change the backup retention settings of a protected system.


Semantics:  Po-TAY-toe or Po-TAH-toe?

  • IMPORTANT: Tiered retention is predicated on retaining the last x snapshots for each given time frame. 

For example: Setting retention to keep seven daily snapshots does NOT (necessarily) mean you'll keep only snapshots from the last seven days. 

Setting retention to keep seven daily snapshots means you'll retain the last seven existing end-of-day snapshots. 

So, if your protected system is configured to only take one backup per week, is that seventh snapshot retained from 42 weeks ago? 

Not quite. It will actually be x number of days (based on your configuration for the Keep All snapshots value) plus 42 weeks old!

  • IMPORTANT: All tiers are each calculated starting the day AFTER the end of the Keep All snapshots period. 

In other words, each tier is calculated independently, starting from the same date: the end of the Keep All time frame. 

  • Unlike most chain-based backup solutions offering tiered retention (sometimes referred to as Grandfather-Father-Son or GFS retention), x360Recover does NOT (a) keep  x  daily  snapshots, and then (b) keep another  x weekly  snapshots etc.  All tiers concurrently overlap one another, by starting from the same date in time. 

For example, if you retain fourteen  daily  and two weekly snapshots, both of those weekly snapshots overlap with one of the fourteen daily snapshots -  and you have not added any length to your overall retention time frame.


Special values

Zero as a retention tier setting

Setting any of the tier retention values to zero disables that tier.  For example, you might set the yearly tier to zero to only retain some number of daily, weekly, and monthly recovery points. 

You can also "skip" a tier by setting it to zero. For example, if you set weekly to zero, you will only keep daily, monthly, and yearly recovery points.

Note:  You cannot set Keep All to zero.


Storage license retention

Storage licensing is a special x360Recover license mode in which protected systems are licensed by the amount of storage they consume on the appliance -  rather than by the number of individual servers and workstations being protected. 

Storage licensing includes

  • unlimited endpoints backed up on a given appliance (within the licensed storage amount)
  • storage within the Axcient Cloud for either a three-year or ten-year period.

Retention settings for endpoints licensed under this licensing mode may be set to any value on the appliance but are restricted (and locked) to a fixed retention policy on the Cloud vault (depending on whether a 3-year or 10-year license is assigned.)

 

Three-year storage license fixed retention policy (vault only)
  • Keep all snapshots for seven days

After seven days:

  • Keep last snapshot of the day for thirty one days
  • Keep last snapshot of the week for nine weeks
  • Keep last snapshot of the month for twelve months
  • Keep last snapshot of the year for three years

 

Ten-year storage license fixed retention policy (vault only)
  • Keep all snapshots for fourteen days

After fourteen days:

  • Keep last snapshot of the day for sixty two days
  • Keep last snapshot of the week for eighteen weeks
  • Keep last snapshot of the month for thirty six months
  • Keep last snapshot of the year for ten years

Data retention safety rules

One major goal of a retention cleanup mechanism in a disaster recovery system is to efficiently optimize storage space. Optimizing storage with tiered retention generally allows for larger numbers of older recovery points from which to recover files or perform wholesale disaster recovery.

But the primary goal, of much greater importance, is to ensure that sufficient historical recovery points are retained to recover from any disaster situation. 

And to ensure that successful recovery experience, Axcient has added additional caveats and conditions to our retention cleanup rules. We prefer to err on the side of expanding the data available for recovery rather than focusing solely on optimizing storage.  

The following rules modify and extend the pure retention schema detailed above: 


  • The value of each tier is calculated as plus 1

Whatever value you enter for daily, weekly, monthly, or yearly, the retention engine adds one to it. 

Why do we do this? Because calculating retention based on keeping the last recovery point is somewhat counter-intuitive. Determining in your head what your oldest snapshot will be is tricky. 

For the most extreme example of this, consider yearly backups.  If you set your schedule to retain one yearly snapshot, intuitively (in most people’s minds) that implies that you retain a whole year of data that you can roll back and recover. The last yearly snapshot will presumably be from one of the last days in December, from last year.  But - if today is Jan 1, that ‘yearly’ snapshot is from last week! 

  • The bottom line is that retaining one extra daily or weekly snapshot has almost no impact on your overall storage usage. And retaining that extra monthly or yearly snapshot ensures that you really have that whole month or year of data history.

Note:  Adding +1 to the value does not apply to values of zero or -1.


  • Backup interruption detection

The start of the Keep All retention period is based on the most recent snapshot. 

If no new backups are taken, no snapshot cleanup will occur.

  • For example, your appliance has the Keep Allsnapshots set to thirty days. 
    • On Friday evening, something happens to your production environment, all your systems are down, and no backups occur over the weekend. 
    • On Monday, your client calls for assistance with recovering their environment.
    • On your appliance, you still have all the original snapshots from thirty days prior to the last backup taken which was on Friday.

A note on end-of-the-week oddities

End of the week snapshots have the following potential peculiarity:

  • The ISO year may consist of either 52 or 53 full weeks. 
  • A week always starts on a Monday and ends on a Sunday.
  • The first week of an ISO year is the first (Gregorian) calendar week of a year containing a Thursday.  For example, if January 1 is a Thursday, the start of the week would be December 27 of the previous year. 

FAQs:

Q: What happens during the retention passes if there have been no backups today?

A: The retention time window starts yesterday.  Essentially, if backups stop occurring, retention cleanup will stop occurring. 

Q: If a system clock is improperly set to the future, and then a snapshot is created, what happens when the clock is set backwards to the correct date/time?

A:  The "retain-all-starting-date" is either the date/time of the newest snapshot, or the current date/time, whichever is older. (This ensures that a "parked" system will not start "losing" snapshots.) 

Q: Are ALL snapshots newer than the "retain-oldest-mark-date" marked for retention?

A: Yes.

Q: What happens if I disable the "effective-number-of-periods" by setting it to zero?

A:  The pass will be skipped without doing anything. No end of period snapshots for that type of period will be marked for retention. 

Q: How is the oldest snapshot for a day interval determined?

A:  The latest snapshot is chosen from all snapshots included in the date interval between 00:00 and 23:59 of the day of the year determined by the date parameters according to the date data type. 

For example: To determine the last snapshot of the daily period, we select all snapshots that relate to the time between 2021-04-28 00:00:00, 2021-04-28 23:59:59. The oldest snapshot will be the one that is closer to the end time of the current day period.

Q: How do I determine the oldest snapshot for a week?

A: The oldest snapshot for a week interval is determined by selecting the latest snapshot for all snapshots included in the date range between 00:00 on the first day of the week and 23:59 on the last day of the week, as defined by the date parameter set according to the Python date data type "isocalendar" function.

Q: When a protected system is configured for replication, are snapshots deleted that have not yet been successfully transferred to all configured vaults and ingested into ZFS?

A: No, snapshots that have not yet been successfully transferred to all configured vaults and ingested into ZFS cannot be deleted and will not be removed by retention cleanup regardless of what settings the user has chosen for retention.

Q: Do retention policies vary based on the license type?  

A: Protected systems licensed on an appliance using Storage Licensing have a fixed mandatory retention policy on Axcient-hosted vaults.  (The actual policy is dependent on the Storage License selected (3-year or 10-year), with changes to retention settings on the vault locked out.  Appliance retention settings may be selected at will.)

Q: Are there any situations were retention cleanup on vaults is blocked?

A: Retention cleanup on vaults is blocked for a protected system while network recovery back to an appliance is in progress.  While the job is active, retention cleanup will not occur and snapshots will not be removed.

Q: How efficient is the retention cleanup algorithm?

A:  The retention cleanup algorithm should use a reasonable amount of memory (< 1 GB) and take a reasonable amount of time (< 5 seconds) even for a protected system that has 10,000 snapshots.


 More questions about retention settings?

Still have questions about setting your retention? 

Contact open a Support ticket or call 800-352-0248


 SUPPORT    | 720-204-4500 | 800-352-0248

802  |  826   |  951  |  1329  |  1350  |  1513