Oracle Retention for RMAN and Veeam Plugin Backup
When it comes to RMAN, Veeam acts as a storage location utilized by the Oracle SBT interface. Therefore, it is RMAN retention settings that are utilized to manage the backups generated when using the Veeam Plugin backup protection.
Typical retention for application aware backup will focus on the most recent restore points and mark backups to be pruned per familiar Veeam retention policies and retention methods.
With Oracle, an incremental strategy employs a combination of incremental level 0, incremental level 1 and incremental level 1 with cumulative incremental or stand alone full. Each of these backup types dictate what data RMAN will identify and capture to create backup sets.
Data captured during any one of these types can be manipulated as desired by the user. Typically, an incremental level 0 backup will capture all files necessary to restore and recover a database including the Initialization or SPFILE, Control File, all database data files and the archived redo logs. Depending on circumstances and settings, it may be necessary to declare each of these types within the backup database commands utilized.
An Oracle incremental level 1 backup will capture all database changes that have occurred since the most recent incremental level 0 or incremental level 1 backup.
The Oracle incremental level 1 with cumulative incremental will capture all database changes that have occurred since the most recent incremental level 0 backup.
It is also necessary to backup oracle archived redo logs at a frequency which provides datafile recovery in between the primary database incremental level backups.
Oracle Backup retention methods
RMAN utilizes one of three definitions for retention.
The first definition for backup file is ‘REDUNDANCY’.
‘REDUNDANCY’ refers to the number of full or incremental level 0 backups that are kept.
The default setting for redundancy is 1. RMAN automatically retains all backups required to restore and recover the most recent level 0 backup. Any not necessary for the redundancy setting chosen are marked as obsolete and subsequent commands
RMAN> delete obsolete
Will remove the backups no longer necessary for the retention setting in place.
A setting of 2 will mean that two incremental level 0 or full backups are necessary for minimum protection.
A specific call must be made to delete the obsolete backup sets at regular intervals to prevent the system from running out of space. This provides an option for retaining only those backups necessary to match the business requirements for the database system.
The second method to protect the database backup files is a ‘RECOVERY WINDOW’
A recovery window is used with a specified number of days. In this scenario, the time in days for retention will identify the incremental level 0 and subsequent incremental and log backups that are to be held to perform a complete restore and recovery for the database.
RMAN retains all archived redo logs and level 1 incremental backups necessary to recover to a point within that time window as current backups and will not identify them as obsolete until the associated incremental level 0 backup is outside of the recovery window and a new incremental level 0 backup is available along with its subsequent incremental and log backups.
Again, as with redundancy based retention, backup sets are only deleted by issuing a delete command and the database backups outside of the window are the only backups considered obsolete and subsequently deleted.
Oracle database GFS concept
The retention described above for Oracle has one variation. That is to use the parameter ‘KEEP’ when creating a backup. This allows a setting to utilize a number of days for retention.
With Veeam, the concept of GFS or ‘Grandfather/Father/Son’ is a method to retain the backups created for long term restore. Options will usually consist of a weekly backup for some number of months, a monthly backup for a longer term such as a year and yearly backups kept for multiple years. Each backup identified are specified recovery points for the backup completed.
These archival backups are usually to comply with regulatory requirements the organization must adhere to.
The individual backups completed within the Oracle Incremental Level definitions can be associated with a retention time that is specific. It is usually generated by a backup command either stand alone or from within a script.
The value used for the parameter will then be the duration of retention utilized. An exact time frame can then be identified for longer term backup retention on desired backup sets.
This third type of retention is also managed from RMAN settings and is considered an ‘archival backup’ by Oracle. Those backups are created using the ‘KEEP’ parameter within the syntax of a backup command.
The keep parameter can be applied to specific backup sets after a backup has completed, but in this document, we are focusing on creating a process in which the desired backup retention will be noted and applied programmatically.
Retention will be maintained using the parameter applied to the backup, and Oracle will consider those backup sets as exceeded when the date used in the KEEP time is exceeded. A simple example of this syntax would be as follows:
RMAN> backup database keep until time ’12 – DEC -22’
Backup sets created with the KEEP option are considered archival by Oracle. This is a self-contained backup. It will have all files necessary to restore the backup and recover it to a consistent state. If the database is open during the backup operations, all archived redo logs files needed to perform the recovery to the KEEP time will also be captured and stored. See the following link for further details:
See detail with KEEP syntax examples for Oracle v19 as follows. Other versions have this option and will also follow this example at the bottom of the page:
The actual time will be stored in a field with each backup. The retention value can be set using scripts to automate each backup taken and provide the appropriate setting for the retention.
It is possible to utilize this method to manage all backups to the extent required where some may be short term and others that qualify as the backup for the week/month/year can also be stored appropriately as needed.
Frequently, a TAG will be applied to provide the user with a visual cue as to the type of backup set and its retention category.
As with the earlier evaluation of retention for redundancy or backup window, when the date stored has been exceeded, the backups will be identified as ‘obsolete’ by RMAN and will be removed with scripts calling for ‘delete obsolete’
Recovery Catalog Database
When using the ‘KEEP’ retention method, it will be crucial to also utilize a recovery catalog database. The recovery catalog database will maintain the metadata for those backups that exceed the typical storage within the control file of the database.
The typical time for the meta data retention is set by the parameter ‘CONTROL_FILE_RECORD_KEEP’ time and applies to the backup information stored in the local control file for each database.
However, when the backup sets become older and space becomes used, Oracle will remove the records from the control file based upon the ‘CONTROL_FILE_RECORD_KEEP’ setting. The backups will still be stored and are accessible, however, further steps would be necessary to utilize the backup set should the need arise.
With the recovery catalog database, records are kept in a database schema that contains the same information as that in the Oracle control file. Because it is a database, these records will be stored for as long as necessary to support the backup being kept.
The recovery catalog database would be able to maintain the older metadata for the backup while the backup data is stored within Veeam repositories.
Syntax and Usage examples
The examples that follow could utilize either the ‘FULL’ backup or the incremental level 0 backup. In these cases, because the output of the backup set is intended to be for longer term backup, using the FULL backup type will capture all required elements to restore the database.
The difference between a FULL backup and an incremental level 0 backup exists with the ability of the Level 0 backup able to function as the beginning of an incremental protection strategy in which the incremental level 0 acts as an initial backup storing all used data blocks within the datafiles making up the database followed by incremental level 1 which stores only those blocks that have been added or updated after the incremental level 0.
If the incremental level strategy is to be used during the use of the KEEP time, a TAG must also be used.
Daily
RUN
{
ALLOCATE CHANNEL VeeamAgentChannel1 DEVICE TYPE SBT_TAPE
PARMS ‘SBT_LIBRARY=/opt/Veeam/VeeamPluginforOracleRMAN/libOracleRMANPlugin.so’
FORMAT ‘dac83d46-0a9f-431c-b427-8a504ba7b074/RMAN_%I_%d_%T_%U.vab’;
BACKUP AS COMPRESSED BACKUPSET FULL FORMAT 'FULL_DAILY_%T_%d_%U' TAG 'FULL DAILY BACKUP' DATABASE INCLUDE CURRENT CONTROLFILE;
sql 'alter system archive log current';
sql 'alter system switch logfile';
BACKUP AS COMPRESSED BACKUPSET ARCHIVELOG ALL FORMAT 'FULL_DAILY_%T_ARCH_%d_%U' TAG 'FULL DAILY BACKUP';
}
This command for daily could also be written as the following using the incremental strategy. Note the difference with the parameters used in the backup database command. In the example above, the ‘COMPRESSED BACKUPSET’ parameter is used, and the format is used to name the output file. In both cases, the tag is used to help provide more information regarding the backupset created.
Also note how the archived redo logs are being captured in the same operation and the use of ‘sql’ commands are in place to roll the online redo log file contents and create archived redo logs just before performing the log backup:
RUN
{
ALLOCATE CHANNEL VeeamAgentChannel1 DEVICE TYPE SBT_TAPE
PARMS ‘SBT_LIBRARY=/opt/Veeam/VeeamPluginforOracleRMAN/libOracleRMANPlugin.so’
FORMAT ‘dac83d46-0a9f-431c-b427-8a504ba7b074/RMAN_%I_%d_%T_%U.vab’;
BACKUP INCREMENTAL LEVEL = 0 TAG 'FULL DAILY BACKUP' DATABASE INCLUDE CURRENT CONTROLFILE;
sql 'alter system archive log current';
sql 'alter system switch logfile';
BACKUP ARCHIVELOG ALL FORMAT 'FULL_DAILY_%T_ARCH_%d_%U' TAG 'FULL DAILY BACKUP';
}
The following blocks are also able to utilize the examples provided with the daily backup.
Weekly
RUN
{
ALLOCATE CHANNEL VeeamAgentChannel1 DEVICE TYPE SBT_TAPE
PARMS ‘SBT_LIBRARY=/opt/Veeam/VeeamPluginforOracleRMAN/libOracleRMANPlugin.so’
FORMAT ‘dac83d46-0a9f-431c-b427-8a504ba7b074/RMAN_%I_%d_%T_%U.vab’;
BACKUP AS COMPRESSED BACKUPSET FULL FORMAT 'FULL_WEEKLY_%T_%d_%U' TAG 'FULL WEEKLY BACKUP' DATABASE INCLUDE CURRENT CONTROLFILE KEEP UNTIL TIME 'SYSDATE+28';
BACKUP AS COMPRESSED BACKUPSET ARCHIVELOG ALL FORMAT 'FULL_WEEKLY_%T_ARCH_%d_%U' TAG 'FULL WEEKLY BACKUP' KEEP UNTIL TIME 'SYSDATE+28';
}
Monthly
RUN
{
ALLOCATE CHANNEL VeeamAgentChannel1 DEVICE TYPE SBT_TAPE
PARMS ‘SBT_LIBRARY=/opt/Veeam/VeeamPluginforOracleRMAN/libOracleRMANPlugin.so’
FORMAT ‘dac83d46-0a9f-431c-b427-8a504ba7b074/RMAN_%I_%d_%T_%U.vab’;
BACKUP AS COMPRESSED BACKUPSET FULL FORMAT 'FULL_MONTHLY_%T_%d_%U' TAG 'FULL MONTHLY BACKUP' DATABASE INCLUDE CURRENT CONTROLFILE KEEP UNTIL TIME 'SYSDATE+457';
BACKUP AS COMPRESSED BACKUPSET ARCHIVELOG ALL FORMAT 'FULL_MONTHLY_%T_ARCH_%d_%U' TAG 'FULL MONTHLY BACKUP' KEEP UNTIL TIME 'SYSDATE+457';
}
Yearly
RUN
{
ALLOCATE CHANNEL VeeamAgentChannel1 DEVICE TYPE SBT_TAPE
PARMS ‘SBT_LIBRARY=/opt/Veeam/VeeamPluginforOracleRMAN/libOracleRMANPlugin.so’
FORMAT ‘dac83d46-0a9f-431c-b427-8a504ba7b074/RMAN_%I_%d_%T_%U.vab’;
BACKUP AS COMPRESSED BACKUPSET FULL FORMAT 'FULL_YEARLY_%T_%d_%U' TAG 'FULL YEARLY BACKUP' DATABASE INCLUDE CURRENT CONTROLFILE KEEP UNTIL TIME 'SYSDATE+2556';
BACKUP AS COMPRESSED BACKUPSET ARCHIVELOG ALL FORMAT 'FULL_YEARLY_%T_ARCH_%d_%U' TAG 'FULL YEARLY BACKUP' KEEP UNTIL TIME 'SYSDATE+2556';
}
Manage backup sets
Use the ‘list backupset’ and ‘list backupset N, n…‘ to view information regarding 1 or more backupsets.
Use the ‘change backupset’ commands to locate and manipulate desired individual backupsets. Your use may appear similar to the following:
