Potential crash by log-shipping databases

  • 18 October 2021
  • 3 comments
  • 51 views

Userlevel 5
Badge

Log-shipping with VBR involves your SQL or Oracle server (VM or agent), optionally a log-shipping server and of course a repo to store the VLBs.
With Hyper-V underneath one might even log-ship through the hypervisor by using PowerShell direct. This is nice, as it allows to isolate your DB workloads from the backup network and still leverage log-shipping.

During the process Veeam creates temp files of the logs to be shipped first on the DB server and then the log-shipping server. Finally, after a LZ4 compression, those are driven to the repo and deleted on those temp locations.
On the DB server the temp path for logs is defined in the registry and defaults to the largest disk the server has (since 9.5U4a). The path can be changed: https://www.veeam.com/kb2642
On the log-shipping server the process is always using the %TEMP% path to store the data.

Problem is, VBR does not seem to check both pathes before it copies the raw transaction logs during each cycle. So, a large transaction log to be shipped might completely fill up the disk of the corresponding system.

This is especially critical with a Hyper-V host that might only have this single boot device. One might even think of killing an infrastructure underneath just through database changes.

Any experience or best practices to be shared here?


3 comments

Userlevel 7
Badge +4

Install a new windows Vm on the Hypervisor,

Add is a managed server to veeam

https://helpcenter.veeam.com/docs/backup/hyperv/add_windows_server.html?ver=110

Configure it as a log shipping server.

https://helpcenter.veeam.com/docs/backup/hyperv/sql_backup_log_shipping.html?ver=110#location-of-log-shipping-server-and-vms

 

This way, veeam will never choose the hyperv as a log shipping server and the temp folder will not be on the hypervisor itself.

 

In theory, you should already have the backup server as a vm. Installing it on the hyperv server is not supported/recommend. You could manually choose the backup server as the log shipping server, if you want. Then you don‘t need to deploy another windows vm.

Userlevel 5
Badge

Thanks @Mildur for your comprehensive answer. You’re referring to the vanilla way.
But my question was slighty different:

  1. When using log-shipping via Hyper-V with PS direct (here no network connection from DB to log-shipping server or repo is involved) your Hyper-V always has to be part of the path and stores the logs intermittently. Thats by design here. One of the rare cases where Hyper-V as something in advantage to VMware with VBR… :wink:
    See second method here: https://helpcenter.veeam.com/docs/backup/hyperv/sql_backup_log_shipping.html?ver=110
  2. Your logs always get pre-handled on the SQL/Oracle itself. Also there a disk might fill up as VBR doesn’t check beforehand if the logs fit. OS disk e.g. might crash, if C: is used.
  3. Even a true log-shipping server might fill up. Here VBR uses the %temp% variable which can be adjusted of course.

One might well take care to point those directories to a large enough drive. Those logs can be hundreds of GB in certain cases… Also no checks seem to be done beforehand which I would consider a feature request. Especially for the Hyper-V with PS direct case.

Userlevel 7
Badge +4

Yes, you are right. I wasn‘t aware of that. :)

We only use HyperV on small customers. They don‘t have databases with big changes between the log backup cycles. And the db server can access the log shipping server over network.

Comment