Hey Scott,

Welcome to the Community! 
You could potentially get the status of running jobs by using PowerShell 

https://helpcenter.veeam.com/docs/backup/powershell/get-vbrjob.html?ver=110

If there are no jobs running, perform the install and do the reboot. You could download the updates in the meanwhile. I’m guessing for your Veeam servers, you are going direct to Windows Update over the Internet? 

That is true, but I personally don’t mind having a bit of control of my backup infrastructure.

The more I start using replicas, CDP, etc. the less I really want to automate something that might not come back if I’m on vacation. 

Yep, that makes sense.

Also, make sure you keep Veeam Configuration Backups from your Veeam B&R server in case you ever need to perform a restore. 

Of course. I have multiple copies of my config backup and have done a few restores in the past. Once you get heavy into the tape it’s a LONG process sometimes. 

We use a patch management system for our patching on all servers which can stage and prepare them.  Once ready then you say install and it installs them.  Once done you can reboot right away or schedule that with the system too.

The Veeam services we treat like all other systems because we send mass notifications to customers 10 days in advance of our patching window for the month (second Thursday of the month for patch Tuesday).

We do the updates to our backup server at another time as to the poduction servers (normally one week later).

So, we have the backup environment up an running in the case there is something wrong with the updste process to the production servers.

The backup servers itself are patched the same way as all other servers- via WSUS.

I was wondering why you were explaining really basic stuff to me, then I just noticed you put welcome to the Community” lol.  I’ve been a member for some time.  It makes more sense why you were telling me to keep my config backups saved now though lol :) 

> For most of my servers I use  a WSUS deployment I have scripted all of my servers to check in weekly and to download the updates at a specified time.

Good to know that you are using WSUS! But why does your client(s) have to check weekly? Seems to me you do not test and approve updates to your clients.

>  I use flags so they won’t download the updates if a reboot is currently pending incase a previous month is missed. I then use VMware to scheduled reboot the hosts as It’s much more consistent this way and easy to disable.

There are tons of GPO settings to help with all these. There are polices to download only as far as I can remember.  Let me introduce you to another exciting tool “Windows Admin Center (WAC)” for server management since you love to manually install these updates. I am sure that you already know about this :-)

Lastly, do you know that you can also target your VBR via WSUS. WSUS can be deployed in a disconnected scenario, where updates and metadata are exported on one WSUS server and imported on another disconnected WSUS server. This scenario makes WSUS useable in highly-restricted perimeter networks. 

• Here is a simplified method: All you need are just some few registry tweaks!  https://learn.microsoft.com/en-us/answers/questions/447288/shared-wsus-server-for-domain-and-non-domain-joine.html

Great points!

Yep, I didn’t see you’d posted here before. For some reason I thought you’d just joined. 

I really don’t want to manage a second WSUS server, it’s not that i have a “Secure” area, it’s that my veeam servers are not domain joined. 

I have used GPO’s to deploy windows updates, however, my script does a lot more than the GPO’s available such as emailing users, sending out notifications that the server will go down etc. It’s an all in one bundle I run as a scheduled task.   It’s easy to disable a scheduled task on individual machines when I am required too. GPO’s can get more tricky to manage when someone need something disabled in 3 minutes from now.  Due to the nature of where I work this is the best option and the others have been considered. 

I understand now! Great script by the way...

Community info: Should in case anyone wants to automate Windows Update from WSUS or Microsoft Update Center via PowerShell and Task Scheduler, please see the following links below. In it, I have got WSUS guides referenced. 

• https://techdirectarchive.com/2020/03/24/how-to-orchestrate-windows-update-with-powershell-and-task-scheduler/
• https://techdirectarchive.com/2022/03/23/how-to-apply-windows-updates-with-powershell/

From my personal experience, if your backup infrastructure (Veeam Servers and related) is not too big, I would like keeping control over it, cause is my last line of defense,

I personally run windows updates manually, downloading the updates and performing the install and reboot between jobs hours, and placing a snapshot before updating if it was a vm, and always keeping close the latest config file in case of needed.

We did monthly patching, and treated the Veeam servers as all other servers, but taking into consideration that they are a bit special, so not taking risk over them.

cheers.

When I was aligned to working on customers in a managed capacity, we used to use an RMM tool to control patching for all systems. Most customers typically had data protection desires of nothing more than daily backups, for these we used to patch during the day, a few hours before their next backup window, a day or two before their next full backup was due. This way processing was just generic incremental that would be completed.

The nice thing about this was that patching was a policy regardless of domain/workgroup status, and it was auto-adjusting based on the detection of Veeam components whether they were on the backup server patching schedule or the “normal” patching schedule.

I know this isn’t that helpful to you, but would your environment have capacity for a bastion domain? This way you’d have centralised management & orchestration options at every level, GPOs etc.

Due to the long backup windows I have, (lots of 40TB+ file servers), Requirements for Tape, and insane retention (60 years, 100 years, forever) my tape jobs sometimes take a fair amount of time. I always need some control over when my reboots take place, and the ability to abort at the drop of a hat too. 

The way my script works for the other servers is fantastic when they say “NO UPDATES TODAY” I just turn off the scheduled task for the reboot. The update stays staged ready for whenever. if it tries again the script sees a waiting reboot and won’t download the updates if someone doesn’t reboot it for a month and it will catch up the following month.

It’s all automatic and only involves setting up a scheduled task upon creation of the servers.  I’ve never looked into a bastion domain, time to do some google learning :)

@HunterLAFR  sounds very similar to me. For the few servers I have in Veeam, although growing, i don’t mind doing them 1 at a time and saving things like config backups, or doing a snapshot etc.

Once you have a large MS update go and bork about 25-40 servers in one go, you learn your lesson. 

Hey @Scott 

Those scripts might be useful for other users. Are they open source and available for other users to use?

If so, would you be willing to write a blog post about it in the Automation Desk?

I’ll go and see if there is anything that needs to be redacted  from them and share what I can.  There may be some very specific stuff I can comment out or remove depending on if it would be beneficial to others.  Might take me a bit though, being pulled in 1000 directions lately :) 

Specific schedule sort BY OU, planned downtime on monitoring tools. Everyone know servers will reboot at this time interval.

Staging servers will be done backup infra. If sometimes goes wrong on staging servers, it blocks.

I think everyone should be able to have server scheduled maintenance window but your perfectionism is cool to read :D

Since I started here I refuse to install new servers without a Window and make it part of every project implementation to have a monthly outage.   Due to how critical our systems are though, this requires an insane amount of HA, FT solutions etc which get VERY expensive.  Getting SQL Enterprise was a challenge but if you want me to keep creating servers, you need to give me windows to reboot them for patches. 

When I started here there were hundreds of servers with 1000 days - 2000 Days of uptime so I have done well lol. 

Hi @Scott, good question! Personally I like automation, especially for Windows updates (after a certain gap, because installing too soon new updates is too risky, because Microsoft does not always release reliable updates). So normally all security and critical updates are being scheduled and installed automatically EXCEPT for all the backup components!

With components I mean : VBR-server, repositories, proxy-servers, …

Personally I dont’ like those being scheduled automatically. I recommended doing it manually at scheduled times.

Why?

You don’t know for sure it there are still backups running.

Also, normally backups are running during the evening - night, also when Windows updates are being installed. So, you don’t want to combine those .

So I schedule in my calendar doing those manually during business hours when mostly backups aren’t running anymore. You could schedule those during the business hours, I know, but I don’t like it.

It’s the perfect time to verify if all backups are OK, I disable the jobs, install the Windows updates manually, reboot the server(s), verify again for new updates, reboot again if necessary and if everything seems to be OK, I enable the jobs again and run normally a small job or run a quick backup for 1 or 2 VMs to see if everything is OK.

Also I doublecheck the day after if the backups are OK.

So I treat the backup-server(s) other than every other server.

That’s my personal recommendation.

And, of course, take snapshots of the servers before patching...because...MS...

Absolutely!!  LOL 

I should probably put more thought into our patching, but we apply Windows updates via our RMM.  Since nearly all of our deployments are using the SQL Express deployment on the same server as VBR, we don’t run into many dependency issues at least, but for the most part we’ve been letting it ride with the rest of the patching.

That's what Veeam is for.. I rarely get to do full VM restores lol