What is the most challenging thing you have done with Veeam?


Userlevel 7
Badge +8

What is the most challenging thing you have done using Veeam, or solved with Veeam?

 

For me it was the Veeam Lab as I have some very odd networking requirements and trying to solve why it would work but I couldn’t get the machines to talk was a challenge. Some others were wrapping my head around retention policies when I first started and Tape.  Now it’s all just day to day :)

 

As for something challenging I have solved, it’s mostly internal stuff at my company such as our insane retention requirements and large data.  With Object storage I'll be able to archive to the cloud and also to tape (as per policy), delete data from production and retrieve it in 20 years if I am required.  This saves having to restore a full VM from Tape to get one file.

 

 


20 comments

Userlevel 7
Badge +20

For me the biggest challenge was re-architecting the Veeam solutions we use to follow Best Practice and be consistent across data centers.  Also ensuring everyone is educated on specific features when to use them, etc.

Userlevel 7
Badge +8

For me the biggest challenge was re-architecting the Veeam solutions we use to follow Best Practice and be consistent across data centers.  Also ensuring everyone is educated on specific features when to use them, etc.

When I started in my current roll I stacked up a totally new Veeam infrastructure, redesigned everything, and migrated the old data. It was the only way to ensure it was done to best practices. Great learning experience too.  I had to do the same for VMware\vCenter\SRM in a multi site environment and all the SANS, Fiber etc. There was zero documentation and things were not done correctly. 

Userlevel 7
Badge +10

Oh this is a good thread. Would love to see this play out. I can’t find the blog I did in like 2010 or so on it, but I had these *MASSIVE* file shares on iSCSI volumes on Windows. There were about ¾ a Billion files that were all like 45Kb each. I used FastSCP to copy them to a target as a migration. 

Hilarious at the time that Veeam FastSCP helped me move a physical server’s volume…  At that time Veeam’s backup products were VM image based only.

Userlevel 7
Badge +9

Biggest challenge for me was, and still is for most people, understanding the networking requirements for SureBackup. Not so much the complexity but the terminology ...”Production network” doesn’t mean your actual production network, means what you consider to be classed “Production network” in your virtual lab setup ...took me a while to get my head around that

Userlevel 7
Badge +20

For me, it was the introduction of object storage. It felt like it was suddenly a thing, but without all of the supporting detail around how it actually works, in a way that could allow people to confidently forecast an estimate to how much performing off-site backup would cost. It’s getting better, but it’s still not “it just works” levels of intuitive yet.

Userlevel 7
Badge +7

For me, 
It was to convince the business that we needed to buy it!

Before Veeam, the other backup solution was like not having a backup, 

confusing console, failed tasks, etc.
at some point, it was normal to receive warning s and errors every day, and nowadays, for me is mandatory to have a “clean” daily backup!

Under the implementation point of view, I guess to move all the backups from one repo to another, 
all the other tasks seems to be very straight forward in Veeam, fortunately! 😁

Userlevel 7
Badge +8

Oh this is a good thread. Would love to see this play out. I can’t find the blog I did in like 2010 or so on it, but I had these *MASSIVE* file shares on iSCSI volumes on Windows. There were about ¾ a Billion files that were all like 45Kb each. I used FastSCP to copy them to a target as a migration. 

Hilarious at the time that Veeam FastSCP helped me move a physical server’s volume…  At that time Veeam’s backup products were VM image based only.

One of the servers I am looking at sending to object storage right now is defiantly in the 10’s or 100’s of millions of files.   ¾ billion is crazy!   moving a large amount of data on it’s own is a pain, but I prefer it over the same size of data with files in the kb range :)

 

I was hoping to see some unique and interesting ways people have used Veeam to solve internal problems, along with their most challenging hurdles. I suppose it could have been 2 threads. My bet is SureBackup tripping most people up in the beginning.

 

That thought adds another one for me. Well not too challenging We had a DC upgrade go BAD. I mean BAD. Everything went down, Long story short, sysvol was full because of someone putting a large file in it, followed by upgrade enabling dfsr and blowing it all up.

 

Veeam support saved my bacon and made me look like a hero doing the DC restore from backup. Was scary the first time never doing it before but that was worth a year of maintenance right there. Following that, I used SureBackup and created a lab for the guy to test the upgrade again and work with Microsoft to make sure our upgrade went smooth. (next time we’ll do that first lol)

 

I still have a weird issue where the network change will cause the ping test to fail every time so I can never leave it enabled, but I get by. If I go into windows and accept the  network change I can ping the VM no problem, but I have to be super quick in the VM console. 

 

 

Userlevel 7
Badge +7

Hi @Scott , thanks for that,

I don't understand the last paragraph,

I still have a weird issue where the network change will cause the ping test to fail every time so I can never leave it enabled, but I get by. If I go into windows and accept the  network change I can ping the VM no problem, but I have to be super quick in the VM console. 

 

Do you mean that you have this failure when you turn on the SureLab machine?
Or when you restore it from a backup in a new hypervisor?

Once again, thanks for sharing the story, I dont like novels or books but I love and enjoy reading other experiences, and get a little record into my mind, “ I read this somewhere”!

thanks.

Userlevel 7
Badge +8

Hi @Scott , thanks for that,

I don't understand the last paragraph,

I still have a weird issue where the network change will cause the ping test to fail every time so I can never leave it enabled, but I get by. If I go into windows and accept the  network change I can ping the VM no problem, but I have to be super quick in the VM console. 

 

Do you mean that you have this failure when you turn on the SureLab machine?
Or when you restore it from a backup in a new hypervisor?

Once again, thanks for sharing the story, I dont like novels or books but I love and enjoy reading other experiences, and get a little record into my mind, “ I read this somewhere”!

thanks.

 

When the application group boots it fails if ping check is on. If I turn off ping check machines boot fine.  I can also get by buy quickly going into the VMware console, logging into to the machine as SOON as I see a windows screen and clicking yes on the windows notification about “do you want to turn on network visibility” or let others view this computer.

 

I just leave ping test off currently but it would be nice to solve this. 

Userlevel 7
Badge +7

Hi @Scott , thanks for that,

I don't understand the last paragraph,

I still have a weird issue where the network change will cause the ping test to fail every time so I can never leave it enabled, but I get by. If I go into windows and accept the  network change I can ping the VM no problem, but I have to be super quick in the VM console. 

 

Do you mean that you have this failure when you turn on the SureLab machine?
Or when you restore it from a backup in a new hypervisor?

Once again, thanks for sharing the story, I dont like novels or books but I love and enjoy reading other experiences, and get a little record into my mind, “ I read this somewhere”!

thanks.

 

When the application group boots it fails if ping check is on. If I turn off ping check machines boot fine.  I can also get by buy quickly going into the VMware console, logging into to the machine as SOON as I see a windows screen and clicking yes on the windows notification about “do you want to turn on network visibility” or let others view this computer.

 

I just leave ping test off currently but it would be nice to solve this. 

Oh, Ok, I see!! XD

is the vm network configured by DHCP?

Should not change adapter or network range, only if changes IP address, am I right?

Also keep in mind, when the vms comes up, if they are in different vlans, networks, or if another vm should run first cause of dependencies.

This is quite difficult to troubleshoot without seeing it, just take my words as a “brainstorm” to give you some ideas.

cheers!

 

 

Userlevel 7
Badge +8

Hi @Scott , thanks for that,

I don't understand the last paragraph,

I still have a weird issue where the network change will cause the ping test to fail every time so I can never leave it enabled, but I get by. If I go into windows and accept the  network change I can ping the VM no problem, but I have to be super quick in the VM console. 

 

Do you mean that you have this failure when you turn on the SureLab machine?
Or when you restore it from a backup in a new hypervisor?

Once again, thanks for sharing the story, I dont like novels or books but I love and enjoy reading other experiences, and get a little record into my mind, “ I read this somewhere”!

thanks.

 

When the application group boots it fails if ping check is on. If I turn off ping check machines boot fine.  I can also get by buy quickly going into the VMware console, logging into to the machine as SOON as I see a windows screen and clicking yes on the windows notification about “do you want to turn on network visibility” or let others view this computer.

 

I just leave ping test off currently but it would be nice to solve this. 

Oh, Ok, I see!! XD

is the vm network configured by DHCP?

Should not change adapter or network range, only if changes IP address, am I right?

Also keep in mind, when the vms comes up, if they are in different vlans, networks, or if another vm should run first cause of dependencies.

This is quite difficult to troubleshoot without seeing it, just take my words as a “brainstorm” to give you some ideas.

cheers!

 

 

Static IP’s.

 

I have a feeling getting a new MAC address when the VM’s is restored from backup might be causing the network adapter to get confused in Windows, but I could be wrong.  

 

The IP doesn’t change, Vlan is correct.

 

I do have to go in and click ok in that warning before i can connect remotely too.

 

Oh well. i’ll try not to flood this post with off topic troubleshooting :)

Userlevel 7
Badge +7

Hi @Scott , thanks for that,

I don't understand the last paragraph,

I still have a weird issue where the network change will cause the ping test to fail every time so I can never leave it enabled, but I get by. If I go into windows and accept the  network change I can ping the VM no problem, but I have to be super quick in the VM console. 

 

Do you mean that you have this failure when you turn on the SureLab machine?
Or when you restore it from a backup in a new hypervisor?

Once again, thanks for sharing the story, I dont like novels or books but I love and enjoy reading other experiences, and get a little record into my mind, “ I read this somewhere”!

thanks.

 

When the application group boots it fails if ping check is on. If I turn off ping check machines boot fine.  I can also get by buy quickly going into the VMware console, logging into to the machine as SOON as I see a windows screen and clicking yes on the windows notification about “do you want to turn on network visibility” or let others view this computer.

 

I just leave ping test off currently but it would be nice to solve this. 

Oh, Ok, I see!! XD

is the vm network configured by DHCP?

Should not change adapter or network range, only if changes IP address, am I right?

Also keep in mind, when the vms comes up, if they are in different vlans, networks, or if another vm should run first cause of dependencies.

This is quite difficult to troubleshoot without seeing it, just take my words as a “brainstorm” to give you some ideas.

cheers!

 

 

Static IP’s.

 

I have a feeling getting a new MAC address when the VM’s is restored from backup might be causing the network adapter to get confused in Windows, but I could be wrong.  

 

The IP doesn’t change, Vlan is correct.

 

I do have to go in and click ok in that warning before i can connect remotely too.

 

Oh well. i’ll try not to flood this post with off topic troubleshooting :)

right, but no worries, this is for sharing and discussing,  :D

that MAC address thing that you just mentioned, should be the reason,

you can give it a try setting up fixed Mac address to the vm, re launching backup, and then booting the sure lab with the vm backed up with the new configuration!

good luck!

.

Userlevel 7
Badge +8

Hi @Scott , thanks for that,

I don't understand the last paragraph,

I still have a weird issue where the network change will cause the ping test to fail every time so I can never leave it enabled, but I get by. If I go into windows and accept the  network change I can ping the VM no problem, but I have to be super quick in the VM console. 

 

Do you mean that you have this failure when you turn on the SureLab machine?
Or when you restore it from a backup in a new hypervisor?

Once again, thanks for sharing the story, I dont like novels or books but I love and enjoy reading other experiences, and get a little record into my mind, “ I read this somewhere”!

thanks.

 

When the application group boots it fails if ping check is on. If I turn off ping check machines boot fine.  I can also get by buy quickly going into the VMware console, logging into to the machine as SOON as I see a windows screen and clicking yes on the windows notification about “do you want to turn on network visibility” or let others view this computer.

 

I just leave ping test off currently but it would be nice to solve this. 

Oh, Ok, I see!! XD

is the vm network configured by DHCP?

Should not change adapter or network range, only if changes IP address, am I right?

Also keep in mind, when the vms comes up, if they are in different vlans, networks, or if another vm should run first cause of dependencies.

This is quite difficult to troubleshoot without seeing it, just take my words as a “brainstorm” to give you some ideas.

cheers!

 

 

Static IP’s.

 

I have a feeling getting a new MAC address when the VM’s is restored from backup might be causing the network adapter to get confused in Windows, but I could be wrong.  

 

The IP doesn’t change, Vlan is correct.

 

I do have to go in and click ok in that warning before i can connect remotely too.

 

Oh well. i’ll try not to flood this post with off topic troubleshooting :)

right, but no worries, this is for sharing and discussing,  :D

that MAC address thing that you just mentioned, should be the reason,

you can give it a try setting up fixed Mac address to the vm, re launching backup, and then booting the sure lab with the vm backed up with the new configuration!

good luck!

.

 

I don’ think changing all my VM’s to static MAC at this point is going to be very easy to do.  I can’t even change it while powered on.

 

Userlevel 7
Badge +7

if it happens to you in all your vms, omg, 
I had a similar issue, but with a license based on Mac address, and I did the trick and worked fine,

other time with a linux machine, that the fixed IP address was related to the Mac address (centos 6)

 

In that case, should be worth it to investigate how to disable the message to accept or something!

cool! thanks for taking your time and explain your case.

Maybe would be good, if you get it solved, to post it at some point.
Or if you can't but want to, you can always ask support!

cheers”!

take care.

Userlevel 7
Badge +8

It has come up a lot in the forums so I'll see if there is a work around. Thanks

Userlevel 7
Badge +6

Biggest challenge for me was, and still is for most people, understanding the networking requirements for SureBackup. Not so much the complexity but the terminology ...”Production network” doesn’t mean your actual production network, means what you consider to be classed “Production network” in your virtual lab setup ...took me a while to get my head around that

 

SureBackup and SureReplica can be difficult at times and what you mentioned I found to be difficult to grasp at first as well. 

I find it hardest (and still struggle) when running the VirtualLab at a different site/network than the server running the backup role because the backup server doesn’t know how to route to the remote site/network to get to the proxy appliance running the sandbox to verify the restored VM’s.  I put in a feature request a few months back to see if it can run these tasks from the appliance rather than the backup server to get around the routing issues.

Userlevel 7
Badge +22

For me it would have to have been using Veeam agents to backup hundreds of remote sensors out in the wild into a cloud connect! 

Plus creating a home grown Tape Archiving solution before Cloud Connect Tenant to Tape became available. 

Userlevel 7
Badge +22

For me it would have to have been using Veeam agents to backup hundreds of remote sensors out in the wild into a cloud connect! 

Plus creating a home grown Tape Archiving solution before Cloud Connect Tenant to Tape became available. 

On a side note, I now hear that those sensors have been containerized and running on k3s so… :)

Userlevel 7
Badge +8

I’m lucky that even with multi site, I don’t have to use agents. We are just dipping our toes into the cloud world now so I expect the learning and training load to get very steep very quick. 

Userlevel 6
Badge +6

Creating and realizing a solution design for large scale replication worldwide for 11,762 VMs in multiple GEOs and datacenters. 

Comment