I haven’t heard of a DB growing like that @PHuber . Aside from a tuning suggestion given in the User Guide:
https://helpcenter.veeam.com/docs/backup/vsphere/postgresql_instance_configuration.html?ver=120
I’d also recommend getting with Veeam Support to see what other DB performance enhancements & modifications can be done to keep your DB more in check.
Best.
This seems quite odd. Following this.
Could you have possibly configured your NAS cache repository to use the same repository as the Veeam Config DB?
To a certain extent, this is expected behavior: The reason is your file-to-tape job, secondary to your NAS job. Even with an efficient NAS job before it, it actually acts as file-to-tape AFAIK.
File-to-tape jobs generate a lot of metadata in the DB, as they put roughly the same amount of data into it per file as a VM job does per VM. Most people have way more files than VMs…
Therefore, it was always recommended to use a fully licensed and well performing MS-SQL server for VBR, when file-to-tape comes into play.
https://bp.veeam.com/vbr/2_Design_Structures/D_Veeam_Components/D_VBR_DB/database.html
https://helpcenter.veeam.com/docs/backup/vsphere/file_to_tape_jobs.html?ver=120
https://forums.veeam.com/tape-f29/file-to-tape-limitation-t62821.html
With PostgreSQL for VBR still being fairly new, there are not much recommendations here.
How many files are we talking about in your case? It’s the number of files, that counts. I would assume >>1M in your case to rectify 650GB of DB size.
To my knowledge, there is no way to reduce the size of the DB with regards to tape data - except for getting rid of it in your tape backups again.
I would engage support to let Veeam know about your usecase and make sure, that everything is setup correctly.
Best, Michael.
To a certain extent, this is expected behavior: The reason is your file-to-tape job, secondary to your NAS job. Even with an efficient NAS job before it, it actually acts as file-to-tape AFAIK.
File-to-tape jobs generate a lot of metadata in the DB, as they put roughly the same amount of data into it per file as a VM job does per VM. Most people have way more files than VMs…
Therefore, it was always recommended to use a fully licensed and well performing MS-SQL server for VBR, when file-to-tape comes into play.
https://bp.veeam.com/vbr/2_Design_Structures/D_Veeam_Components/D_VBR_DB/database.html
https://helpcenter.veeam.com/docs/backup/vsphere/file_to_tape_jobs.html?ver=120
https://forums.veeam.com/tape-f29/file-to-tape-limitation-t62821.html
With PostgreSQL for VBR still being fairly new, there are not much recommendations here.
How many files are we talking about in your case? It’s the number of files, that counts. I would assume >>1M in your case to rectify 650GB of DB size.
To my knowledge, there is no way to reduce the size of the DB with regards to tape data - except for getting rid of it in your tape backups again.
I would engage support to let Veeam know about your usecase and make sure, that everything is setup correctly.
Best, Michael.
I can confirm when you use file to tape job the DB increase a lot because each file you have on all tape are also présent in database…
However I m sorry but I don't find any recommendation about sql should be préfèred to postgre in this scénario...in your links I don t find it too... sql standard was recommended before postgre because it was the only way... 10gb can t handle a lot of file to tape so you were forced to migrate if you want to continue to backup your datas.…
The only way to reduce space support gave me was to remove from catalog some tapes and restart veeam service to get back the space.
I can confirm when you use file to tape job the DB increase a lot because each file you have on all tape are also présent in database…
However I m sorry but I don't find any recommendation about sql should be préfèred to postgre in this scénario...in your links I don t find it too... sql standard was recommended before postgre because it was the only way... 10gb can t handle a lot of file to tape so you were forced to migrate if you want to continue to backup your datas.…
The only way to reduce space support gave me was to remove from catalog some tapes and restart veeam service to get back the space.
It’s only the metadata that fills up the database. The files themselves go to tape, not the database. I guess, you meant that, @damien commenge.
There is no specific recommendation to prefer MS-SQL over PostgreSQL for file to tape. But it’s not only about database size hard limitations. For SQL-Express the main problem was most of the time the sluggish performance. You are restricted to 1 socket / 4 cores and 1410MB of buffer pool memory here. The latter being the most important restriction here with VBR when it comes to performance.
This is, what I tried to emphasize with the links provided.
There is a recommended limit for VM backups with PostgreSQL: With 12.0 ist was 5k VMs which was lifted to 10k VMs with 12.1.
Current number of VM limitations - R&D Forums (veeam.com)
It was always recommended not to have more then ~500VMs with SQL-Express while there was no VM limitation with the full blown SQL.
650GB is quite a size for a VBR database which would usually be expected for way more than 10k VMs - probably we’re beyond the recommended limit for PostgreSQL here. At least you would need to specifically size the database server here according to CPU, RAM and IOPS provided.
I would be interested what the actual number of files and versions is here.
I can confirm when you use file to tape job the DB increase a lot because each file you have on all tape are also présent in database…
However I m sorry but I don't find any recommendation about sql should be préfèred to postgre in this scénario...in your links I don t find it too... sql standard was recommended before postgre because it was the only way... 10gb can t handle a lot of file to tape so you were forced to migrate if you want to continue to backup your datas.…
The only way to reduce space support gave me was to remove from catalog some tapes and restart veeam service to get back the space.
It’s only the metadata that fills up the database. The files themselves go to tape, not the database. I guess, you meant that, @damien commenge.
There is no specific recommendation to prefer MS-SQL over PostgreSQL for file to tape. But it’s not only about database size hard limitations. For SQL-Express the main problem was most of the time the sluggish performance. You are restricted to 1 socket / 4 cores and 1410MB of buffer pool memory here. The latter being the most important restriction here with VBR when it comes to performance.
This is, what I tried to emphasize with the links provided.
There is a recommended limit for VM backups with PostgreSQL: With 12.0 ist was 5k VMs which was lifted to 10k VMs with 12.1.
Current number of VM limitations - R&D Forums (veeam.com)
It was always recommended not to have more then ~500VMs with SQL-Express while there was no VM limitation with the full blown SQL.
650GB is quite a size for a VBR database which would usually be expected for way more than 10k VMs - probably we’re beyond the recommended limit for PostgreSQL here. At least you would need to specifically size the database server here according to CPU, RAM and IOPS provided.
I would be interested what the actual number of files and versions is here.
Yes all the files names are in DB as known as metadatas :)
Then I think I probably miss understand your message here "Therefore, it was always recommended to use a fully licensed and well performing MS-SQL server for VBR, when file-to-tape comes into play." because I believed you talked about sql is recommended when file to tape ! I totally agree with you about sql express limitation and can t be use on big company.
My dayabase was arround 250Gb and was pain full to perform the sql to postgreSQL migration but I can see the problem should be fixed now (change log mentioned improvement on scénario with big sql database to postgres).
To a certain extent, this is expected behavior: The reason is your file-to-tape job, secondary to your NAS job. Even with an efficient NAS job before it, it actually acts as file-to-tape AFAIK.
File-to-tape jobs generate a lot of metadata in the DB, as they put roughly the same amount of data into it per file as a VM job does per VM. Most people have way more files than VMs…
Therefore, it was always recommended to use a fully licensed and well performing MS-SQL server for VBR, when file-to-tape comes into play.
https://bp.veeam.com/vbr/2_Design_Structures/D_Veeam_Components/D_VBR_DB/database.html
https://helpcenter.veeam.com/docs/backup/vsphere/file_to_tape_jobs.html?ver=120
https://forums.veeam.com/tape-f29/file-to-tape-limitation-t62821.html
With PostgreSQL for VBR still being fairly new, there are not much recommendations here.
How many files are we talking about in your case? It’s the number of files, that counts. I would assume >>1M in your case to rectify 650GB of DB size.
To my knowledge, there is no way to reduce the size of the DB with regards to tape data - except for getting rid of it in your tape backups again.
I would engage support to let Veeam know about your usecase and make sure, that everything is setup correctly.
Best, Michael.
Hi Michael,
Thank you veery much for your thoughts and input here. I am aware that NAS to Tape Jobs are treated as “File to Tape Jobs”. But i still wonder why DB is growing that big… We will have a look into it and will check the limits and requirements again. best regards
Could you have possibly configured your NAS cache repository to use the same repository as the Veeam Config DB?
Nope checked that already.
@PHuber: Just out of curiosity, what is the total # of files in your NAS backups?
Hi
we started with around 1M. But customer added more shares recently so we need to check if that breaks the limits . We need to analyse…