Greetings Community,
I have been working for a few weeks on parsing all things Veeam Logs, deep stuff that is for sure. On this specific Post I want to cover the Monitoring of the Capacity Tier Jobs and Tasks.
This is work in progress, so I am writing this post seeking help from you, to try it on your lab, or on your environments (always remembering it is Community Stuff and not supported)
System Requirements
- You should have Telegraf+InfluxDB+Grafana installed
- Grafana should be the version 8.0.2 - In case you have an inferior version, or superior, please just run this sudo apt-get install grafana=8.0.2
- You should have telegraf installed on the VBR, I hope properly configured sending metrics to your InfluxDB. Ping me if need anything. But it is really simple.
- The telegraf.conf at the end should contain this at the end of the file (this is the fairy dust that makes us fly :))
# Offload Job - ID and final status
[[inputs.tail]]
files = ["C:\\ProgramData\\Veeam\\Backup\\*\\Offload*.log"]
from_beginning = true
name_override = "veeam_capacitytier_offload"
watch_method = "poll"
grok_patterns = ['%{CAPACITYTIER_LOG}']
grok_custom_patterns = '''
DATESTAMP_VEEAM %{MONTHDAY}.%{MONTHNUM}.%{YEAR} %{TIME}
CAPACITYTIER_LOG \[%{DATESTAMP_VEEAM:timestamp:ts-"2.1.2006 15:04:05"}\] \<%{NUMBER:nrjob}\> %{WORD}%{SPACE}%{WORD}%{SPACE}%{WORD}%{SPACE} '%{DATA:jobid:tag}' has been completed, status: '%{DATA:status:tag}', '%{DATA:totaltransferred} %{DATA:valuetransferred:tag}' of '%{DATA:sourcetransferred} %{DATA:valuesource:tag}' bytes, '%{DATA:totaltasks}' of '%{DATA:sourcetasks}' tasks, '%{DATA:taskoverviewsuccess}' successful, '%{DATA:taskoverviewfailed}' failed
'''
data_format = "grok"
# Offload Job - Task ID and final status per VM
[[inputs.tail]]
files = ["C:\\ProgramData\\Veeam\\Backup\\SOBR_Offload_*\\*\\Task.*.log"]
from_beginning = true
name_override = "veeam_capacitytier_offload_vm"
watch_method = "poll"
grok_patterns = ['%{CAPACITYTIER_VM_JOBID_LOG}','%{CAPACITYTIER_VM_TASKID_LOG}','%{CAPACITYTIER_VM_BOTTLENECK_LOG}','%{CAPACITYTIER_VM_TASKSTATUS_LOG}'
]
grok_custom_patterns = '''
DATESTAMP_VEEAM %{MONTHDAY}.%{MONTHNUM}.%{YEAR} %{TIME}
CAPACITYTIER_VM_JOBID_LOG \[%{DATESTAMP_VEEAM:timestamp:ts-"2.1.2006 15:04:05"}\] \<%{NUMBER:nrjobid}\> %{WORD}%{SPACE}\[%{WORD}\]%{SPACE}%{WORD} '%{DATA:jobid:tag}'
CAPACITYTIER_VM_TASKID_LOG \[%{DATESTAMP_VEEAM:timestamp:ts-"2.1.2006 15:04:05"}\] \<%{NUMBER:nrtaskid}\> %{WORD}%{SPACE}Set status '%{WORD}' for task session '%{DATA:taskid:tag}', object name '%{DATA:protectedvm:tag}'
CAPACITYTIER_VM_BOTTLENECK_LOG \[%{DATESTAMP_VEEAM:timestamp:ts-"2.1.2006 15:04:05"}\] \<%{NUMBER:nrbottleneck}\> %{WORD}%{SPACE} Busy: Source %{NUMBER:source}% > Proxy %{NUMBER:proxy}% > Network %{NUMBER:network}% > Target %{NUMBER:target}%
CAPACITYTIER_VM_TASKSTATUS_LOG \[%{DATESTAMP_VEEAM:timestamp:ts-"2.1.2006 15:04:05"}\] \<%{NUMBER:nrvmtask}\> %{WORD}%{SPACE}Task session \[%{DATA:taskid:tag}] has been completed, status: %{DATA:status:tag}, %{DATA:totaltransferred} of %{DATA:sourcetransferred} bytes
'''
data_format = "grok"
After this, just restart telegraf service and it should start parsing everything, in real-time and without impacting VBR, even if parsing years of logs.
Grafana Dashboard
I will not give you something without the proper Dashboard to follow, just download the next .json and import it on your Grafana as per usual - Download now If all is as it should be, you should see something cool like this:
Dark theme
![](https://uploads-eu-west-1.insided.com/veeam-en/attachment/499a824e-2c48-4a69-a0ed-7b0deb380bd3.jpg)
Light theme
![](https://uploads-eu-west-1.insided.com/veeam-en/attachment/9b8e9d99-2b3e-469f-8906-70cd1ed4b417.jpg)
As said at the start of the post, your help would be welcome, feedback, ideas, etc. Still work in progress, and better parsing is needed, etc. Thanks guys!