• EN
Solved

Veeam Oracle 8 Linux Agent snapshot error

T
Userlevel 2

Hello,

we are just some newbies in Veeam and have following problem.

The Veeam Linux Agent 12.1 is correctly installed on an Oracle Linux Machine i have tried both methods, manually and per install service from Veeam Backup and Replication Server, the installation were completed with DKMS and build step what is workig very well. If i try to make ab backup of the machine i receive errors in Serverlog as follows:

creating volume snapshot;

than

Failed to take snapshot;

Broken Pipe;

Failed to write data into pipe.

the linux log:

[29.01.2024 08:01:29.072] vmb | [SessionLog][processing] Creating volume snapshot.
[29.01.2024 08:01:29.073] lpbcore | Polling request: [job][UpdateManagedJobXmlLog]
[29.01.2024 08:01:29.780] lpbcore | Polling request: [job][UpdateManagedJobXmlLog]
[29.01.2024 08:01:29.780] lpbcore | Created snapshot [b9e92c64-5562-479d-8e2a-625ead940a8f]
[29.01.2024 08:01:29.783] lpbcore | Finish oracle dbs backup
[29.01.2024 08:01:36.774] lpbcore | PostgreSQL is processed.
[29.01.2024 08:01:36.774] lpbcore | PostgreSQL is processed. ok.
[29.01.2024 08:01:36.774] vmb | [SessionLog][error] Failed to take volume snapshot.
[29.01.2024 08:01:36.775] lpbcore | Polling request: [job][UpdateManagedJobXmlLog]
[29.01.2024 08:01:36.775] lpbcore | Taking snapshots. Failed.
[29.01.2024 08:01:36.777] | Thread finished. Role: 'Keep alive VBR notifier (polling).'.
[29.01.2024 08:01:36.777] | Thread finished. Role: 'Session checker (via polling queue) for job: [{c97c069a-8915-4524-893c-b608e3e73035}].'.
[29.01.2024 08:01:36.777] lpbcore | ManagedBackupJobPerformer: Creating backup. Failed.
[29.01.2024 08:01:36.777] vmb | [SessionLog][error] Failed to perform managed backup.
[29.01.2024 08:01:36.778] lpbcore | Polling request: [job][UpdateManagedJobXmlLog]
[29.01.2024 08:01:36.779] vmb | [SessionLog][error] Broken pipe.
[29.01.2024 08:01:36.780] lpbcore | Polling request: [job][UpdateManagedJobXmlLog]
[29.01.2024 08:01:36.780] vmb | [SessionLog][error] Failed to write data into pipe.
[29.01.2024 08:01:36.781] lpbcore | Polling request: [job][UpdateManagedJobXmlLog]
[29.01.2024 08:01:36.782] vmb | [SessionLog][error] Processing finished with errors at 2024-01-29 07:01:36 UTC.
[29.01.2024 08:01:36.783] lpbcore | ERR |Broken pipe
[29.01.2024 08:01:36.783] lpbcore | >> |Failed to write data into pipe.
[29.01.2024 08:01:36.783] lpbcore | >> |--tr:Failed to write into pipe. Data size: [339718].
[29.01.2024 08:01:36.783] lpbcore | >> |--tr:in std::streamsize io::TPipeIoDev::write(const byte_t*, size_t) [with TPipe = boost::shared_ptr; std::streamsize = long int; byte_t = char; size_t = long unsigned int] at /mnt/Sources/CommonLib/posix/../io/TPipeIoDev.h:33
[29.01.2024 08:01:36.783] lpbcore | >> |--tr:Failed to execute method [2] for class [lpbcorelib::oracle::IOracleProcessing].
[29.01.2024 08:01:36.783] lpbcore | >> |--tr:in void remoting::CStubBase::ExecuteMethod(remoting::method_id_t, remoting::IStub::SDispatchContext&) const [with TStub = lpbcorelib::interaction::proxystub::COracleProcessingStub; TObject = lpbcorelib::oracle::IOracleProcessing; remoting::method_id_t = unsigned int] at /mnt/Sources/LPBCoreLib/lpbcorelib/interaction/proxystub/../../../../ProtocolLib/remoting/StubBase.h:77
[29.01.2024 08:01:36.783] lpbcore | >> |--tr:Failed to invoke method [2] in class [lpbcorelib::oracle::IOracleProcessing].
[29.01.2024 08:01:36.783] lpbcore | >> |--tr:in void remoting::CProxyBase::CMethodInvocationStream::FinalizeExc() [with TObject = lpbcorelib::oracle::IOracleProcessing] at /mnt/Sources/LPBCoreLib/lpbcorelib/interaction/proxystub/../../../../ProtocolLib/remoting/ProxyBase.h:92
[29.01.2024 08:01:36.783] lpbcore | >> |--tr:COracleBackupObject: Failed to execute postThaw
[29.01.2024 08:01:36.783] lpbcore | >> |--tr:in virtual void lpbcorelib::oracle::COracleBackupObject::postThaw() at /mnt/Sources/LPBCoreLib/lpbcorelib/oracle/OracleBackupObject.cpp:160
[29.01.2024 08:01:36.783] lpbcore | >> |--tr:Failed to take volume snapshot.
[29.01.2024 08:01:36.783] lpbcore | >> |--tr:in static lpbcorelib::interaction::ISnapshotResourceLockPtr lpbcorelib::backup::CSnapshotHelper::TakeSnapshot(vmbplatform::model::CBackupJobPtr, const std::__cxx11::list&, const object_id_t&, vmbplatform::model::CSessionLoggerPtr, lpbcorelib::backup::CCbtDataContainer&, bool) at /mnt/Sources/LPBCoreLib/lpbcorelib/backup/snapshot/SnapshotHelper.cpp:142
[29.01.2024 08:01:36.783] lpbcore | >> |Managed backup job has failed.
[29.01.2024 08:01:36.783] lpbcore | >> |--tr:in virtual void lpbcorelib::backup::CManagedBackupJobPerformer::execute() at /mnt/Sources/LPBCoreLib/lpbcorelib/backup/ManagedBackupJobPerformer.cpp:205
[29.01.2024 08:01:36.783] lpbcore | >> |An exception was thrown from thread [140611777447680].
[29.01.2024 08:01:36.786] | Closing socket device.
[29.01.2024 08:01:36.786] lpbcore | LpbManSession: Processing commands. ok.
[29.01.2024 08:01:36.786] | Closing socket device.
[29.01.2024 08:01:36.786] | Thread finished. Role: 'lease keeper'.
[29.01.2024 08:01:39.316] lpbcore | JOB STATUS: FAILED.
[29.01.2024 08:01:39.316] | Thread finished. Role: 'Managed volume backup job execution'.
[29.01.2024 08:01:39.316] | Closing socket device.
[29.01.2024 08:01:39.317] jobman | Application session has been finished.

Any idea maybe?

Best regards and Thank You in advance.
Thomas

icon

Best answer by thomas.soporowski 19 February 2024, 10:16

View original

17 comments

coolsport00
Userlevel 7
Badge +17

Hi @thomas.soporowski -

Welcome to the Community Hub!

Have you gone through, thoroughly...the sys requirements for Linux Agent? Have you installed kernel-uek-devel? Is python3 installed?

Also, are you using Oracle 8 (UEK R7)? It appears there’s a Veeam KB for this version.

Have you made sure your system does not have: hcdriver , snapapi26 , hcdriver , snapapi , snapper , dattobd , dattobd-dkms , dkms-dattobd, cdr or cxbf?

Did you update your system before the install? (e.g. sudo dnf update )?

And, to confirm, the install worked fine, but attempting to run a backup fails?

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
T
Userlevel 2

Hi, thank you for the hints,

all of them has been checked, the installation of agent was also sucsessful

 dkms status
veeamsnap/6.1.0.1498, 5.15.0-202.135.2.el8uek.x86_64, x86_64: installed

 

Systeminfo

System information:
Running [cat /etc/*release].
Oracle Linux Server release 8.9
NAME="Oracle Linux Server"
VERSION="8.9"
ID="ol"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="8.9"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Oracle Linux Server 8.9"
CPE_NAME="cpe:/o:oracle:linux:8:9:server"
Red Hat Enterprise Linux release 8.9 (Ootpa)
Oracle Linux Server release 8.9

From the listed Modules there are no on eof them installed

 

 yum list installed | grep -E 'hcdriver|snapapi|snapapi26|snapper|dattobd|dattobd-dkms|dkms-dattobd|cdr|cxbf'
#

[30.01.2024 09:39:33.589] <139864696919808> lpbcore  | ERR |Broken pipe
[30.01.2024 09:39:33.589] <139864696919808> lpbcore  | >>  |Failed to write data into pipe.
[30.01.2024 09:39:33.589] <139864696919808> lpbcore  | >>  |--tr:Failed to write into pipe. Data size: [372404].
[30.01.2024 09:39:33.589] <139864696919808> lpbcore  | >>  |--tr:in std::streamsize io::TPipeIoDev<TPipe>::write(const byte_t*, size_t) [with TPipe = boost::shared_ptr<posix::CPipe>; std::streamsize = long int; byte_t = char; size_t = long unsigned int] at /mnt/Sources/CommonLib/posix/../io/TPipeIoDev.h:33
[30.01.2024 09:39:33.589] <139864696919808> lpbcore  | >>  |--tr:Failed to execute method [2] for class [lpbcorelib::oracle::IOracleProcessing].
[30.01.2024 09:39:33.589] <139864696919808> lpbcore  | >>  |--tr:in void remoting::CStubBase<TStub, TObject>::ExecuteMethod(remoting::method_id_t, remoting::IStub::SDispatchContext&) const [with TStub = lpbcorelib::interaction::proxystub::COracleProcessingStub; TObject = lpbcorelib::oracle::IOracleProcessing; remoting::method_id_t = unsigned int] at /mnt/Sources/LPBCoreLib/lpbcorelib/interaction/proxystub/../../../../ProtocolLib/remoting/StubBase.h:77
[30.01.2024 09:39:33.589] <139864696919808> lpbcore  | >>  |--tr:Failed to invoke method [2] in class [lpbcorelib::oracle::IOracleProcessing].

How can i better understand what does he mean with Broken Pipe is there anywhere security/access issue?

the module is lbpcore may be there is a possibilty to get out on why the Pipe is broken and waht does it particulary mean?

Best regards

Thomas 

 

 

coolsport00
Userlevel 7
Badge +17

What filesystem are you using on the Volume you’re attempting to backup?

Also, run:  lsmod | grep veeam ; if nothing returns, run: modprobe veeamsnap

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
coolsport00
Userlevel 7
Badge +17

Another thing I’d like to see if kernel headers is installed. For Debian systems, the cmd is:

dpkg-query -s linux-headers-generic

I think for RH-based systems it’s: sudo yum install kernel-headers-$(uname -r) or:

yum list kernel-uek-devel-$(uname -r) 

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
T
Userlevel 2

hi, thanks,

so we have xfs file system → e.g. /dev/sdc on /ora_home type xfs

lsmode

# lsmod | grep veeam
veeamsnap             237568  0

# yum list kernel-uek-devel-$(uname -r)

kernel-uek-devel.x86_64 5.15.0-202.135.2.el8uek

 

Today i have removed all of the veeam software and installed everything new, but nothing is better all the same behaviour.

On this server we have an Oracle Server, and i tried the Veeam RMAN Agent and this is working very well, i can make backup and restore with no problems, very strange.

 

coolsport00
Userlevel 7
Badge +17

Hmm..ok. The only other potential issue I found was secure boot. If you have this enabled, it is recommended to disable then try a backup and see if it works.

Otherwise, at this point, I recommend getting ahold of Support if you want to use the Linux Agent. But, if the RMAN Agent works for you, you may just want to stick with that.

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
coolsport00
Userlevel 7
Badge +17

Hi @thomas.soporowski -

I never did ask...is this a physical server? I presume it is because of you’re wanting to use Agents, but it was never explicitly shared it was.

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
T
Userlevel 2

Hi Shanon, thank You,

it is not a physical machine, its VM, but running on Oracle KVM solution, thats why we can’t use VM snapshots, all this stuff with Oracle make some strange troubles :-).

I thought about this issue again, is it possible that the headers are the cause for this troubles?

The snapshot modul must be run very near of kernel/hardware so if the headers are unsuitable to it, than it ‘s failing, maybe i have to remove the headers and install them again?

As you mentioned before, i have already opened a ticket by Veeam Support.

Have a nice day

 

coolsport00
Userlevel 7
Badge +17

Hi Thomas... Yes, it very well could be. The main culprits I've seen causing issues like this are dkms, veeamsnap, & headers pkgs. Give it a try and see if it works. 

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
coolsport00
Userlevel 7
Badge +17

I found a forum post where the poster had similar issue on a few systems, and each system was different. On one system, secure boot was enabled; he ended up disabling it and backup worked; on another system, even though he had veeamsnap installed, it wasn’t done so ‘properly’, so he reinstalled it and backup worked; on the last system, he was running an unsupported OS (Rocky Linux), even though it’s a “flavor” of RHEL. To get it working, he modified the OS ID value in /etc/os-release, and found veeamsnap will not load if it finds "rocky" instead of "rhel". So I think he changed the value there (ro rhel?) to make it work.

Lastly, which kernel are you running (run: uname -r)? You may be on a newer, or older, version which isn’t supported. 

Thanks.

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
T
Userlevel 2

Hi Shanon, its very wired, as is said i have remove updates and tried again, all same nothing was better than i have created a new job to just copy some files as backup, this has worked, also i can do backup of the devices  /dev/sda etc, the problem seems to be “Entire Computer” and “Application Awarness”, there is something wrong, Veeams support has adviced me to just create a backup without application or to disable the processing of application, and -->it’s running…..no comments

So, now they have to find out what the problem really is with the Application...

For me very frustrating at all…….

Thank You again, best regards Thomas

coolsport00
Userlevel 7
Badge +17

Ahh..ok. So, it does work, just not a couple ‘feature sets’ of the job (AAIP and Entire Computer).

Ok. Thanks for the update from Support. At this point, I think I’ve exhausted all my ideas. Hopefully Support will come up with something. Sounds like it’s a bug with the Agent on Oracle Linux.

Best.
Shane

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
T
Userlevel 2

Hi Shanon, i thank you for all the hints and tips, i think i will wait what the Veeam Support find out.

For now i have tried all possibillities and try to configure alternative solutions out.

If we find a solution i will you an update here.

Best regards

 

coolsport00
Userlevel 7
Badge +17

Yes Thomas, please do share what they end of doing to resolve this for you.

Thank you.

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
T
Userlevel 2

So i have found the solution for this issue.

The Veeam support told me the error is coming from the RMAN Backup.

I checked it, what i found out: in the preparation stack of the full backup job, the RMAN Client is trying to check if he have an access to Oracle Database, this may fail if you try to backup two Oracle Servers with the same access account that is stored on the Veeam Backup Server in accountwallet.

After i have create two different root and oracle user accounts one per server the error was gone away. now the backup is working fine.

coolsport00
Userlevel 7
Badge +17

Wow...interesting cause noted by support. Seems odd. But, glad they got your issue sorted. Thanks for the update Thomas. 

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
coolsport00
Userlevel 7
Badge +17

Hi @thomas.soporowski - I think you’re all sorted via Support. As such, I would go ahead and mark your comment as ‘Best Answer’ so others with a similar issue as yours who come across your post may benefit with your solution.

Thanks.

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00

Comment


Terms & Conditions