Skip to main content
Solved

Unable to restore mailbox items that contain attachments


Forum|alt.badge.img
  • Comes here often
  • 21 comments

When I using Veeam explorer to restore mailbox items with attachments to a user’s mailbox on On- premise Exchange Server the restore job fail with the below error:
"Failed to create items. The underlying connection was closed: An unexpected error occurred on a send. Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host."

Any ideas?
 

Best answer by Arin

So I managed to solve this one. The problem was that the connection was going over a load balancer which does SSL offloading and then connects to exchange over port 80.
If I enter the hostname of the Exchange server itself in the “restore to..” wizard, to attachment restore works fine- from backup server directly to the exchange server.

View original
Did this topic help you find an answer to your question?

20 comments

Chris.Childerhose
Forum|alt.badge.img+21
  • Veeam Legend, Veeam Vanguard
  • 8401 comments
  • January 19, 2024

Need more details other than that error as it does not help to narrow things down.  What version of VB365 are you using?  Latest patch?

Check the logs - C:\ProgramData\Veeam\xxxxxx


coolsport00
Forum|alt.badge.img+20
  • Veeam Legend
  • 4109 comments
  • January 19, 2024

My first thought here is networking issue of some kind. Are you restoring over a WAN connection? Or, to a location where connection can be sketchy? Veeam has a KB on part of this error:

https://www.veeam.com/kb1781

I’d try a ‘perpetual ping’ to see if you have any dropped packets to the location you’re restoring to for starters. You can also look at Veeam logs to see if anything more definitive shows there: C:\ProgramData\Veeam\Backup. Another potential issue could be Antivirus software?


Chris.Childerhose
Forum|alt.badge.img+21
  • Veeam Legend, Veeam Vanguard
  • 8401 comments
  • January 19, 2024

Unsure if this will help but was related to SSL expired certs (may not be related but comes up during search of the main error you mention in the OP) - (1) The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel (veeam.com)


Forum|alt.badge.img
  • Author
  • Comes here often
  • 21 comments
  • January 19, 2024

VBR 12.1420.  Target is on-premise Exchange 2016 mailbox.

I can confirm its any item with attachment that is failing with the same error. Tried different mailboxes, even tried restoring from an entirely new backup. Restore is over the LAN. If I restore to .pst it works fine, the items are all there. If I restore to the mailbox, once the item has an attachment it fails..


MicoolPaul
Forum|alt.badge.img+23
  • 2358 comments
  • January 19, 2024

Check your VBR logs for restoration information but also check the logs and health of your CAS server.

 

The restore data flow is port 443 to the EWS connecter on the CAS server you’re targeting.


Chris.Childerhose
Forum|alt.badge.img+21
  • Veeam Legend, Veeam Vanguard
  • 8401 comments
  • January 19, 2024

I get 12.0.1420 but which patch based on this KB - KB2680: Build Numbers and Versions of Veeam Backup & Replication

Also check the logs as mentioned as well to get further details.  This might come down to a Support ticket, but we will try to help our best.


coolsport00
Forum|alt.badge.img+20
  • Veeam Legend
  • 4109 comments
  • January 19, 2024

Hmm..yeah, not sure what else to suggest. May need to hit up support at this point. 


CarySun
Forum|alt.badge.img+7
  • Veeam Vanguard
  • 200 comments
  • January 20, 2024

What’s the vbr license? Restore of an entire mailbox or selected mailbox items to the original location is available with Veeam Universal License. When using a legacy socket-based license, the Enterprise or Enterprise Plus editions of Veeam Backup & Replication is required.

Also, there are some restore limitations when you are using the Veeam explorer for Microsoft exchange to restore.

https://helpcenter.veeam.com/docs/backup/explorers/vex_considerations.html?ver=120
 


Forum|alt.badge.img
  • Author
  • Comes here often
  • 21 comments
  • January 20, 2024

Enterprise plus edition.

Is there a name for the restore log file? Im checking programdata\veeam but not seeing it in particular..


Chris.Childerhose
Forum|alt.badge.img+21
  • Veeam Legend, Veeam Vanguard
  • 8401 comments
  • January 20, 2024
Arin wrote:

Enterprise plus edition.

Is there a name for the restore log file? Im checking programdata\veeam but not seeing it in particular..

Under the backup folder.


CarySun
Forum|alt.badge.img+7
  • Veeam Vanguard
  • 200 comments
  • January 20, 2024
Arin wrote:

Enterprise plus edition.

Is there a name for the restore log file? Im checking programdata\veeam but not seeing it in particular..

%ProgramData%\Veeam\Backup\<Veeam_Explorer_Name>\Log


PeteSteven
Forum|alt.badge.img+2
  • Experienced User
  • 47 comments
  • January 22, 2024

Have you checked that the ports on the firewall are getting through? In this case 443?
https://helpcenter.veeam.com/docs/backup/explorers/vex_ports.html?ver=120


Forum|alt.badge.img
  • Author
  • Comes here often
  • 21 comments
  • January 23, 2024

I dont think its a firewall issue since all mailbox items that do not contain attachments are restored fine. 


Forum|alt.badge.img
  • Author
  • Comes here often
  • 21 comments
  • January 23, 2024

Log file output - 


  13:27:11.176]    1 (40672)  Use specified Exchange server: owa.companyx.com
[ 13:27:11.239]   14 (45580) Initializing Exchange Web Services...
[ 13:27:11.239]   14 (45580) Connecting to Exchange Web Services (server: owa.companyx.com, account: <null>, ID: 248d9923-313a-4da9-a75a-52a8381a7f22)...
[ 13:27:11.239]   14 (45580) Microsoft Exchange Web Services binding path: https://owa.companyx.com/EWS/Exchange.asmx
[ 13:27:11.239]   14 (45580)  Opening root folder using impersonation...
[ 13:27:11.266]   14 (45580)  Root folder opened successfully
[ 13:27:11.266]   14 (45580) Exchange Web Services initialized
[ 13:27:11.273]   14 (45580) Counting items...
[ 13:27:11.276]   14 (45580)  found 1 item(s)
[ 13:27:11.276]   14 (45580) Restoring items...
[ 13:27:11.561]   14 (45580) Creating 1 items (total size: 871991 bytes)...
[ 13:27:11.608]   14 (45580) Error: Failed to create items.
[ 13:27:11.608]   14 (45580) Type: Veeam.Ews.Internal.ExNetworkException
[ 13:27:11.608]   14 (45580) Stack:
[ 13:27:11.608]   14 (45580)    at Veeam.Ews.ExError.Throw(WebException error, String format, Object[] args)
[ 13:27:11.608]   14 (45580)    at Veeam.Ews.ExError.Catch(Action action, String format, Object[] args)
[ 13:27:11.608]   14 (45580)    at Veeam.Ews.ExError.Catch[T](Func`1 func, String format, Object[] args)
[ 13:27:11.608]   14 (45580)    at Veeam.Ews.ExItemOperations.Execute[T](Func`1 action, String errorMessage, String logFailedFormat, Int32 expectedRespNum)
[ 13:27:11.608]   14 (45580)    at Veeam.Ews.ExItemOperations.CreateItems(IEnumerable`1 data)
[ 13:27:11.608]   14 (45580)    at Veeam.Exchange.Restore.Restore.Retry.ExFolderRetry.CreateItems(IEnumerable`1 data)
[ 13:27:11.608]   14 (45580)    at Veeam.Exchange.Restore.ItemsPageCreator.CreateItems(ItemsPage itemsPage, IExFolder targetFolder, Dictionary`2 timedOutItems, Boolean perItemMode, Boolean& timeoutOccurred)
[ 13:27:11.608]   14 (45580) Error: The underlying connection was closed: An unexpected error occurred on a send.
[ 13:27:11.608]   14 (45580) Type: System.Net.WebException
[ 13:27:11.608]   14 (45580) Stack:
[ 13:27:11.608]   14 (45580)    at System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request)
[ 13:27:11.608]   14 (45580)    at Veeam.Ews.ExBinding.GetWebResponse(WebRequest request)
[ 13:27:11.608]   14 (45580)    at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
[ 13:27:11.608]   14 (45580)    at EWServices.ExchangeServiceBinding.UploadItems(UploadItemsType UploadItems1)
[ 13:27:11.608]   14 (45580)    at Veeam.Ews.ExItemOperations.<>c__DisplayClass10_0.<CreateItems>b__1()
[ 13:27:11.608]   14 (45580)    at Veeam.Ews.ExError.<>c__DisplayClass7_0`1.<Catch>b__0()
[ 13:27:11.608]   14 (45580)    at Veeam.Ews.ExError.Catch(Action action, String format, Object[] args)
[ 13:27:11.608]   14 (45580) Error: Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host.
[ 13:27:11.608]   14 (45580) Type: System.IO.IOException
[ 13:27:11.608]   14 (45580) Stack:
[ 13:27:11.608]   14 (45580)    at System.Net.Sockets.NetworkStream.MultipleWrite(BufferOffsetSize[] buffers)
[ 13:27:11.608]   14 (45580)    at System.Net.Security._SslStream.StartWriting(SplitWritesState splitWrite, SplitWriteAsyncProtocolRequest asyncRequest)
[ 13:27:11.608]   14 (45580)    at System.Net.Security._SslStream.ProcessWrite(BufferOffsetSize[] buffers, SplitWriteAsyncProtocolRequest asyncRequest)
[ 13:27:11.608]   14 (45580)    at System.Net.TlsStream.MultipleWrite(BufferOffsetSize[] buffers)
[ 13:27:11.608]   14 (45580)    at System.Net.Connection.Write(ScatterGatherBuffers writeBuffer)
[ 13:27:11.608]   14 (45580)    at System.Net.ConnectStream.ResubmitWrite(ConnectStream oldStream, Boolean suppressWrite)
[ 13:27:11.608]   14 (45580) Error: An existing connection was forcibly closed by the remote host
[ 13:27:11.608]   14 (45580) Type: System.Net.Sockets.SocketException
[ 13:27:11.608]   14 (45580) Stack:
[ 13:27:11.608]   14 (45580)    at System.Net.Sockets.Socket.MultipleSend(BufferOffsetSize[] buffers, SocketFlags socketFlags)
[ 13:27:11.608]   14 (45580)    at System.Net.Sockets.NetworkStream.MultipleWrite(BufferOffsetSize[] buffers)
[ 13:27:11.608]   14 (45580) Connection closed by the remote host. Starting item-by-item creation for current items page...
[ 13:27:11.611]   14 (45580) Connection 248d9923-313a-4da9-a75a-52a8381a7f22 is closed
[ 13:27:11.611]   14 (45580) Connecting to Exchange Web Services (server: owa.companyx.com, account: <null>, ID: 011b92ad-2069-469e-9528-0671ff2e7d4a)...
[ 13:27:11.611]   14 (45580) Creating 1 items (total size: 871991 bytes)...
[ 13:27:11.619]   14 (45580) Warning: Exchange Web Services request failed
[ 13:27:11.619]   14 (45580) Warning: Failed to create items.
[ 13:27:11.619]   14 (45580) Warning: The underlying connection was closed: An unexpected error occurred on a send.
[ 13:27:11.619]   14 (45580) Warning: Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host.
[ 13:27:11.619]   14 (45580) Warning: An existing connection was forcibly closed by the remote host


Chris.Childerhose
Forum|alt.badge.img+21
  • Veeam Legend, Veeam Vanguard
  • 8401 comments
  • January 23, 2024

I think at this point your best option is to contact Support as there is not going to be much more, we can do here on the community site.


Forum|alt.badge.img
  • Author
  • Comes here often
  • 21 comments
  • January 23, 2024

Investigating further,
attachments in drafts and sent items are able to be restored. 
 It seems only attachments that are in the inbox give the error.

 

Yes indeed, looking like something Veeam support needs to look at for sure


coolsport00
Forum|alt.badge.img+20
  • Veeam Legend
  • 4109 comments
  • January 24, 2024

Keep us posted what they find out.

Thanks.


Forum|alt.badge.img
  • Author
  • Comes here often
  • 21 comments
  • Answer
  • February 1, 2024

So I managed to solve this one. The problem was that the connection was going over a load balancer which does SSL offloading and then connects to exchange over port 80.
If I enter the hostname of the Exchange server itself in the “restore to..” wizard, to attachment restore works fine- from backup server directly to the exchange server.


coolsport00
Forum|alt.badge.img+20
  • Veeam Legend
  • 4109 comments
  • February 1, 2024

Great investigative work @Arin ...and thank you for sharing what you did to resolve it.


coolsport00
Forum|alt.badge.img+20
  • Veeam Legend
  • 4109 comments
  • February 7, 2024

Hi @Arin - since you resolved this on your own, I recommend selecting what you did as the ‘Best Answer’ so others who experience the same issue and come across your post know what best to do to address it.

Thanks.


Comment