TLDR: Make sure you do not configure virtual disks to grow dynamically beyond the size that is actually available on the disk where your virtual disk resides.
Ok, this has eaten away the last day and a half. I have a Sitecore development environment running in a VirtualBox solution using a virtual disk. The virtual disk itself resides on a physical disk (obviously). Anyhow, I needed to restore recent Sitecore database backups and strange things started to happen…
The Core and Web databases restored just fine, but the Master database complained about incorrectly formed media types or some equally cryptic SQL Server error. I restarted downloading the backup since something could have gone wrong in the transfer. While this happened (90 minutes wait time) I reverted to a local Master backup to keep moving.
But I immediately got errors trying to install a Sitecore package, and looking at the log files was unsettling:
ManagedPoolThread #19 2017:12:14 13:37:58 ERROR One or more exceptions occurred while processing the subscribers to the ‘item:saved’ event.
Message: SQL Server detected a logical consistency-based I/O error: incorrect pageid (expected 1:320622; actual 0:0). It occurred during a read of page (1:320622) in database ID 5 at offset 0x0000009c8dc000 in file ‘C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\DATA\Sitecore_Core.mdf’. Additional messages in the SQL Server error log or system event log may provide more detail. This is a severe error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online.
Source: .Net SqlClient Data Provider
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at System.Data.SqlClient.SqlDataReader.TryHasMoreRows(Boolean& moreRows)
at System.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more)
at Sitecore.Links.SqlLinkDatabase.GetLinks(String sql, Item item, Object parameters)
at Sitecore.Data.DataProviders.NullRetryer.ExecuteNoResult(Action action, Action recover)
at Sitecore.Links.SqlLinkDatabase.UpdateItemVersionLink(Item item, ItemLink contextitemLinks)
at Sitecore.Links.ItemEventHandler.OnItemSaved(Object sender, EventArgs args)
at Sitecore.Events.Event.EventSubscribers.RaiseEvent(String eventName, Object parameters, EventResult result)
“This is a severe error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB).”
Not pleasant. Running a DBCC CHECKDB took forever on the Core database and ultimately informed about a few hundred thousand “consistency errors”.
After quite some time of unsuccessful messing around with this problem I randomly noticed that the hard drive that hosted my virtual disk was full. Well, in that case it doesn’t seem so strange that I get problems with large database files, now does it?
But why didn’t I get any warnings about this in my virtual environment? I’ll tell you. The virtual disk was configured to grow dynamically to 500 GB and the hosting disk also had some other stuff on it that prevented this dynamic growth from happening. Apparently this does not cause errors in the virtual environment about low disk space, since that environment thought I had plenty of room left on my disk. So instead the files started to break when they ran out of available space to exist on.
So if the virtual disk had more room to grow to its configured maximum size, this bizarre behaviour would never had occurred.
Moral: DO NOT CONFIGURE A VIRTUAL DISK TO GROW DYNAMICALLY IF THERE IS NOT ENOUGH SPACE.