Few days ago, through our alerting at the Service Desk, we receive an alert about a database in an AlwaysOn cluster with the message:

CRITICAL: Database xxxx [replica: xxxxxx] [role: SECONDARY] state is SUSPECT, synchronization health is NOT_HEALTHY, synchronization state is NOT SYNCHRONIZING [suspended: true] [failover ready: false] [joined: true]

After creating the ticket, I go to the environment and see with SSMS that on the secondary, the database is in the state Not Synchronizing / Suspect

Like usual, I go to the errorlog and see this error:

Error: 17204, Severity: 16, State: 1.
FCB::Open failed: Could not open file \\ xxxx \BackupSQL\Temp\Backup_Table\ xxxx.ndf for file number 307. OS error: 5(Access is denied.)

Error: 5120, Severity: 16, State: 101.

Unable to open the physical file “\\ xxxx \BackupSQL\Temp\Backup_Table\ xxxx.ndf”. Operating system error 5: “5(Access is denied.)”.

Error: 3459, Severity: 16, State: 1.

Recovery of database “xxxx” failed to redo a file add for file “\\xxxxx\BackupSQL\Temp\Backup_Table\xxxx.ndf”. Please delete the file and retry.

It seems that we have a file on a share which is not accessible, and which is putting the database in a suspicious state. If you look at the database on the primary, you can see that someone has created a data file on a share…

SQL Server has allowed you to create files on shares for years.

When you’re in an HA system with AlwaysOn AG and with automatic seeding(the problem coming from this point), adding a file on the primary will be replicated on the secondary and so in our case, it will be necessary to create the file on the same share with the same name, which isn’t possible.

The second consequence is that the Tlog file doesn’t get emptied and continues to grow despite the backup logs, because the pending synchronisation anticipates the commit… until the big alert that the Tlog disk is full…

The moral of the story is simple: before making a structural change to a database, think about how it works and the consequences. 

In any case, we’ll always be here to help! 😉