How I killed my M-Files instance…and brought it back to life!

I work with M-Files since 1 year now, and to be honest the solution is pretty robust.
I had really few issues at “Server level”, but sooner and later while playing sorcerer’s apprentice you finish by breaking something.

The inevitable happened, my M-Files instance became unusable, I assume the root cause is a combination of several things:

• First I instanced too many Vaults on an undersized VM.
• Then I restarted (violently) the host while M-Files was still working on some Vaults (using embedded Firebird DBs).

As a result, impossible to launch the Admin console: infinite loading when I wanted to list the Vaults.
Same behavior with the Desktop client and the Web Interface.

My first and, naive thought was:

“OK, I applied the last monthly update and it might be related to that”.

But no one is talking about this problem in the community….so I need to check for something else.

Then I checked the logs, and I found some events like:

I was in a situation where the snake bites its tail, the problem is services cannot be stopped gracefully because some Vaults are not responding properly.
The admin tool is also not responding, I can’t bring the Vault offline neither run a “Verify and Repair”.

It’s unusual to say that, but one of the problems with M-Files’ stability is that there are very few resources (admin guide or community threads) that talk about troubleshooting and recovery.

How I fixed it

I had then to “improvise” and find a way to move back in a stable state.

1. First I changed the M-Files services startup mode to “Manual” and I restarted the host.
2. After the reboot, I moved all the folders with Vault data to a temporary location.
3. I changed back the services startup mode to “Automatic” and started M-Files.
4. I was able to access the Admin tool and list the Vault (all flagged Offline as folders are missing)
5. I moved back the Vault folder one by one, bring the Vault online and run the “Verify and Repair” on each.
6. Some of the Vaults required to be fixed:

Luckily it worked

Finally my M-Files server is back with all the Vaults running and without having to restore any backup.

In conclusion

This mishap highlighted me one thing that may have an impact. My server is hosted in the Cloud and stopped during the night. I moved the backup schedules during evening hours when the server is still up, but I forgot to re-schedule the Maintenance activities when the VM is running.

But the most important thing is to point out that it took me a year before I encountered a major incident. I have worked on several ECMs in recent years, and I remain impressed by the stability of M-Files.

Feel free to contact us for any question about M-Files.