As some may have seen, here is the transcript of the excitement as it happened.
2019/01/14 16:10UTC · Servers are going down for scheduled maintenance, to upgrade operating systems. They will likely be down most of today. I will post status here as things progress.
2019/01/14 16:39UTC · Universe database backup completed. Backups moved off site.
2019/01/14 18:23UTC · Server operating systems upgraded. Web server and SQL server are still in progress.
2019/01/14 20:07UTC · Web server appears to be ok. SQL server is still in progress.
2019/01/15 17:32UTC · Part way through the upgrade yesterday, the SQL server stopped accepting network logins. With help from the remote hands assistance at the computer center, we worked for four hours to get the ssh server running, without success. As it happened, the last thing I asked them to try last night worked! I was able to log in this morning and continue with the operating system upgrade. That server is rebooting now; fingers are crossed.
2019/01/15 17:41UTC · Success. I can remotely login to the SQL server again.
2019/01/15 20:31UTC · SQL server is alive and well. Web CGI programs are working. SoH Login and Mail servers are up.
2019/01/15 20:58UTC · SoH Scene servers are starting up. Wiki is still down; I'm working on it.
2019/01/15 21:12UTC · Wiki is up.
2019/01/16 17:10UTC · Upgrades are complete. A final wave of builds is finished. The next server restart will be a "normal" update and restart. With that, we are back on track for regular (weekly?) updates.
That pretty well sums it up. The SQL server had me worried Monday night. I was not looking forward to driving to Denver tomorrow, Thursday, the earliest I could have gone. It is a long drive, one we made recently at Christmas, about 9 hours each way.
2019/01/14 16:10UTC · Servers are going down for scheduled maintenance, to upgrade operating systems. They will likely be down most of today. I will post status here as things progress.
2019/01/14 16:39UTC · Universe database backup completed. Backups moved off site.
2019/01/14 18:23UTC · Server operating systems upgraded. Web server and SQL server are still in progress.
2019/01/14 20:07UTC · Web server appears to be ok. SQL server is still in progress.
2019/01/15 17:32UTC · Part way through the upgrade yesterday, the SQL server stopped accepting network logins. With help from the remote hands assistance at the computer center, we worked for four hours to get the ssh server running, without success. As it happened, the last thing I asked them to try last night worked! I was able to log in this morning and continue with the operating system upgrade. That server is rebooting now; fingers are crossed.
2019/01/15 17:41UTC · Success. I can remotely login to the SQL server again.
2019/01/15 20:31UTC · SQL server is alive and well. Web CGI programs are working. SoH Login and Mail servers are up.
2019/01/15 20:58UTC · SoH Scene servers are starting up. Wiki is still down; I'm working on it.
2019/01/15 21:12UTC · Wiki is up.
2019/01/16 17:10UTC · Upgrades are complete. A final wave of builds is finished. The next server restart will be a "normal" update and restart. With that, we are back on track for regular (weekly?) updates.
That pretty well sums it up. The SQL server had me worried Monday night. I was not looking forward to driving to Denver tomorrow, Thursday, the earliest I could have gone. It is a long drive, one we made recently at Christmas, about 9 hours each way.