[Update 2008-04-10 22:00]: The central servers have been updated. However, the rest of the deploy will not proceed this week. We will do a rolling restart next week to deploy the fixes in this patch.
[Update 2008-04-09]: We found a very minor hitch as we began the deploy– this is why we do these things cautiously and in stages now. We have postponed the rolling restart again. Watch this space for updates as to when it will actually happen.
[Update 2008-04-09]: The rolling restart will begin tonight. At 18:30, we will update the central servers. There will be brief in-world service interruptions (e.g. group chats, profiles, friends online) as the servers are restarted. After this is done, we will deploy the new server version to 500 regions. The scheduling below has been updated. –Prospero Linden
[UPDATE 2008-04-08]: This will be delayed by at least one more day, as we continue internal testing. The earliest projected roll-out is on Wednesday/Thursday (i.e. a one day slip in schedule). — Joshua Linden
We will be doing a rolling restart this Tuesday and Wednesday (April 8 and 9) to roll out server version 1.21. We will follow our standard procedure of doing the rolling restart in stages, so that we will have the opportunity to detect any serious production problems before the whole grid is affected. The schedule will be:
- [updated] Wednesday 04/09, 18:30-? : Server 1.21 will be deployed to the central servers and to ~500 regions.
- [updated] Thursday 04/10, 05:30-07:00 : Sever 1.21 will be deployed to an additional ~2000 regions.
- [updated] Thursday 04/10, 18:00 : The deploy to the full grid will begin, and will take 5-6 hours to complete.
During the rolling restart, each region will be down for a few minutes while it restarts. If your region stays down for more than 20 minutes during the restart, you can contact support to find out if there was a problem with it. Residents in-world will receive warnings beginning five minutes before the region they are in is restarted. Currently, it is not possible to delay the restart of a given region.
Two things are included in this server deploy. First, there are performance and security patches. This includes the memory allocation issue that has led to some increased server lag in void (openspace) sims. Sidewinder Linden will post a separate blog entry with the details of these fixes. Second, we will be rolling out code that changes the way regions are assigned to hosts, although this change will be initially disabled. This new code is not visible to residents; however, it adds tools that allow us to manage database load more effectively. The goal is to partially alleviate the problems we’ve been having recently near peak concurrency.
(None of these changes are related to the long outage we had last Saturday; that was a networking issue with our networking provider, and didn’t have anything to do with the Second Life code itself.)