Update 2008-05-30 09:20 : The rolling resart is complete (as of about 1/2 hour ago).
Update 2008-05-30 07:49 : The second-half rolling restart is 81% complete.
Update 2008-05-30 06:54 : The second-half rolling restart is 47% complete.
Update 2008-05-30 05:30 : The second-half rolling restart has begun.
Update 2008-05-29 17:00: We’re updating some of the central systems to 1.22 over the next hour or so. When we get to the updates of the hosts that store agent presence, there may be some brief disruptions of logins and the concurrency count (seen on the secondlife.com page when logged in) will briefly dip.
Update 2008-05-29 13:36: The first-half rolling restart is complete (as of 10am this morning, sorry for not updating the blog earlier.)
Update 2008-05-29 09:01: The first-half rolling restart is 69% complete.
Update 2008-05-29 07:59: The first-half rolling restart is 30% complete.
Update 2008-05-29 07:03: The rolling restart to deploy 1.22.1 to the first half of the grid has begun (a bit behind schedule).
Update 2008-05-28 12:24: We are currently deploying an updated “1.22.1” version of the server code to the Preview Grid that reverts the sound throttling change and fixes the issues with flight assists. Our plan is as follows:
- Proceed with QA validation of the changes over the afternoon of 5/28
- Update the 1.22.0 regions to 1.22.1 on the morning of 5/29
- Update the remaining 1.21 regions to 1.22.1 on the morning of 5/30
(Times/dates are Pacific Daylight Time)
Issues with 1.22:
- We are receiving scattered reports that the new sound throttling (mentioned below) is impacting content. This is not intended behavior.
- We are also receiving reports that “flight assist” scripts are not functioning as before (motion continues after keys released, and so forth). This is not intended behavior.
- We are currently investigating and evaluating our options; it is likely that we’ll need to adjust the code (or revert to a previous version) and may need to re-restart the affected regions. We’ll post more details when available.
Update 2008-05-28 09:50 : The first-half rolling restart is complete. There was a database hiccup in the middle of the roll which caused some manual cleanup to be required. Regions should all be back up.
Update 2008-05-28 08:37 : The first-half rolling restart is 75% done.
Update 2008-05-28 07:32 : The first-half rolling restart is 46% done.
Update 2008-05-28 06:32 : The beginning was rocky, but we’re on track now. The first-half rolling restart is 13% done.
Update 2008-05-28 05:42AM : The first-half rolling restart began at about 5:20AM.
Update 2008-05-27 10:00PM : it was more rocky than usual, and a handful of regions were down for about an hour or so, but we have deployed 1.22 to 150 regions. What we learned from tonight should make the rolling restart over the next two days go more smoothly.
We will be rolling out a deploy of an updated version of the Second Life server over the course of the next three days. The release notes for this, server version 1.22, are as follows:
* Changes to eliminate storage wasted by single-use, temporary assets. (Helps our asset server!)
* Introduced automatic sound playback throttling for scripts. (Should not affect existing well-behaved content, but should help reduce specific instances of griefing.)
* More specific IM-to-Email subject line: “Second Life – <name> has sent you a message”
* Corrected text encoding issues in classifieds.
* Addition of several metrics for tracking system failures. (Helps us fix more bugs.)
* Made viewer->simulator message queue length configurable at runtime (Helps address teleport failures.)
* Several changes to reduce load on central databases.
* Several Havok4 fixes. (Will be listed later tonight.)
Physics Updates (Sorry these did not make the original post… my schedule was too crammed to get this list assembled by post time. /Sidewinder):
DEV_14452: Grey goo fence enhanced
VWR-6267: Can now log into “last location when last location is above 768m
SVC-2216: Avatar no longer rotates when unsitting
DEV-13444: Physical sphere roll duration reduced (balls would roll too long)
DEV-14808: Sit target rotation corrected on Dominus Shadow
DEV-13057: Missing move object error message added (explanation for why an object did not rez due to permissions failure)
SVC-1648: Push applied to avatar by another avatar’s scripted attachment now pushes correctly (pushed minimally if at all pre-fix)
DEV-14056: Can now sit on object within a huge phantom or volume detect object
DEV-14658: Removed simulator crash mode
DEV-14646: Added higher resolution simulator performance tracking and logging to use in investigation of cyclic time dilation issues
DEV-14805: Tall avatars no longer get stuck in falling state in objects that they fall on
DEV-14057: Avatar no longer twitches and stops when changing flying direction
These changes have been in testing on the Preview Grid for the past week or so.
The rolling restart will proceed in three stages.
* Tuesday, 05/27, 7:30PM : we will do a pilot rolling restart to about 150 regions
* Wednesday, 05/28, 5:00-9:00AM : we will do a rolling restart of half of the grid
* Thursday, 05/29, 5:00-9:00AM : we will do a rolling restart of the rest of the grid
As with all rolling restarts, regions should only be down for a few minutes. If your region stays down for more than 20 minutes, it almost certainly means something went amiss; in that case, please contact support. There is no easy way to predict exactly when a given region will be restarted, and it will not be possible to delay the restart once it has been announced for your region.