[RESOLVED] Short Region Outages Planned

Further to the work carried out in late February to move some of our simulator nodes to new IP addresses, we now need to change the IP addresses for parts of our network infrastructure. This will mean 5 to 10 minute network outages for around a thousand regions at a time, spread over the next few days, as they become “disconnected” and then “reconnected” to the Second Life grid network. Note that regions will not be shut down unless necessary, but we will alert all affected regions when their times come.

(Updated 2008-03-19 03:32am PDT) We will start doing this from 3am-8am 5am-8am PDT (12noon-3pm GMT) Wednesday 19th March. Due to earlier, unrelated problems with our network infrastructure, we are pushing this work back 2 hours on Wednesday; work will commence at 3am PDT on subsequent days until completion.

When we modify the configuration of the affected network systems, it will result in 5 to 10 minutes downtime to the simulator nodes. While the work will be performed over the next two or three days, it will only affect a certain number of simulator hosts at a time, not the entire Second Life grid. If you are connected to an affected region while it is being worked on, an alert will be issued within that region, and once “disconnected”, you may still appear connected to the region, but you will be unable to teleport, rez or chat. Once the regions become “reconnected”, if you are still in that region, you will see an alert saying the outage for that region is over, and you should be able to continue activities as normal.

We’ll keep this post up-to-date with the details as we progress. (Complete 07:40am PDT) The parts of this process which affect your experience in-world have now ended and we’re marking this resolved here in the blog.

(Updated 2008-03-19 08:25am PDT) This has been delayed due to the earlier network issues, but we are pressing ahead with one round of region outages. This should take no longer than 5 to 10 minutes. We’ll update here when we’re done. (Complete 09:05am PDT) Work complete for this round; further rounds will not take that long!

(Updated 2008-03-20 05:15am PDT) Here we go again; another round of region outages as we reconfigure more equipment. (Complete 05:45am PDT) There are some issues with our process which we’re ironing out; the remaining outages will be quicker. We will continue with this process at around 06:15am PDT.

(Updated 2008-03-20 06:25am PDT) Second batch of outages for this morning. (Complete 06:35am PDT) We will continue with this process at around 07:00am PDT.

(Updated 2008-03-20 07:30am PDT) Third batch of outages for this morning. (Complete 07:41am PDT) We will continue with this process tomorrow from 03:00am PDT onwards.

(Updated 2008-03-21 04:33am PDT) Good morning, Second Life! It’s time for another round of network infrastructure updates. (Updated 2008-03-21 04:37am PDT) Ook! Something’s not right; we’re holding off on this right now. No regions have been affected at this time, and we’ll update here when we’re good to go again. (Updated 2008-03-21 07:47am PDT) We’re calling off any updates this morning; we will continue with this process tomorrow (Saturday) from 03:00am PDT onwards.

(Updated 2008-03-22 06:20am PDT) We are continuing our network reconfiguration work. The outage earlier this morning was due to some unexpected issues related to the reconfiguration. Again, we have reviewed the process and further outages will be planned and stable. (Complete 07:40am PDT) The parts of this process which affect your experience in-world have now ended and we’re marking this resolved here in the blog.

This entry was posted in -Miscellaneous, Announcements & News, Bugs & Fixes, Operations. Bookmark the permalink.