[UPDATED] Rolling restart to deploy 1.23.3, Thu-Wed July 24-30

Update 2008-07-30 07:48am : But for a small amount of residual clean-up, the rolling restart is complete.

Update 2008-07-30 05:45am : The second-half rolling restart has begun. Today, all regions still on 1.22.4 will be restarted, upgrading them to 1.23.4.

Update 2008-07-29 09:08am : The first-half rolling restart is complete, but for a small amount of cleanup.

Update 2008-07-29 05:09am : The rolling restart of even numbered hosts has begun.

Update 2008-07-25 10:04pm : Those regions (about 1/10 of Second Life) that had been running server version 1.23.3 are now running server version 1.23.4. (90% of the grid remains on 1.22.4.)

Update 2008-07-25 03:32pm : We have found and corrected some bugs in the code. These were very rare crash modes (the crash rate on 1.23.3, even though a bit higher than 1.22.4, still corresponds to a mean time between server crashes of nearly a month). As such, they’re not easily reproducible, and we’re not 100% sure that the bugs found were responsible for the crashes we saw (although it’s likely). Our plan is to roll out 1.23.4 to the pilot group of hosts and regions currently running 1.23.3; that will happen tonight at 7:30 or 8:00PM. The full roll will happen early next week. The schedule below has been updated.

Update 2008-07-25 05:21am : We have decided to delay further rollout of 1.23.3, but we are not going to roll back the regions currently on 1.23.3. The simulator crash rate is not alarmingly higher, but we have seen a couple of new crash modes that we want to investigate before rolling out 1.23.3 further. We will update more as we know more.

Update 2008-07-24 01:22 : The rolling restart to deploy 1.23.3 to 1/10 of the grid is now complete.

We have found and fixed the bug that was causing an increased crash rate in Server version 1.23.2, the version that was initially rolled out earlier this week.

We will be doing a pilot roll to about 1/10 of the grid very shortly. We will watch the crash rate of servers, and look for other problems, throughout the rest of the day. If all goes well, we will deploy 1.23.3 to the rest of the grid according to the schedule:

Schedule Updated

  • Fri evening 07/25, 7:30-9:30PM : the 1/10 of the grid running 1.23.3 will be updated to 1.23.4.
  • Tue morning 07/29, 5-10AM : one half of the grid;
  • Wed morning 07/30, 5-10AM : the rest of the grid.

Please see the earlier blog posts (here, here and here) for a list of fixes and upgrades included in server version 1.23.

As with all rolling restarts, regions will receive warnings starting five minutes before they restart. Regions will only be down for 5-10 minutes. If your region stays down for more than 20 minutes, please contact support.

This entry was posted in -Miscellaneous. Bookmark the permalink.

149 Responses to [UPDATED] Rolling restart to deploy 1.23.3, Thu-Wed July 24-30

  1. SeanMcPherson Senior says:

    Good luck 🙂

  2. Jack says:

    The “fun” continues…

    Odd or even tomorrow?

  3. Darien Caldwell says:

    Was the double script time issue fixed as well? My bot logged quite clearly that script time in the sim doubled during the short time the last version was up. I hope this won’t be a repeat of that.

  4. Prospero Linden says:

    The slow script performance was almost certainly due to the error in building the code with debug symbols; as such, yes, that will be fixed too. The crash rate was *not* due to that, but was due to another bug that has been found and fixed.

  5. Very Keynes says:

    /me looks to see if any events can be rescheduled on Fri/Sat. Damn Live event streaming from Amsterdam Saturday, no hope of rescheduling will just have to hope that they realy, realy, realy did find and fix it this time, and that my server is in the first batch and not the second.

    I know your schedule is tight and you try to avoid updates on weekends, but whenever you do it somone has to take a hit, I guess it was just our turn 😦 Good luck Prospero, for both our sakes.

  6. Meade Paravane says:

    /me builds Prospero a Valium Pez dispenser.

    Hope this round goes smoothly!

  7. Marianne McCann says:

    Got my fingers crossed!

  8. Once more unto the breach, dear friends, once more,
    Or close the wall up with our English dead!

    Good luck this time… 🙂

  9. Tree Kyomoon says:

    darn…logged into my island for the first time in 2 months and I get “region restarting in 5 minutes…” talk about bad timing!

  10. Jahar Aabye says:

    /me crosses his fingers

    I suppose it’s a testament to both the importance of this update and the longer-term plans upcoming that LL is taking the time and expense to do this over the weekend.

    I’m assuming that a rolling restart on Saturday means having more techies on hand in the office than normally would be there over the weekend, which translates into overtime pay for LL.

    Just wanted to thank you guys for putting in the time and effort to make sure this gets done right rather than half-assing it.

  11. Ann Otoole says:

    Bring it on. Smells done. Time to get it in and commence working on the next version.
    That’s right! *crack goes the whip! No rest mwuhahahaahahhh!

    Just kidding. hope it goes in nicely and you get a chance for a break.

  12. Curtis Dresler says:

    They must be confident – if they screw up, they will be working the weekend. Personally, I never scheduled anything major on Friday – need those Saturday mental health days. Or so said my psychiatrist just before his unfortunate, untimely and slightly suspicious death…

  13. Totem says:

    10 minutes ago my sim was functioning normally…now its crashing… The sound script throttling I reported last time you tried to roll out this code, is doing the same thing again….

  14. Darien Caldwell says:

    @ Totem,
    I think this is most certainly due to bad scripting on the part of the creator of those orbs.

    After you mentioned your issues last time, I went back on the beta grid and tried every sound orb I had to check against the 1.23 server they had running. In no case could I get any to trigger the issue. If i could have obtained one of that particular brand you have on Beta, I would have tried it, but alas, I didn’t have one.

    I would pick the orbs up and take up this issue with the orb creator. There is no reason they should be making sound calls 10 times a second or more.

  15. Totem says:

    Already picked them up Darien otherwise my SIM would be in the gutter. I raised the issue with the creator, last time, now it looks like she could have a big problem and in honesty I’ve been using this sound stones for well over 9mths on my own SIMs and others that I have created for clients with utterly no issues. I can send you one inworld if you like?

  16. Alyx Sands says:

    Good luck, Lindens! *crosses fingers*

  17. FROM PREVIOUS BLOG:

    ———————————————————-
    137 toybodacho Ireland Says:

    July 22nd, 2008 at 11:57 AM
    Question for JOSHUA LINDEN (not anybody else, please): Why aren’t upgrades tested for longer periods of time on a few special sims before they are applied to the entire grid? I think that at this point it is more than obvious that a few days are not nearly enough.

    I, as well as several other users, have asked this same question many, many times, but for some reason it keeps getting ignored by Prospero.

    ***

    144 Joshua Linden Says:

    July 22nd, 2008 at 12:30 PM
    @137 toybodacho Ireland:

    Don’t beat up on Prospero Linden too much – he juggling a lot of things, and trying to keep the grid running smoothly before, during, and after these updates. He manages to comment more than I do, which is amazing!

    As to why the upgrades aren’t tested for longer – well, at this point, the 1.23 Server changes have been on the Preview Grid since July 2nd (3 weeks). But most of the issues that have been found did not show up on the Preview Grid. It’s much smaller and gets much less activity and traffic, so it tends not to manifest the same problems – such as crashes under load. With the change rate we want to maintain (i.e. making lots of bug fixes), we also can’t mandate that the code sits on the preview grid for extended periods of time after each tweak, or we’d push to the main grid, notice an issue, roll back, push to the preview grid, then e.g. wait a week before iteration. At that rate, we’d never ship!

    That’s not to say we can’t improve our processes. Prospero Linden and others are working on a strategy to get more focused testing on the Preview Grid, and we’re rapidly iterating on our analysis tools so we can detect anomalies on smaller grids – or subsets of the main grid – before they are widely deployed.

    ———————————————————-

    First of all, thank you for replying, Joshua. Now for a clarification: was the version that was on the Preview Grid for 3 weeks the exact same version that was deployed a few days ago? If even the tiniest tweak was made to it, then it is not the same version, and thus it was not tested for that long.

    I can understand your explanation, even if I don’t agree with it. I think that a vast majority of your customers would like to see a lot more stability and are not in any rush whatsoever to have most of the bugs that you mention above fixed, and I think you should give priority to what your customers want.

    Why not test the upgrades on a few special sims in the Main Grid instead of on the Preview Grid? I’m sure you’ll have no trouble finding volunteers, and if you do, you could always offer incentives. And if testing for 3 days is not enough, then try 4, and if 4 is not enough, then try 5, and so on.

    And no, I’m not trying to beat up on Prospero. It’s just that whenever anyone suggests something, the suggestion gets ignored, while other people are getting answers just to be corrected on a trivial matter. It is extremely frustrating when you guys keep making the same mistakes over and over again and nothing ever seems to change. Test, test, test, guys! And then -and only then- roll it out!

  18. Neon Larnia says:

    I really hope that this is the reason for Second Life not allowing me online for the past two days. I’ve been trying and trying to get online and play, but the same thing keeps popping up! It says “Login failed. Second Life cannot be accessed through this computer”. It has never done or said this before. Until two nights ago at 9:00p.m.

  19. Tracey Humphreys says:

    Don’t think your problem is anything to do with SL, Neon. Is it your firewall blocking you?

  20. Cappy Frantisek says:

    /me grabs the popcorn and dons me helmet!

  21. Cincia Singh says:

    @20/21 … or it sounds like maybe your ISP is blocking or throttling … I’ve heard of that message from my friends in Australia when they hit their download or upload limits.

  22. Argent Stonecutter says:

    toybodacho@19: “Why aren’t upgrades tested for longer periods of time on a few special sims before they are applied to the entire grid?”

    THey are. They have a whole *grid* for testing upgrades before they deploy them on the main grid.

    Testing on a few sims on the main grid wouldn’t test it any better than testing it on the preview grid, the problem that are holding up the upgrade and forcing the rollback aren’t found until a lot of users and a lot of sims test it. If you want hem to have more testing on teh preview grid… GO TO THE PREVIEW GRID AND TEST THEM.

  23. Well kids, let’s see, 0 for 4, will they make it 0 for 5?

    Already, Linden Lab has entered the record books for Major Release failures, so the odds are they will fail again; why break up a tradition?

    Does anyone, older then me (47) ever remember a Gaming Company having such a disastrous time with a Major Release? I would love to hear about, but coming from the Beta Background for the past 15 years, I don’t remember any Online World that has failed this badly on a release.

    I would say “Good Luck” but you know what guys, “LUCK” doesn’t have anything to do with it. It’s called checking everything out, once, twice, three times, before you decide it’s good to go. Perhaps you should give Cory a call, I’m certain he’d be willing to help; right?

  24. Good luck… again! There is a whole lot of things to be released in such a short time: first, 1.23, then, Mono, then, the Open Grid Protocol interconnection with OpenSim grids, and I suppose the Shadow client will go to Beta too… whew! That’s tiring!

    I definitely wish you the best of luck to roll out all those improvements swiftly!

  25. Jahar Aabye says:

    @Toybodacho:

    I think you’re misunderstanding the problem that Joshua is describing. The reason that many of these bugs are not caught on the beta grid is because the beta grid sims do not get nearly as much traffic as the main grid, and don’t wind up running every possible scenario. Rolling out the code to a few special main-grid sims isn’t going to change that, because you still have the same problem! It won’t be getting much traffic and it won’t be running the millions of different scripts that are running across the grid!

    Rather than volunteering to have their sims added to some special “etended beta” on the main grid, people could just download the beta viewer and go test things out on the beta grid! More people testing things on the Beta Grid, and reporting them via JIRA, would improve the bug-fixing process and help ensure that the final code is as bug-free as possible before it’s sent out.

  26. @24 Argent Stonecutter and @ 27 Jahar Aabye:

    I do understand what the problem is, guys, but we have to be practical. In theory, the Preview Grid idea is great; the problem is that not enough of us are willing to spend time on it testing things out. That has been a fact up until now. Can it be changed? Maybe. Let’s give Linden Labs 3 months to spread the word that it is absolutely essential that everybody chip in and spend time on the Preview Grid testing stuff. If in 3 months we’re not getting the desired results, then we move towards another solution, right? Not Linden Labs philosophy, but very logical, correct?

    I strongly doubt that there will ever be enough people willing to help here. That’s why I suggested the ‘special’ sims. Pick a bunch of sims with popular sandlots, where most of the scripters and builders usually do their work. Pick a bunch of popular sims with stores, where lots of money and object transactions are made. Pick a good, representative bunch of sims and test the upgrades on them. Wait until 2 or 3 days have gone by without anybody reporting any more bugs, and then apply to the entire Main Grid. It might not work, but being that what they’re doing now is not working either, why not give it a try?

    My main point is, if something doesn’t work, try something else. Don’t get stuck doing the same thing, as they have with the rolling restarts for years. Every time it’s one day of testing and then all hell breaks loose. Snap out of it! Please!

  27. Tom says:

    Prospero… I notice that mainland land listings in search each sim returns diffrent results hence if i cross over to another sim the results seem to be 10 mins old but then if i cross back to a sim the results are totally diffrent mmm is this a catch issue or simply a new search feature Anyway good luck with the deploy 🙂

  28. Ivana says:

    and forwards
    and back
    and twist
    and evade
    and forwards
    and back
    and twist
    and evade

    That concludes our lesson on the SL hop, the new dance craze sweeping all regions in SL.

    Surely proper stress testing instead of constandtly redeploying is the right way to do this? It was when I qualified as a software tester anyway.

  29. Argent Stonecutter says:

    tobodacho@28: we’re talking about bugs that don’t show up until a few hundred sims are updated. Half a dozen early adopters aren’t going to help.

  30. Ready 4 it! says:

    @ 30…yep…now all we need is 60000 volenteers and another grid to test it on! 😉 JK too many variables to insure it will work right off ..you know that!

  31. TheTree says:

    @all the people critisizing the lindens approach for updates.

    you guys have no idea about how this beast (SL) really behaves. nor do the lindens have. the ionly difference is, lindens are aware of that, you are not. SL is so big and complicated, there’s no other way of doing it that the way they do, really, it has a life of it’s own. running the server code on the beta test wont give you the same result as having it on the main grid, simple because its huge and complicated.

    what’s really important is: they roll back if their release is bad – that’s neat. stop complaining about them doing their jobs right. oh wait, I guess you’re the kind of people who complain anyway (now don’t tell me “nooo, I don’t” – well, you just did, so what).

    lindenlabs and sl have improved a lot in the last months, and I enjoy this. They do a pretty fine job in improving what you people complain about all the time, give them credit.

  32. Mook says:

    @31 …. Then they should allow more time between the pilot roll and the first half …. expecting the run of the mill user to go to the beta grid just to hang out isnt going to happen.

    Beta has its uses for those that have an interest in releases, but its an artificial environment so the pilot roll should be monitored much more carefully, not just something done in a rush before going home, to come back in to roll half the grid- its potential is minimized like that.

  33. Darien Caldwell says:

    @17, Sure, you can send me one. really I can’t tell much without seeing the script. But believe me, I don’t doubt for a second you’re having an issue. It’s just given how the throttling works, they would have to be doing something mighty unusual to make it kick in. 🙂

  34. Balpien Hammerer says:

    Prospero, thank you for the update on the script performance problem. Much appreciated. Now will you update the JIRA report, please?

  35. Rocky Pickles says:

    anyone have a list of the regions running the new release so we can try stuff out in a production environment?

  36. cobalt says:

    “regions will receive warnings starting five minutes before they restart.”

    No, they won’t. And, I just lost an entire project because of that.

    No warning. No annoucement, and the sandbox sim I’m in rolls back a whole hour.

    Nice.

  37. Ivana says:

    @32 Hmm, fair comment about the 60000 users. But then others have managed to write a bot that only takes 30Mb of memory and can effectively move, teleport etc from a script, capable of running 100 times on a single PC. Reading these blogs you see enough people complain about such bot technology lagging hell out of SL etc, well perhaps it’s time someone put such technology to good use.

    Surely the people that write the server code and design the browser can effectively code similar with built in testing and test metrics.

  38. Ryu Darragh says:

    It’s funny.. no one seems to have noticed the Lindens already do partial rollouts to the main grid. And I’ve seen problems caught right then and the rollout reverted and back to the beta grid.

    I know what it’s like to go through dozens of live updates from the engineers perspective, and that’s from one multithreaded operating system I designed. This is quite a few more major components and millions more clients than I ever dealt with.

    May the power of Jolt Cola and Twinkies Extra Stuffed be with you guys on the Dev teams and the Bug Hunters that sweep up after them.

  39. Not so fast says:

    OK, have been on for a bit and getting nothing but hang ups and crashes. How many times to people have to tell you Lindens? IT IS RELIABILITY STUPID. And for you blogging sycophants… yes i am running the latest and best with the best connectivity. Look… how many bites at the apple do you think you are going to get?

  40. Alf Lednev says:

    #31 you MISS the point entirely. Lindens is a business and wants to be profitable in the commercial world. Using its paying customers as beta testers has worked in the past, geeks love to feel important. But to transition to mainstream and maintain its paying customers that way is a receipe to failure. Its not Prospero’s fault but an organistaional flaw, Lindens have to position themselves for when competition arrives. The way they are going, now spells failure. They have not yet made the mind shift fron an RD company with a pool of immense goodwill to draw on to a commodity that consumers want to relax with. When competition arrives, a lot of users will simply take their money elsewhere.

    Quality Assurance is a joke still, to much fails to often. From comments here, the beta grid is inadequate for the task, but using the main grid is potentially commercial suicide. I suspect QA testing is grossly under resourced and Prospero and co make do the best they can. Thats something the Board will need to look at, else they will simply vanish in a year or so.

  41. Joshe Darkstone says:

    @40/@Argent

    People keep saying the same thing and its like there are 2 different conversations going on, apparently in 2 different languages. Everyone knows its tested on the beta grid. The beta grid is for finding out if a script works, if the interaction between servers is right, if theres anythign obviously borked. Whats missing, and people seem to suggest is unwarranted, is adequate stress testing of a limited but sufficent rollout on the main grid. A stress test is an important part of widely distributed platforms and they basically ignore it, following up the initial rollout with the grid rollout without takign adequate time to identify problems between them, and here we go again. The limited rollout should be left in place for a week or more if it appears to be working, so that the hard to spot problems dont affect the entire grid.

    This is a much better process then the old one, but its easily improved by adding a few days between the intial and gridwide rollouts. The couple times that the initial rollout was rolled back before it impacted the entire grid was a good approach, but they did it because they spotted obvious problems. If the problem is less obvious then it will be gridwide within 2 days.

    What irks me is no one seems to address this issue directly from LL. “Damn the torpedos, Full speed ahead” appears to be the SOP. In these rollouts, In the land rush, they completely ignore the obvious push ahead as if they could not see any of it.

    Islands are being offered for rent at the cost of tiers, and sold at losses, while LL talks about auctioning more land in a couple weeks as if we are salivating for it. 20000 sims for 60000 users… more please!

    “we hear you, we feel your pain, we appreciate your patience, Its for the good of the community”. ugh already – the better way is obvious so by not taking it you are letting us know that there is no community. Just tell us the big secret plan that keeps you running over us already 😛

  42. Prospero Linden says:

    OK, so, re: testing and all of that….

    1.23 has been on the preview grid for weeks, but, no, not the version that’s going out right now. But, every change since 1.23.0 has been a small, scoped change. Yes, in principle, every tiny change is a new version, and you could argue that we should test every tiny change exhaustively. And, if we did that, we would freeze in place. There are changes and there are changes. The changes between 1.21 and 1.22 were big. The changes between 1.22 and 1.23 were big. but the changes to the third digit have been very small, and have been explicitly targeted fixes to problems found.

    The crash bug found for today’s deploy is a very simple, small fix. It was hard to find, but once found trivial to fix. What’s more, we were able to verify that it was fixed by baking it overnight on the Preview Grid, *including* the simstate from a region that was known to trigger the bug.

    Why didn’t we find it before? People have addressed this above. Second Life is *huge*. There are two problems. One is load; sometimes, there are issues that cause increased load. Those will only rear their heads when we have a substantial fraction of the grid hitting our central servers. The second is statistics. When a crash bug is relatively infrequent — as was the case with the 1.23.2 crash bug — the preview grid, and even the pilot roll, doesn’t have enough different regions running the code to highlight the bug. Once we found it, we could pick out a region that triggered the bug and test with it; but, we had to find it in the first place.

    Part of the reason the current pilot roll is 500 hosts, as opposed to the usual 50, is so that we will have more statistics overnight to make sure that the crash rate isn’t higher with 1.23.3 than it is with 1.22.4.

    Regarding testing and all of that: user beta testing isn’t a cure-all, but it does have a place, and we are planning on getting more formal with that than we are right now. Watch this blog for more information as more things develop. But, right now, please do test things out on the preview grid when you read about new deploys. Second Life is huge, and the residents are more creative than any one individual can keep track of. We obviously aren’t going to test every script, every creative use of Second Life– because do doing is the same as just deploying to the main grid. Anybody can log into the Preview Grid; you don’t even need a special viewer to do it. For more information, see http://wiki.secondlife.com/wiki/Preview_Grid

  43. I’ll be sooooo happy when this issues fixed fully, since monday I haven’t been able to move really or do anything on the grid 😦

    *is really sad*

  44. Joshe Darkstone says:

    @Prospero

    Again no answer the obvious question… Why schedule another gridwide rollout 24 hours after the intial rollout? Let it breathe. Let the cheese melt. Take the most obvious precaution available to you.

  45. Prospero Linden says:

    Joshe : we may do that at some point, but that doesn’t work with how we do things right now. Yes, it is under consideration, but there’s no “perfect” solution. As it is, we *do* find problems in the pilot roll, which is why a couple of our attempts to roll out 1.23 were canceled after just that. However, because of the statistics problem I mention above, the pilot roll may not be enough to find everything– not big enough compared to the Preview Grid to give us enough new information. At some level, we can get most of the information we’re going to get in 12 or 24 hours of a pilot roll. There is a tradeoff between letting it sit further to get the additional few percent of information, or moving forward with the roll to either keep the process moving, or to find the problems that won’t show themselves until the code is on a substantial fraction of the grid.

    There is no magic bullet in any of this.

  46. DhouZhang Fhang says:

    Prospero and the Linden team,
    This is just a suggestion, but I think everyone (well, most everyone) will agree with me on this. If you really want to know what’s going on, what’s happening, what the inworld conditions are like, how about actually going inworld yourselves for a week or so before doing a roll-out? There should be enough of you to visit every single location in Second life. By doing this, you get a first hand experience of what your “Valued Customers” are experiencing and you can actually talk to them and find out from them what is happening. Would that be too hard to do? At least that way, we – the customer – can be assured that the Linden Lab crew really does give a damn about their customers and their product.

    I know there will the requisite naysayers who will come in and shoot this idea down (they know who they are), but I feel this is a fair and even way to go about things.

  47. Prospero Linden says:

    There are currently 27,000 regions in Second Life. If you wanted every region to be visited by a Linden in a week, each Linden would have 100 regions on his or her list. Are we really going to have time to experience what’s going on in every region — that is, every 256m x 256m area of land — at that rate? Second Life is *huge*. Nobody knows everything that’s in Second Life. That’s the beauty of it; hundreds of thousands of creative residents, all building the world in a huge diversity of ways.

    In any event, it’s a fallacy that Lindens are *not* in-world. You don’t see Lindens around all the time because there are hundreds of thousands of residents, but only about 300 Lindens. And, yes some Lindens spend more time in-world than other Lindens. But a lot of us spend a fair amount of time in-world both for our jobs. Also, there are Lindens who own quite a bit of land, and spend time in-world as alts– meaning that people who run into them won’t even know that their Lindens. I’m fairly rare in that I am pretty open about who I am with my primary alt, but there are other Lindens who have good reasons to want to have an in-world presence unburdened by their identity as a Linden. I own more than one sim’s worth of mainland as my alt, and there are other Lindens who have multiple private islands or a substantial quantity of mainland, or both. Do we know everything that’s going on in-world? No. But we do have a sense of what’s going on, and a lot of us do, *as residents*, feel the pain when things are not working as well as we want them to be.

  48. Bud Parnall says:

    All is well with the new version on my sim.

  49. Prospero you are a genius!

  50. @44 Prospero Linden:

    So the changes that were made were very small, scoped changes, yet the results were disastrous, to say the least. But you say that exhaustive testing is not necessary? Help me understand, please. Are we playing “what’s wrong with this picture”?

    Longer test periods will not “freeze us in place,” they will only slow us down a bit, which is not a bad trade-off if it’ll keep us from going through this every week. Moving ahead at full speed should not take precedence over customer satisfaction.

    I think we are all aware that finding and fixing bugs is no easy task. It’s not that we don’t appreciate the hard work of the whole team, Prospero; it’s just that the way you’re going about it is affecting us so much and so often that we’re unhappy with it. We want that changed.

    I know we all look at this from different perspectives, understandable and natural. We all have our own interests to look after – Linden Labs as a business, you as developers, each one of us as users. I also understand that you might not be at liberty to discuss the reasons behind some of Linden Labs actions. So be it.

    I’ve stated my suggestion and I think I’ve made my point. I hope they’re taken into consideration when making decisions.

  51. Lars Donardson says:

    I am sorry to say that you don’t get the point Toubodacho!

    Some problems will not show on ANY smaller scale testing no matter how long you will do it.

    If an error in the code causes a crash that happens very sporadically (as the last one did) the chances that it shows on a test environment of 10 or 50 servers is small. If you roll the server code out on 6000 sims the situation is vastly different. The chances that the error shows are vastly bigger. Add the load factor and possibilities of scripts that are inworld to this equation, you surely understand that LL can’t possibly try all scripts that are inworld, and LL can test an update for months and it will appear to be rock solid. In the meantime they sit and wait and we will get important upgrades in months if not years.

    I think the trade-off by having rollbacks or server code are imho much smaller than waiting months for bugfixes/updates that we all want and ask for.

  52. Neon Larnia says:

    No, it can’t be the firewall. I don’t know what it is. All I know is that it worked fine for over two weeks, and now this. I really, really want to play! It’s not just me that has this problem, either. I know a few others that have the same problem. I’ve done anything and everything to help. I unstalled it, restarted my computer, disk clean up, defragmented, and reinstalled it. Yet, it proceeds to say that.

  53. Tristin Mikazuki says:

    @49 OMG Lindens are land barons hehehe (sorry had to)

  54. Ron Crimson says:

    @54 Neon, I wish I had any advice for you – I did run into the “Second Life can’t be accessed from your computer” error myself before, but the two times it ever did, it resolved itself later the same day. I simply waited it out.
    Have you contacted LL support about it?

    @25 Bob, you’re really beginning to tick me off… I try to ignore your incessant negativity but it’s becoming hard. ARGH!

  55. Ron Crimson says:

    And Prospero: you and your team are heroes. Don’t let anybody tell you otherwise. Rock on 😀 And good luck this time 😀

  56. /me gives Prospero and team lots and lots and LOTS of chocolate!!

    Looking forward to the upgrade 🙂

  57. Joshe Darkstone says:

    @Prospero,

    Im certainly not in your seat so I don’t have the birdseye view of this but my experience tells me there are more advatages to letting it bake in the wild then meets the eye. I’m glad to hear you are considering it and I’ll drop the matter now – Thank you for the reply.

  58. Zi Ree says:

    @Second Life cannot be accessed from this computer:

    As far as I know and my research on the web has shown it usually has to do with the address you’re coming from. If you are on a dynamic address (you get a new IP every time you restart your internet connection hardware), you might end up on an address that was used by a malicious person before and has been blacklisted. The fastest and easiest solution to this problem is to completely restart all internet hardware (router, modem) to get a new address assigned to you. You can retrieve your currently assigned address on e.g.

    http://www.ip-adress.com/

    Note down this number and reset *all* hardware that’s being used to connect to the internet (pull the power plug for a minute). Then restart your connection and open this web page again. If the number changed, you are on a dynamig address and chances are good that SL will work for you now. It might be necessary to try this 2 or 3 times, depending on how many addresses are blocked in your address range.

    If the number stays static, you might need to contact SL support by filing a ticket on the support page. Tell them your IP address and the error message.

    Good luck!

  59. Ron Crimson says:

    Zi Ree: that’s great advice… but in my case, the “…cannot be accessed from this computer” was definitely a random occurrence. My router and cable modem are on 24/7, though I put my computer into standby when I’m not using it, and for some reason my IP rarely ever changes…I’ve seen it stay the same for MONTHS at a time. o_O Not sure if this is specific to Roadrunner or why, but even on the occasions I unplug everything and start it again after a minute, the new DHCP lease *usually* gives me the same IP address I had before.
    It’s weird… but your advice is excellent. (I just wanted to point out my experience’s been different with this error, and it went away on its own both times I saw it.)

  60. Prospero:

    Why were people being transported into Cellate then the SIM they were trying to get to AFTER the rollout?

    Did you think I was just kidding with you when I reported it in-world? Has anyone else had this problem? You say it’s not correct that Lindens aren’t in-world, I think the point you are missing here is, there’s a difference being in-world and available, then in-world and unavailable or downright nasty when someone IM’s you.

    But then again, I’m sure you have some slick polished answer that will put me in my place and tell me just how much Linden Employee’s help everyone. Just remember, I’ve been here long enough to know a duck when it quacks.

  61. Ron Crimson says:

    Geez, Bob… you always know better, don’tcha? Just who do you think you’re speaking for besides yourself? I for one didn’t have this problem, and nobody else has mentioned it either. So you ended up in a wrong sim, BIG WHOOP. X-(

    Is this slick enough for ya!!???

  62. Joshe Darkstone says:

    Ive found the lindens to be generally responsive and helpful in-world, obvious there has to be a balance between being responsive to everyone in secondlife and getting your job done. Sometimes its necessary to just ignore everyone, put your head down, and get your work done. I have never had a Linden be actually rude. Im sure it happens, Im also sure its rare.

    There is no way it would be reasonable to expect a company to pay salarys to people to roam around and interview residents all day long. More responsiveness on the blogs, a more usable jira, opening the grid status page for comments, Cant expect much more then that from a software vendor.

  63. Well Ron, good for you, about 20 of us did, and it had no rhyme or reason as to why it dumped everyone there.

    Sorry you weren’t invited to the party there, but hey, you must just be too good for us schmucks eh?

  64. Jack says:

    Prospero, can you tell us if will you update even or odd sims today?

  65. pepita says:

    what numbers are we doing today, odds or evens? 😉

  66. Zi Ree says:

    @65 Bob Bunderfeld: Why is it such a problem that you logged in at another place than you logged out? This happens at times, when the Sim you want to log in to is not available for whatever reason. Usually you get a popup dialog telling you that you have been “moved to a nearby location” (which usually is not true 😉 ) and you wind up somewhere else. I usually resurface in Ahern when this happens. I can’t really see what the problem is.

  67. Prospero Linden says:

    toubodacho @52 : the changes from 1.23.x to 1.23.x+1 have all been small and scoped. The roll-forward-roll-back process we’ve had hasn’t been as a result of them, but has been as a result of uncovering deeper bugs within 1.23 itself; the change from 1.22 to 1.23 was big and included a lot of things.

    You probably noticed that I delayed the rollout *again*. This is *not* evidence that the 1.23.2 to 1.23.3 change was bigger than we thought. Rather, what it means is that we’re finding crash modes in 1.23 now that were completely hidden by the much higher crash rate we saw earlier, and have fixed in 1.23.3.

  68. Prospero Linden says:

    Re: Lindens responding or not responding in-world, again I have to point to the hundreds of thousands of residents versus few hundred Lindens. By and large, we explicitly *avoid* responding quickly to random IMs from residents. The reason is, there are residents who bypass the usual support channels, and instead go down the list of Lindens and IM them one-by-one until they find somebody who will give them personal attention for their issues. This is bad for two reasons. First, it’s unfair to the people who do it “right”, for they get slower service than than the people who short-circuit the process. Second, once word gets out that you can get your problem solved by just spamming Lindens, the spamming of Lindens will increase, and we won’t be able to do our regular jobs — the customer support, the fixing of bugs, and everything else that you really want us to be doing.

    So, yes, Lindens are often going to be unresponsive in-world when you IM them directly. When you do get a response, most often it’s going to be “please contact support for help with your issue.” This does *not* mean that we aren’t in-world, experiencing and using Second Life, understanding what’s going on out there. It does mean that we are not all in-world to provide live help upon demand, for we simply don’t have the manpower to do that while providing the other support options we provide, and while doing our jobs (for those of us not in customer support).

    If you want to chat with a Linden directly, please look at the list of Lindens who hold office hours. The topics that each of them talk about are listed on that page, so you can find somebody who is most likely to be able to talk about the topics you want. Note, again, that these office hours are not for individual support — they are not for Lindens to help you fix your individual problem. Again, for that, go through the regular support channels. Office Hours are for you to learn more about how we do things, how things work, to raise general concerns and give general feedback, and that sort of thing.

  69. Mook says:

    What kind of delay are we talking here? a not today delay, or maybe just a few hours delay?

  70. Dilbert D says:

    Thank you for taking your time on this Next roll out to alleviate potential problems.. Good luck Guys hope to see a clean update soon 🙂

  71. Yuo Rang says:

    Oops sorry, using 2 precious reply slots here- and will you reroll the pilot sims first? (which would probably mean a postponement of first half anyway)

  72. Prospero Linden says:

    Length of delay is unclear at the moment, but probably it won’t happen today. We will also probably reroll the pilot sims first, but that’s also not clear. Watch this space for more information.

  73. Iustinian Tomsen says:

    3rd time within this release: EPIC FAIL!!!

  74. shockwave yareach says:

    Prospero said: 1.23 has been on the preview grid for weeks, but, no, not the version that’s going out right now.

    In Avionics, if you make a change after test, then you aren’t running the same thing. Period. Deploy what you test and test what you deploy. In my biz, you’d be fired if you changed and released code without testing the entire system again.

  75. Ron Crimson says:

    Wow, 1.23 is definitely making history as one of the most difficult rollouts in SL history. I feel your pain, guys, and I admire your sticking with it and getting those pesky bugs fixed. Keep hammering away at it, Prospero & Co., you’re getting there… I can feel it in the air 😉

  76. River Ely says:

    So, why was my posting pulled? was the truth so painfull I had to be censored?

  77. Prospero:

    Since you brought it up, tell me, what are Residents that do go thru channels and still get no response, do?

    Let’s ask this question as well. When a Griefing Script is on Linden Land and it’s been AR’d by 5 seperate individuals, are we just to cancel the event going on, or should we expect immediate response to the situation, since it sits on Linden Land? When it happened during our New Resident Q&A, the hostess got a very nasty and rude response, after waiting 10 minutes for someone from Linden Lab to come take care of what is YOUR problem. Perhaps the next time it happens we should just AR the owner of the Land and you can deal with Govenor Linden in whatever way you choose, that is of course, is if the Logs don’t get corrupted again.

  78. Kenny Devoix says:

    @79 Bob,
    I have been here going on 3 years and every ticket to support I have opened has been taken care of within a reasonable timeframe,with the longest wait 6 hours. Of course I see reasonable as in within 24 hours. Ever stop to think that the reason they might not drop every thing and fix your problem within 5 mins is that they are busy fixing others that were in que through the proper channels before yours. Are you really so special that you and your friends should get put to the top of the list while others have to wait longer? Another thing I have found in general is that if one is rude and nasty they can expect rude and nasty in return. Try politeness and courtesy for a change instead of bluster and rudeness.

  79. unhappy says:

    Dont know if its the region im in is effected, But im not on my 8th crash in 3 hours only one of those have i managed to save the shape im working on.
    Yesterday was fine as was the day before.

    Would be nice if the crash reasons where given back …. or the crash logger told a reason why.
    I presume if i was to put a ticket in would be told its my system

  80. Prospero Linden says:

    unhappy : are you talking about viewer crashes or server crashes? If viewer crashes, I don’t know much about it, and it’s not really on topic for this thread.

    None of the regions running 1.23.2.92922 have had eight crashes.

  81. Tanya Spinotti says:

    Has anyone read the Tao of Linden recently?

    I applaud Prospero, Sidewinder, Torley and every Linden who stands by the Tao. “Sometimes you will fail, and in those cases it is very important to fail fast and fail publicly – that is how we learn and iterate and ultimately win.” “Be Transparent and Open”.

    Think about it for a moment. How often have you dealt with a company and felt that you’re being ‘fobbed off’ with excuses because they won’t tell you what’s really going on?

    Yes SL has problems, yes LL suffer set-backs, but I can’t think of any other company that I deal with which is open, and honest enough, to tell their customer’s when things aren’t right.

    How quickly people forget LL’s successes and focus on the negative.

    All those who want to be involved in something genuinely innovative and fabulous, stay right where you are. As for the rest…no-one forces you to be here…

    Your normal programming of cheers and boo’s will now resume….

  82. Cincia Singh says:

    @79 Bob … perhaps you should stop by that new store I heard about, “Lindens ‘R Us” and just pick yourself up your own Linden. Then maybe you’d get the customer service you think you so richly deserve! (hehe)

  83. Zi Ree says:

    Oh! Before I forget! Happy System Administrator Appreciation Day to you, Prospero and all the other Lindens who keep the Grid and Website up and running!

    http://www.sysadminday.com/

  84. Alx says:

    Partial roll outs to discover problems, and rollbacks if problems are discovered is the way to go, I feel. So I like the current way of rolling out server updates.

    However, it’s clearly less than optimal if a sim restart happens during or just before a big event in a sim. It seems that if the club and shop owners where able to decide when their service window is, that is, when it’s ok for their sim to be restarted, then this wouldn’t be much of a problem.

    So just make sure the restart can be postponed several hours, like scheduled to a certain time within the next 12-24 hours, and I’m sure most people that run businesses would find this much less of a bother.

    It will make things more complicated for the people deploying a new server version, sure, but since this is the main grid people do schedule events that can’t be moved. So the server deploy need to move instead. There needs to be some give and take in this area, I feel. The compromise would be to still allow deploys to happen often, but for them to happen at a time when it doesn’t break people’s events.

  85. Not so fast says:

    Well “M’s” comments are closed. But let me tell you something M. If you cannot complete a real time dialog with someone in rp without two freezes and then a crash while using the latest gear and software then M you just do not have a viable product except for the Newton-types hangers on… do you understand me? You dont get it yet. You must embrace complete reliability for forget it… you will not have a business if you rely on the nerds who say “well thats ok… just keep trying.” Get real and get real fast.

  86. Joshe Darkstone says:

    @86 an interesting idea, something to shoot for might be an automated method of scheduiing your own upgrade, could be as simple as “reboot when ready”. with a swap process scheduled for next reboot.

    Of course that would have to happen after some extended testing on the live grid anyway, but tools can be made to manage that. the events schedule can be checked automagically to select a group of sims for the initial rollout that have no conflicting events scheduled. Its pot luck now so at least thats pot luck with a brain 😛

    @83 I can only speak for myself… I do tend to “focus on the negative” on the blogs – I think its the nature of the beast. When things are going well I’m busy doing it, when things are going poorly, in my case at least, its affecting my income.

    I do try to be thoughtful enough to pay homage the improvements, which are huge by comparison to only a few months ago, at the same time as Im trying to control my own destiny with what I hope is usually constructive criticism.

    Im here till the wheels fall off, I’ve got alot of time/energy/money invested. I love sl and the people I’ve met here, and for the most part it has been an enormously rewarding experience. The open-ness of which you speak allows me a way to have some input, occassionally even be heard – but it cant be all cheers.

    Its no more anti-SL to speak my mind against the policy shifts of this community/government then it is anti-american to speak up when I dont like the latest episode of the Bush Follies. In the end, SL is a better place when it listens to its customers, good and bad alike, If they didnt think so I guess we wouldn’t be sharing this thread 😛

  87. cat cotton says:

    Le sigh, all that’s left to do now is….

    LAUGH.

    I would wish you luck but you guys don’t need luck. Do you.

  88. Alyx Sands says:

    @Prospero, who IS your “public” alt? I might pop down some day and say hello then! 😉 Good luck for the next iteration of 1.23….

  89. Balpien Hammerer says:

    #16 wrote:I would pick the orbs up and take up this issue with the orb creator. There is no reason they should be making sound calls 10 times a second or more.

    You know, if someone wants to go pirate some music and play it in a sound, yes, they only need call paysound once every9-10 seconds. But, if you want to see original music played live, then it is not a stetch to see burst of sound calls 8 or 16 times per second as a chord progrssion is play by an inworld synthesizer.

    Think broader scenarios here, please, before making such proclamations.

  90. Balpien Hammerer says:

    Prospoero, the doubled script time problem seems to have been resolved by this latest version of the server code. THanks!

  91. Tom says:

    If its not perfect this time plz try leaving it deployed for a few days instead of rollback let things bed in first aslong as its nothing major dont worry about minor things everything can be worked around even small bugs most wouldnt even see most of them 🙂 until people point them out then you will have a relative overveiw of causes…

  92. Tristin Mikazuki says:

    Ya know that might be a good idea after you put the new code on beta do 1000+or- on the main then do the rest of the grid the next week so you have a full week to check for bugs on the main grid. I could put up with clicking the new region pop up for that.

  93. Wondering about Bob says:

    Bob Bob Bob…….ever tried WoW? please do! Your whining tones are depressing. 🙂

  94. Wondering about Bob says:

    Before Bob sees my last blog, lets corrupt the logs so he can’t complain again. M Linden

  95. Not so fast says:

    Well Prospero… how long before you learn in SF? If each server update cannot be better than the last one then don’t release it. You have fallen victim to the Car Guy Detroit Syndrome. You are going in the wrong direction. I know it is hard to accept this advice from a stranger… but please step back for a reality check or the marketplace will give it to you far sooner than you ever imagined.

  96. Paddy Wright says:

    All I want to say to Prospero and the Tech guys and gals is well done! SL is becoming a stable platform for the first time ever. Keep it up and have a drink (mineral water) on me.

  97. @80. I have a Support ticket that should be simple to fix still marked at “Work in Progress”. It was filed on 28th November and has never been answered. I have commented asking “what is the progress” a few times.

    I would love my support tickets to be answered within a week. I was offered Enterprize support but.. I just don’t have the US$25,000 a year to pay for it.

    I have other old tickets but that is the oldest unanswered one.

  98. Ryu Darragh says:

    How to create a bug in a few simple steps. One, take a large set of libraries of code. Write additional code to make them talk together. Write a bunch of code of your own for unique functions not provided by anyone else. Install all this new code on your system and run it.

    Easy to do.

    The problem comes in all the way back as far as those second and third party pieces of code in the libraries. Programmers use tools that *assist* them in catching all the silly mistakes, like not initializing variables before use (the infamous infinite FOR loop where the end value of 32 became 2 billion+) or forgetting to properly scope a variable (the use of a variable named INDEX as a global variable all routines must see and forgetting and creating a *local* variable also named INDEX and then using INDEX there while meaning to use the global one… and getting the local one).

    And those are *simple* and easily found bugs. The more subtle bugs are not knowing what boundary conditions and combinations of inputs, which could number in the *trillions* of possibilities, will overwhelm a function causing it to only output a *slightly* off value if not crash outright.

    It’s those “fringe boundary conditions” no amount of canned, preview and “small rollout” tests will find. Might as well expect your broken coffee mug to magically jump off the floor and reassemble itself. Statistically, it’s unlikely, but, it’s still *possible*.

    There’s only a few simple things I’d like to see LL give us, and I urge all builders/sellers do a jira or vote on this issue if it exists. We need a dataserver event confirming inventory delivery. If we had this, the worst hole in our vending machines would be plugged.

  99. Garth, come on now, I still have an open Bug Report from 4 years ago! You really think Linden Lab is going to slow down to fix anything?

    Read M’s comments, and I’m sure you will agree, Second Life as we know it is dying. Linden Lab is moving towards a more OPEN SYSTEM, which will leave all of us Residents in a world that just can’t be added in.

  100. Darien Caldwell says:

    @91

    The item being discussed was not a synthesizer, nor ‘pirated music’ (not sure why that reference was made), but was in fact an environmental sound orb. I stand by my statement, and would urge you to understand the context of other people’s statements before attempting to bash them. Thank you 🙂

  101. LittleRed Snook says:

    *Hugs of support for prospero and the LL team* 🙂

  102. Tegg B says:

    Somehow I suspect Bob is related to Usagi, he seems to believe everything should work on first attempt.

    Good try Prospero and team, don’t listen to the naysayers who would prefer you give up, we know you can do it. The beer after sucess will just be sweeter.

  103. DhouZhang Fhang says:

    Okay, so maybe a week is too short a time. Would a month be better? And i wasn’t talking about IM’s either – i understand the issues surrounding unwanted IM’s, believe me I’ve had to tell some guys off for IMing me when they can clearly see me. I’m talking about actually chatting with just the everyday schmoe’s – and not just in office hours. The Linden Lab office hours are geared to the US times, not international times – so when I’m on, the offices are closed. How then does that help me? I’m certain that not a single linden has been outside their own little piece of “paradise” to see the entire world. If the did, they would know the troubles that are plaguing SL. Come on, guys get out of your shells and see the world in it’s entirety. If that takes a year, so be it.

    Note to Ron Crimson – how about you take the same advice i gave the Linden Lab team and get out and see the rest of the world. Then you’d know what people are going through and you’d also find that your smarmy attitude is not appreciated.

  104. DR Dahlgren says:

    25 Bob Bunderfeld – Gaming companies just push out the date until everyone forgets about it. Serioulsy, while many of us may call this a game, the scale and complication of SL simply has no peer to compare it to.

    While the repeated rolling restarts are a pain, I still say they are far better than the old method of taking the grid down. If this rollout had been done that way, it would have gone something like this:

    07:00 SLT Grid is down while we roll out Server 1.23.3. We hope to have logins open by 12:00

    12:00 SLT We have opened logins to everyone.
    13:00 SLT Logins restricted to Lindens – it appears there is a crash bug in the new server code. Please be patient while the grid monkeys bang on things to get it sorted out.
    17:00 SLT It appears we are going to have to roll back the servers to 1.22.4 Logins are restricted to Lindens, with apologize for the inconvenience
    21:00 SLT The rollback is complete and logins are now open for everyone.

    This would have been repeated for each attempt. Trust me, it happened more than once. Anyone remember Dec ’06.

    Keep hammering on it guys, better right than rushed.

    Maybe going back to more sims and parcels available for purchase and temp use on Beta would get more ppl there. Seems there were a lot more ppl going when there was a sort of Beta subculture. Probably wouldn’t get you more feedback, but might allow for more monitering… just a thought.

    DRD

  105. Lecina Enigma says:

    Well, it`s a hard Job to get something 100% stable running like SL is.
    The work of the Lindens (even if i could be like “grrr what are they doing” sometimes) is a imho allmost flawless thing they do.

    As mentioned by some of you here, just help as much as possible..report back any lil bit of errors, crashes, failitures….and things are getting tested.

    Even, i dont know how much fixed employees there now at LindenLab…it`s a hard Job there to..because as i know LL hasnt the ressources like other great Companys (like EA, or similars).

    I hope for them, and us that the 1.23 update will be flawlessy work on the next Rollout and that the Sysadmins and Dev can take a short but healthy Breath..before going to new Stuff.

    Sorry for my English 🙂 it`s not my native Tounge

  106. Jay Townsend says:

    It would be nice if ‘pilot’ (re – guinea pig’s) being used were not the sims of people trying to run a real business. The extra restarts, script lag and increased crashes of test software are an additional burden that people paying $200-$300 a month in tier and a couple of hundred dollars a week in classifieds should not be subjected to.

  107. TC says:

    @Ryu (101) : If Ruth ever lifts the curse she inflicted on 1.23, there is supposed to be a partial fix in there toward SVC-71, not delivery confirmation but the missing money events (VWR-4513), a big part of the problem especially recently.

    (I’m sure that Ruth is behind this, she has the motive. Doubtlessly the years of snide remarks took their toll. Being told in no uncertain terms that even bling clouds would be better to look at was the last straw. The time was ripe for revenge.)

  108. DR Dahlgren says:

    Love it, forgot about here. Hell hath no fury…

    I have to say even though I often taken LL to task here, I really do believe they are trying to make things better. I don’t see how some of you seem to think you have it both ways, a stable grid, but no upgrades. Bugs being eliminated, but no upgrades. Features being added, but no upgrades. LL seems to be trying to communicate with us better, they even seem to be listening sometimes ( that horrible UI now can be eliminated). Maybe some of us should look at our priorities, just as I often chide LL to do.

    Those of us who run a business in here… well, it might be my priority, but I don’t really expect it to be LL’s. Some of you who scream so often about the restarts etc and how they affect a business, might want to go actually read the TOS you agreed to. You operate a business, or create content, or have a house, etc etc, at your risk. Period. Lindens are not money. They can turn off the exchange tomorrow. You rent server space, no matter how it is advertised, and that can be turned off tomorrow. You lose all the content of your inventory, oops, no warranty.

    I would suggest that we continue to hassle them when they bork it up, continue to push to get issues resolved, but good grief, get off their backs when they are trying to do it right.

    That said, when the HECK are you going to fix alpha textures LL???

    DRD

  109. Tegg B says:

    DhouZhang Fhang Says: “Okay, so maybe a week is too short a time. Would a month be better? And i wasn’t talking about IM’s either – i understand the issues surrounding unwanted IM’s, believe me I’ve had to tell some guys off for IMing me when they can clearly see me. I’m talking about actually chatting with just the everyday schmoe’s – and not just in office hours. The Linden Lab office hours are geared to the US times, not international times – so when I’m on, the offices are closed. How then does that help me? I’m certain that not a single linden has been outside their own little piece of “paradise” to see the entire world. If the did, they would know the troubles that are plaguing SL. Come on, guys get out of your shells and see the world in it’s entirety. If that takes a year, so be it.”
    I’m also on the opposite side of the planet and you are certainly wrong or haven’t been in the game long, many Lindens travel travel to many places on the grid, I’ve seen them at sandboxes, charity events, science lectures, clubs, shopping, even combat sims. Probably many more than a lot of residents stuck on shop islands with bot friends..
    Using the normal channels. I have never had a problem getting a Linden response, which being a smalltime Landlord I require maybe evert 2 months or so. I had one TP to my land nearly immediately 2 nights ago for a quick easy task that I didn’t expect them to have time for.
    Perhaps you need to be less aggressive in your opening line of communication and plenty of courtesy and thanks to the staff and they may remember your name for good reasons.

  110. Coventina Dalgleish says:

    Well I don’t know what these blogs really do as there are very few salient issues raised, mostly just people wanting to hear themselves talk. Also I see the same 100 or so names on every issue this is a very poor representation of this community. Perhaps most of our residents are just enjoying the game. Even the normal crowd seems to have slowed down this time normally all I get to do is read 150 comments )).

  111. Moshe Janus says:

    guys some1 can tell me what i need to download to see a TV on Second Life? plz help on that
    thx

  112. The main grid is indeed huge… but the beta grid is tiny. In particular it lacks mature sandbox space. It does have a very few typical mainland sims (on the order of 2 or 3), but those are mostly no-build and no-script. So it’s hard to even find a place to test stuff. I asked some Linden (I forget who) at an office hour if the beta grid could be made bigger… the answer was basically Probably Not, not much bigger. The beta grid is currently mor prominent than useful because it is being used to test inter-grid TPs.

    (BTW, I hope all the Lindens who own a sim or more of land of alts own it in medium or large parcels. It would be embarrassing if a Linden turned out ot be owning a sim’s worth of land in the form of many hundreds of 16s and 32s on many hundreds of sims— unless perhaps he or she was using them for server space.)

  113. Abolish16mparcels says:

    Abolish 16m parcels NOW nobody wants or has a need for parcels under 512m FIX SL

  114. Ron Crimson says:

    @114 Moshe: I’m assuming you want to view streaming video within SL – in that case you need to install the latest Quicktime. There’s no additional setup needed, just install it, then run SL again and open the audio/video prefs to tick the checkbox next to “Play Streaming Media When Available”. Once that’s done you’ll have a new movie control next to the music control and can watch videos in-world. 🙂

    @116: Wow, you really know for sure nobody wants or needs parcels smaller than 512sqm, huh? What a narrow-minded attitude.

  115. cat cotton says:

    @117 well given the fact that most 512 lots only support 113 prims I really don’t see much of a non commercial need for smaller plots either. I would however suggest that residential and commercial zoning within SL would be one such suggestion. I have already begun to zone areas within my control/ownership with enforceable covenants. That however is not possible on the mainland and only can be done by LL. I would prefer we had the same abilities on the mainland as we do on estates in order to enforce zoning issues.

    Cat Cotton

  116. Balpien Hammerer says:

    #103, wrote in part: “The item being discussed was […]in fact an environmental sound orb. ”

    Yes, I understood your intent and object to it. Decent interesting sound genertors (orbs, cubes, whatever), are a tapeestry of multiple sound events. Yes, you can have a nice sedate recording that requires a sound requeue once an eon, but something that is not boring after a few minutes of listening will have a richer (and faster) set of sound events. Making the proclamation that sound requests at more than a sedate rate is bad is what I call to question. Let’s stop castrating functions and thus limiting creativity to mediocrity.

    Now what are the documented parameters to playsound in terms of requeus per second, sustained and busts? That’s what is necessry to make a decent test suite.

  117. Ryu Darragh says:

    That isn’t the problem, Balpien. llLoopSound works perfectly. llPlaySound maybe kinda sorta works whenever it feels like working 😛 Even my Unbreakable Sound player, which plays fine on places like Ahern, still messes up at random with only 2 others nearby >.<

  118. Crash rate reported way higher than average.
    Put that with the unresolved freezing condition on the Mac Book Pro
    and I am going out tonight, just not in SL.

  119. Tegg B says:

    We don’t need to abolish 16m plots just make it so the minimum land you can own in any sim is 128m or 256m or 512m
    That would fix the adfarmers, the ideas of those who strive to own a 16m in every sim are just making things hard for actual sim residents and businesses.
    If you need a 16m elswhere to put backup servers on, put it on a friends land or rent 16m from someone.
    Make advertisers pay residents to put signs up, that way only the ones that make actual profit will remain.
    At the moment Advertising in SL is like email spam, they do it because it’s free, and some of may even be using an exploit to get free land.

  120. Prospero Linden says:

    Velouria — client problems aren’t really on topic for this, but the freezing in MacBook Pros comes from the fact that the NVIDIA 8600M graphics chip has a tendency to overheat– see, for instance, this article. I’m frustrated frequently by this myself. Steps you can take:

    (1) Make sure that you have good ventillation around your MBP. Different people report different results; some have claimed worse problems that I, some not as bad. Using a laptop stand may help.

    (2) Turn down your rendering qualities. Yes, the graphics chip can handle “high” quality, but frequently the cooling of that chip cannot. Try going down to “Medium” quality in Edit:Preferences:Graphics.

    (3) Check the “custom” box on that preference page, and try unchecking the “atmospheric shaders” box.

    (4) Under World:Environment Settings:Environment Editor, select “Advanced Sky”. From that, select “Clouds”. Turn *off* “Classic Clouds”.

    You can also try turning other things down, e.g. the rendering quality of Linden trees. I can’t guarantee that any of this will fix your problem, or even help, but these things will reduce the amount of work your graphics chip has to do, and may reduce the heat that the graphics chip produces enough that your Mac doesn’t freeze all the time while in Second Life.

  121. Jonathan Snow says:

    I got a report from one of our landowners (yes from a sim that is running on the new version) that he found that most of the building he did yesterday was gone when he arrived today.
    No, I will not report it to Jira. First make that system easier for the avarage resident AND start listening to what residents want you to solve.
    I gave the standard advice as I gave up long ago that individual problems get solved by Linden. MAKE COPIES WHILE YOU BUILD. At least every hour. Especially when Linden is ‘messing ‘ with the servers again. 🙂

  122. Jonathan Snow says:

    And by the way: The sinking while hovering bug is back on this new server.

  123. re: Prospero on #70, and a tiny note for Mr. M Linden, IF indeed that is his real name…

    “If you want to chat with a Linden directly, please look at the list of Lindens who hold office hours. The topics that each of them talk about are listed on that page, so you can find somebody who is most likely to be able to talk about the topics you want.”

    Hiya Prospero! Fancy meeting you here!

    This, above, is more or less the same thing you told me when I was at your most recent Office Hour, this past week, and asked whose job it is to deal with Resident ‘communication issues’. I am a Resident and have some questions and thoughts on this very subject, ones I’d like to share with the good folks at LL, so I asked. You, Prospero, kindly referred me to a wiki page listing various Lindens and their individual office hours, pointing out that Everett Linden is the guy who deals with Resident communication channels.

    Unfortunately – o harsh fate! – no sooner had you said this than someone piped up and said ‘I think he’s on hiatus.’ Meaning Everett Linden.

    Guess what? Sure enough, the next day, on that wiki page under Everett Linden’s name there was a little note saying that his next Office Hour – which was to have happened the day following yours, Prospero – was canceled. Canceled! And I’d planned on being there, too, bright-eyed and bushy-tailed (well, 50% of those things), making use of existing proper channels, presenting my questions and concerns to the appropriate Linden at the appropriate Office Hour, as per LL-sanctioned protocol. Darn the luck! These coincidences are downright (un)canny, I tell you.

    So, ok, Everett Linden, a.k.a. ‘Resident communication channels’ guy, is unavailable. (Temporarily? Permanently? No indication on the wiki page. Mysteries abound!) Given that LL’s ‘Resident communications’ guy is out of service – and how is it even possible that there is really only ONE guy specifically assigned to interface directly with Residents? that’s boggling in and of itself, to be quite frank, and why shouldn’t we, here among friends? – is there any word from the top on what exactly ‘spending more time with Residents’ means? Those were M Linden’s words, recently, when talking about his vision for how to make things better; that more and better solutions would emerge ‘as we spend more time with Residents.’

    Spending more time with Residents! You like us! You really really like us! The mere prospect sounds so snuggly that I almost hate to ask, but… well, if M Linden can tear himself away from the shiny new marketing and ‘monetizing’ ideas that have him so giddy, I sure would appreciate it if he could take a couple minutes and clarify his comment. Two specific questions, M:

    1. When you say ‘…as we spend more time with Residents’, who exactly is ‘we’?

    2. What qualifies as ‘spend more time with Residents’, i.e. when and where?

    Lastly, in the event you were speaking rhetorically, please allow me to withdraw those questions and ask in their stead this one: WHAT KIND OF MONSTER ARE YOU, LEADING US ON LIKE THIS?! You know, rhetorically.

    That is all, and not a moment too soon…

  124. Urantia Jewell says:

    @126 helena whittlesea,

    Perhaps Torley Linden can help you. His office hours are at 12 noon on Thursday’s. 🙂

    http://wiki.secondlife.com/wiki/Office_Hours#Torley_Linden

  125. Ah, so this is why business has caved in over the last couple of days. I knew I was feeling too prosperous! This should slow things down nicely for a while. I’ll let you know when things start going to well, and we can do this again!

  126. DhouZhang Fhang says:

    Come on guys, get your act together. I had a good session of over an hour and since then I’ve crashed three times in less than five minutes. This is clearly a server problem – fix it.

  127. Coventina Dalgleish says:

    It must be Sunday the game is borked again now texture searches are non functional really makes it painless to find a texture in 10,000 with subtle differences.

  128. Ron Crimson says:

    No server problem here right now…

  129. Balpien Hammerer says:

    So sad to report that the viewer memory leak is back. See: http://jira.secondlife.com/browse/VWR-7376 (which, BTW, has not been assigned to anyone since it was opened two months ago). The latest main viewer, 1.20.15… has a memory leak that causes it to crash after about an hour. You can speed up this process by visiting different SIMs. The texture cache builds up and up then after a while the viewer crashes. Crash dumps sent.

  130. Bud Parnall says:

    Sluggishness is back. Scripts are executing very slowly on the new sim version.

  131. Taff Nouvelle says:

    since the question was brought up about proper channels here, I think it should be on topic to ask who I should speak to about racial discrimination by LL.
    It seems that LL are anti european, making sure that all europeans pay at least 15% mopre than anyone else for all services.
    I would like an explanation before I send this to the race relations board in the UK.

    If the reason is that LL are a uk compant, then everyone should be charged VAT and have to claim it back from the company if it is not due, or prove that they do not have to pay it.
    If LL are an American company, then they should not be charging a european tax, which is it??
    I have asked this numerous times, and will continue to ask until I get an answer. Or will LL do their usual trick of ignoring hrad questions and banning the questioner.?

  132. Prospero Linden says:

    Re: VAT, that’s very far outside the area of what I know about, so I cannot address that.

    Bud: Please file a JIRA on our issue tracker, including the data, and, ideally, a script that performs differently on two different regions (and the name of the regions). If we are to search for problems, we need to be able to reproduce them.

    Helena : I would be surprised if M were monitoring this thread. We’re all too busy to monitor every blog thread.

    Jonathan : if you are the estate manager, you have access to concierge support. You can take advantage of that in addition to using our public adventure tracker.

    Balpien : this blog post is about the server, not the viewer, and I’m not the right person to ask about the viewer. Update and comment on the JIRA if you have additional information, or file a new JIRA if you see a new issue.

  133. rsgoldbox says:

    The hosting service where Zubux is hosted it was hacked.
    Probably, it will have damages in the database.

    Zubux was not hacke, but the hosting service. All the sites of PTServidor.com were.

    We are to try to solve the situation

  134. Jayme Llewellyn says:

    Here’s a novel idea, Lindens….. how about getting a server system that is universal? That is – it works with all types of service connections and doesn’t pick and choose at whim who it’s going to work with depending on distance? I’ve had eight – yes- crashes today with my best time being an hour and a half. This is just getting absolutely rediculous. Get in some people who actually know what the hell they are doing because obviously those you have working on things don’t. This rolling restart has been going on for the last two weeks. Admit it – you have no idea what you are doing.

  135. viviennegraves says:

    @134: Go here, read, and learn: http://europa.eu/scadplus/leg/en/lvb/l31044.htm

  136. Ron Crimson says:

    “This rolling restart has been going on for the last two weeks. Admit it – you have no idea what you are doing.”

    With all due respect, you have no idea what you’re saying. There hasn’t BEEN a rolling restart – the most of the grid that was ever subjected to a restart is 10% of it. And it so happens that those 10% are running 1.23.4 (was it 1.23.4?) without any serious trouble at this time…while the other 90% continues to run on 1.22.

    Maybe you should investigate the cause of your crashes…make sure your system is in good order before you blame SL, it *is* resource-intensive and *will* cause software problems or even spurious hardware issues to pop up whereas most other programs may not. There are a *LOT* of variables that affect how well the viewer runs, and the majority of them are under your control.

  137. Kugel says:

    Love the way some folk have stoppped posting now they have nothing to complain about 🙂 This softly softly approach is an excellent idea with 10% of regions upgraded…. I just hope those regions (wherever they are) are well/heavily populated with a variety of “industries” … so that the tests are begin ran against region population, stores, clubs, etc, etc so there is a capability for the user base to (contstructively) report back.

    Good luck LL and hope the testing is going well.

    PS Will this latest forthcoming server update fix the problems with many “Huggers”..both old and new.. where you end up humping your “hugee” ? Something to do with avatar collisiob detection?? Dunno. Just curious :)))

  138. Argent Stonecutter says:

    Velouria@121, Prospero@123: I have a first generation MBP and I have to remove the battery when running anything computationally intensive to keep it from overheating. I have already replaced the battery once under warranty, and it didn’t help. In any case, you might want to try that… also try using something like “Temperature Monitor” to warn you when you’re likely to be overheating, and “SMCFanControl” to overdrive the fans and improve cooling.

  139. Pingback: WTF Items stolen directly from your inventory? - Page 2 - SLUniverse Forums

  140. Very Keynes says:

    @126 helena whittlesea
    “is there any word from the top on what exactly ’spending more time with Residents’ means? Those were M Linden’s words, recently, when talking about his vision for how to make things better; that more and better solutions would emerge ‘as we spend more time with Residents.’ ”
    I bumped into M Linden in world yesterday, exchanged pleasantry’s as with any other resident and he tipped the Musicians that were playing.
    Seeing as it was a Sunday afternoon for him, and he took the time to mingle with the residents, I for one can say that he is standing by his word. And was a very personable chap to chat to I might add.
    Overall I think I have seen more lindens in world in the past 2 months than I have in the past 2 years, especially the high profile ones, so my guess is the first team have taken him seriously and they are spending far more time with the residents.

  141. Renee Faulds says:

    well 5 for 5 huh, lol.

    …………maybe August will be better.

    but then again I’ve been saying that since February.

    What’s up with this new exploit anyways??? People stealing directly from others inventory? Why worry about the borked asshat servers when people just go in and steal directly from you now.

    Who is going to be the “next” communications Linden now that Katt has suddenly disappeared?

    When are you opening up comments on the status blog like you said you were two or three months ago?? Never I suspect that was just another lie.

    Lastly – pretty low that you strip the script from my ‘FIX SL’ rotating billboard !

  142. Jayme Llewellyn says:

    Okay Ron Crimson, you seem to have put yourself forward as the expert in everything Second Life – explain to me this. For the two weeks previous to this, i was experiencing a “normal” crash rate – 2, maybe 3 crashes a day tops. Now, I have all of my SL viewer settings at absolute minimum – yes this mean slow rez, but that’s something i can put up with. Yet yesterday, I was hit with eight crashes – the sole time i used the regular client as opposed to the Release candidate, i crashed after 45 seconds. This is telling me that the problems are not at my end. I will even tell you the places i tend to frequent – Freebie Dungeon at Deva Loka (if i need to get some clothes), Cafe Trivia at pimushe (earning money, playing trivia and socialising) and Briarhaven (where my avi’s crash). Please, come and visit these places, Ron. You’ll find me more often than not at Cafe Trivia playing one of my five avi’s – Jayme Llewellyn, Jayse Llewellyn, Kittyn Honi, DouZhang Fang or Dee Tunwarm. Come on, Ron, Lindens, visit these places and if i am there, talk to me avi to avi – none of this IM crap. Come and experience my world. Because it may just be that – shock! Horror!! – Linden Labs are at fault in this case. Universal servers that acknowledge all connection types and providers are essential for a game like Second Life – get that through your heads. And Ron – the rolling re-start has been going on for two weeks, it’s just been stopped and started that many times it goes beyond being funny and into the realms of the rediculously inadequate.

  143. Ron Crimson says:

    Well, you’re entitled to your opinion. But I try to help out a little and suddenly am (sarcastically) called an expert… Good grief, far from it. I dunno why I sometimes even bother around here.
    Feel free to send me the landmarks in-world and I’ll check them out… especially Briarhaven, since you mention that’s where you crash. BTW, the main viewer (1.20.15) and the RCs use separate settings files so it’s always possible some setting in the main viewer (that’s different from its RC counterpart) is causing your frequent crashes… also, it’s not a good idea to have ALL the settings at minimum. If (and I’m guessing here), if you took the network speed slider to minimum I strongly suggest you take it to maximum instead, otherwise you’re more or less just choking your bandwidth and that’s why things rez slowly – plus, in my experience, slow connections are partially at fault for crashes too.
    As for the rolling restart, well, it was never funny to begin with but what’s your point anyway? There were unexpected problems and LL doesn’t like them anymore than we do. *shrugs* Nothing ridiculous about it, you can’t always expect a smooth ride, especially with a system as huge and complex as SL…but then, this has been said many times before.

  144. Jayme Llewellyn says:

    Ron, when i refer to Briarhaven being where my Avi’s “Crash” i meant it as in sleep. most of my inworld crashed have occurred at Cafe Trivia. I will PM you the locations when i’m on with on of my “older” avi’s.

  145. Well, I’m running the 1.19 viewer and my crashes are worse than ever. I’ve voted on Jira’s that don’t get resolved. I’m convinced and belive are the root of some of your issues.

    You can run a 1.19 client for example and get out of memory issues after running it for about 8 hours…. that equals a mem leak. For me to work in Second Life, I have to run my graphics at bare bones now…

  146. Tuyula Cazalet says:

    1.20.15 works almost as nice as 1.20.14 RC, only the zooming is borked? Camera gets out of control when zooming in, especially from some distance. Switched back to the RC, and all is fine again.

  147. Tuyula Cazalet says:

    Tried to fill in a Jira about this issue, but i cannot login with my regular password? (noticed this issue before when trying to get help with support, i had to change my password, and back again). Possibly because of special characters.

Comments are closed.