Region Data Corruption / Loss Problem Resolved

On January 15th we started noticing a small number of Regions that lost many or all of their objects after the Region crashed. This required us to rollback those Regions in order to recover the Region’s assets. This ranged from about 6 to 15 Regions per day. This coincided with a spike in Resident support tickets regarding large numbers of objects vanishing from their Regions. After an intensive effort we tracked the problem down to an Internet networking hardware problem between our data centers at one of our service providers. The problem was reported to them and then promptly fixed on Saturday, February 2nd. After heavy testing we have confirmed this problem has been fixed.

This entry was posted in -Miscellaneous, Operations, Service Status. Bookmark the permalink.

102 Responses to Region Data Corruption / Loss Problem Resolved

  1. Meade Paravane says:

    Yikes! Sounds scary!

    Hamilton, does this have anything to do with the website being unhappy? Those problems seem to have started around the same time…

  2. luquinhas says:

    eu queria jogar mais nao esta tando nao…

  3. Wagahai Oddfellow says:

    Have all island owners who had rollbacks been informed about them? ..and what happened to any new items placed on the land between the time of the lost items and the rollback? ..are they now lost?

  4. Claudius Hedrick says:

    All Rotations with switch or direct touch are not working longer after the last rollup. Very heavy!

  5. Nyoko Salome says:

    heavy (the prob) but beautiful (your response)… 🙂 … are there caching probs between far external routers and the ‘home base’ servers??

  6. Nyoko Salome says:

    p.s. i don’t want that to sound like my own ‘glossing over’ the situation; i’ve had ‘severe’ inventory loss myself (at least, not with my own products/inventory that i have backed up, but with other inworld sellers’ products i had in inventory/rezzed out inworld/lost to risky situations, bad timing when the ‘official word’ was ‘do not rez non-copy items’, etc etc…)

    inventory loss is a nightmare for any sl citizen. for those selling wares, doubly so… :\ i’m wearing both hats here.

  7. Stephen Zenith says:

    I was hit by this on one of my sims, it actually happened to me on the 14th but I didn’t request the rollback until the 15th so I might have been one of the first to experience it.

    Trying to organize a rollback, warning all your residents about new work being reverted when they don’t always speak the same language as you is not fun, believe me. The rollback itself went fine, and was done very quickly after the request went in.

  8. MelissaJeanne Flores says:

    Well all I know is I hope things get fixed soon cause after the roll over, sl its messed up compeletly for me. I can’t rez the world, I don’t have inventory, I can’t talk to people, I can’t walk I basically can’t do anything and I’m not getting much help either with it. I hope they can fix things soon.

  9. Oh wow – that explains the wild rollback my sim experienced. Sim crashed on Saturday and the rollback was something like two or three hours prior.

    OUCH! Had a resident (sim resident who owns land on my sim) screaming about how she had just placed down L$12,000 worth of new furniture (no copy, of course) just before the crash.

    I know the rollback was something like two or three hours because I was working on swapping out my rental boxes (one brand for another) and when I returned after the crash – the old stuff was there LOL.

    I’m not whining.
    I know sh*t happens, so I take it all in stride. LOL

    So here’s a question – as I know this must have happened to a lot of people (and I really don’t expect an answer here, but I am *positive* Lindens do in fact read many if not most or all of these responses)…

    There was a huge issue of missing inventory previously, and Linden Lab engineers (or whom ever) were able to restore most of it post-haste.

    Will an effort like this be repeated?

    I’m fortunate that I didn’t lose anything. but I certainly do feel for all those who might have. L$ *is* real money, when Linden Lab chooses to acknowledge it or now. 🙂

    Thanks, Lindens.
    Unlike many whiners on this blog, I know I speak for a LOT of peaople when I say thank you, I know you *are* all working very hard to make things better and that yes, you do care.

  10. Berry Steinhoff says:

    Thanks for the update Hamilton. Some extraordinarily difficult times for Residents and a number of levels atm. Good to hear you are actively addressing the problems.

  11. Immortalus Blessed says:

    Yesterday all my pose balls, dances, & about half of my clothes went missing. I keep hearing i’m gonna get these items back but still nothing. Does any1 have any idea when those items will come back 2 me? I look ridiculous standing there not dancing & I would hate 2 have 2 respend all that money. I’M A GIRL & MY WARDROBE IS LIMITED, IT’S A NIGHTMARE =oO

  12. Wayfinder Wishbringer says:

    Got no problem with this folks. Hardware problem can crop up at any time. Problem recognized, found, fixed. Good job on this one.

    Is there someway you could encode a detection system that would let your people know when a piece of hardware goes out? That might help future problems. 🙂

  13. Belial Zadeh says:

    do I get the items I lost back?

  14. jade carnot says:

    no joking i have had 110k items missing since last week! relogged/rebooted/manually deleted caches/reinstalled sl/tried nicholaz and on rez viewers! i went to the additi grid and found the same thing there even though it was a months old database!

  15. Cocoanut Koala says:

    How about an update on the 503 problem that is plaguing the forums, making accessing transaction history dicey, along with, apparently, buying money (and possibly even signing up for sl)?

    coco

  16. Michael Timeless says:

    I know that there will always be inventory issues. I accept that computers and the humans who use them are fallible. But how does Linden Labs “recommend” we protect ourselves from the inventory loss? I bought several “storage” devices from two different inworld vendors and have lost them with tens of thousands of lindens worth of things that I wanted to save. Now the storage devices themselves have gone missing.
    I’ve been working with computers since the early seventies and I have been burned before…who hasn’t. However, in RL I make backups (I’m a fanatic about it and have backups as far back as my college papers). Here these is no way to do this that I know of. I am reluctant to sink another 1000 or so real dollars into this unless I know some way to protect myself without violating the rights of content creators. Can anyone make any suggestions to this ongoing problem?

    Mike

  17. Michael Timeless says:

    Sorry about making two posts in a row but another thought occurs to me.

    Is there a way for vendors to verify when we lose items? I would love to be able to show a vendor that the 6000L object that I bought from them is gone from my inventory so that they could replace it. I realize and accept they would be reluctant to do so without proof of the actual loss. I can show the transaction record but this is often not enough to prove actual loss. Any suggestions?

    Mike

  18. Solomon Nightfire says:

    Is there a way to back up inventory, without utilizing in-world storage? Sounds like I need to get on the curve about this…

  19. coventina dalgleish says:

    Yes hardware problems can pop up any time thats is not the problem I observe the 18 days to rectify the problem is. Yes there were roll backs but the massive inventory loss is the problem for those who lost items.

    I am sure they were purchased with funny money unfortunately this money has a real world value and thus represents a real dollar loss to the player. Maybe it is time for the Linden Lab insurance program where for penny’s on Linden you can be assured of item returns, oh wait Lindens are penny’s.

  20. Zlad Voom says:

    A list of the regions would be great, so everybody know is affected.
    And how about recently added items ??

  21. Moll Dean says:

    oh man

    That explain everything that was not told last week…
    Today we have more problems with inventories. Is that one related?

    THANK YOU

  22. Restless Roizman says:

    Thank you Lindens for telling us everything is fine as usual, could you please tell me when you will be informing us that all the crashes timeouts and slow loading i and many friends have had over the last few weeks will be fine?

  23. Restless Roizman says:

    oh yes and are still having that is?

  24. Lee Ludd says:

    #18 I don’t think there is a way to prove you don’t have something. It would be nice if LL would provide a method to check that a given key no longer points to an existing object. Then sellers and buyers could figure some way to keep a record of a key in RL and use it to verify actual loss when it happens.

    Many sellers, however, will replace missing things without requiring any proof that they have disappeared. The next time before you buy something expensive, you might ask what the seller would do if the object disappeared, and base your buying decision on that answer.

    I, for one, always replace objects my customers say they lost (or even broke.) No questions asked. I want customers to come back, and I want them to recommend me to their friends. Replacing a lost item doesn’t cost me anything, makes my customers happy, and entails only a tiny risk that I give something away that I might have sold.

  25. Digital Digital says:

    Are you going to be letting estate owners which estates had this problem, because I am sure some of us would like to know.

  26. Lilly says:

    nope, the 503 Service Unavailable
    The service is not available. Please try again later.

    is still there…… just had it again

  27. Chavi Skomoroklov says:

    And the lost objects?? Will they come back???
    Please Linden, provide your residents with full information, even if it’s bad news!

  28. Chavi Skomoroklov says:

    You know, reading the posts here above and in many other reactions, i get the feeling of frustrations and anger. Is there any way Linden will ever notice that clients here are PAYING clients?? When I buy something in the store that doesn’t work properly, I go back and get my money back. At least they attend me and give me answers. Here I NEVER see any answers, let alone solutions.

    Start forming a team at Linden Labs that know something about relation management, act towards your clients, see to it that creeping delays are history and stop with that silly frase “thanks for your continuing patience”. Your clients want solutions, fair information and solutions and/or refunds when applicable. But at least get back to your clients!! We are paying lots of money that you use for food!!!

  29. Moll Dean says:

    # 19

    SOLOMON

    There is a way to backup you inventory and build/edit things in offline mode. I named it OFFLINE SANDBOX (lol).
    But it is not a reality because during those 2 years LL is not avaible to listen us.
    As I had commented in another post, the copybots are able to clone your objects (not scripts) and after you can open your simulator to a limited edition. I never tried that before but looks like those cloned objects are storaged onto your HD. And that is possible to open the simulator even disconected to the internet. Half way done?
    My suggestion is that LL gives us the ability to backup our creation, copyable objects and not transferable ones. Just not copyable objects combined with transferable should not be allowed to backup.
    Even saving them in HD those file would required you to be runing your simulator (online or offline) and you should be loged with yous password. Doing that we could not be able to share those files over the web.

  30. Stacey Sugar says:

    Chavi

    Thats how I feel

    :/

  31. That’s amazing, you mean your ISP FIXED the problem just days after you reported it?

    Too bad you can’t provide the SAME LEVEL of EXCELLENCE to your customers.

  32. 23skidoo Outlander says:

    So is this related to the fact I keep getting “Login Failed” errors whenever I’ve tried to log in today?

  33. janeforyou Barbara says:

    Server Issues : Cant read profiles- payments dont work
    LL pz fix?

  34. Claireana Munro says:

    Problem is not solve for us yet H. Linden. We still have several items on the lose and object flying on our land. Wich is a all mall. We missing walls, windows, objects got duplicated like crazy and crash the mall again and again. can you please take action. check in my profile. Danzer Park Mall is a all mess. besides that we are unable to rezz objects and edited and so on 😦

  35. Budd Raymaker says:

    Nice work keeping those problems under control. SL is a much better place since this fix. Thanks LINDEN LABS

  36. Jonathan Sprawl says:

    I am just at a loss to understand the Linden philosophy here, and why it’s tolerated.
    Of course hardware problems will happen. BUT THEY SHOULD NOT RESULT IN DATA LOSS. A decent software architecture will include transactional protection. Database replication is not a new art, why do you seem to be reinventing the wheel without learning from the last 20 years of evolution?
    There are many large-scale widely distributed systems doing more complex things than the SL grid, and on larger scales. They don’t lose data once a transaction has succeeded. Period.
    Is LL staffed by young, enthusiastic, but not very experienced coders? Where are the software architects who have built systems like this before? Stop hiring execs from the gaming industry, and bring in a few gray-haired veterans of second-generation Internet companies and learn how to do this stuff right. Please.
    Otherwise, your services is just not going to continue to be worth paying for and it won’t scale.

  37. Cloe Despres says:

    Well, I don’t know if my problems got something to do with it,but SL is makin me go really mad!!! It’s about 3 days I’m having serious difficulties to loggin. When I’m offline everybody’s keep seein me online. And I could not loggin.I spent almost a day to loggin again.The advice always tells me my account is not avaiable. I’m pretty sure that my password wasn’t stolen,because I always change it.Sincerelly this bug is almost makin me give up my second life…

  38. Penguinfan Beck says:

    Payments denied- Money transfers stale, objects not being received when purchased. Teleports not working *AGAIN*. Need things fixed here once and for all. PLEASE!!! Two plus weeks now of the *SAME ISSUES!!*

  39. Anarchy Tigerpaw says:

    congrats, SL just ate 300L$ you may have fixed damned regions, but doing so is causing huge problems for the players now, great job, and i agree with ‘Jonathan Sprawl’

  40. sloopy cooder says:

    Why was the network issue not found for nearly 2 weeks? Network management systems have had the link monitoring problem solved for many many years.

  41. Ravanne Sullivan says:

    I have one question for Linden Lab.

    How does a group that can not manage their own mission critical network and their own web site claim to be the future of the Internet?

  42. Sera Lok says:

    I would also like to know if sim owners were notified of these rollbacks…

    Also… you have noticed the “show in search” checkbox is broken… right?

  43. Drako Nagorski says:

    yay.. now can you turn the friends online page back on? or maybe give us a reason its turned off? its like you’ve forgotten that it exists…

  44. Drako Nagorski says:

    #38
    I am just at a loss to understand the Linden philosophy here, and why it’s tolerated.
    Of course hardware problems will happen. BUT THEY SHOULD NOT RESULT IN DATA LOSS. A decent software architecture will include transactional protection. Database replication is not a new art, why do you seem to be reinventing the wheel without learning from the last 20 years of evolution?
    There are many large-scale widely distributed systems doing more complex things than the SL grid, and on larger scales. They don’t lose data once a transaction has succeeded. Period.
    Is LL staffed by young, enthusiastic, but not very experienced coders? Where are the software architects who have built systems like this before? Stop hiring execs from the gaming industry, and bring in a few gray-haired veterans of second-generation Internet companies and learn how to do this stuff right. Please.
    Otherwise, your services is just not going to continue to be worth paying for and it won’t scale.

    excelent point… im gonna spam this to LL in every open blog they start 😛

  45. Marcus Antonelli says:

    Why does it take two weeks? Because they’d rather give us shiny new things instead of fixing what’s broke, like Friends Online (again), unable to teleport, etc.

    Marcus

  46. Kerik Rau says:

    @ Jonathan Sprawl

    Because they know that if they offer poor service they still get paid tier and people will still dwell in the grid. After all if more new people come and leave earlier they get to boast about their increasing numbers. Unless their is a competitor you will likely not see LL even try to keep customers happy. They will continue doing as they please so long as the money keeps coming in.

  47. Proxima Saenz says:

    I think it is time for Linden Lab to step forward and face with all the problems of SL:(
    Which means they have to stop running away.. and hide.
    Communicate with the Residents and inform us WHY something is NOT working and WHAT you are going to do about it.

    Furhtermore.. delay any NEW projects and SOLVE the old PROBLEMS and BUGS. All we ask is a STABLE Second Life.

    Please Linden Lab.. DONT ignore our advices.

  48. Moneca Sands says:

    LL recommends that you have an alt and to back up whatever you can in your alt’s inventory… may help in some cases. I have done this to some degree but it’s a hard way to share/save stuff esp. depending on the various combinations of perms. Alas its an imperfect world 😦

  49. So, are the newer problems related to the three times asset server “upgrade” fiasco, or the simultaneous attempted rollout of Windlight, Havok4 and Mono? Or just incomoetence?

  50. Kat Hynes says:

    Wheres MY STUFF ?

  51. Wanja Lubitsch says:

    Jonathan is totally right.
    Nowadays there is no excuse for data loss, specially not for valuable data. Data safety is affordable for a reasonable price and LLs practice (Letting you pay for things and let it happen that they disappear without even think of replacement or compensation) is very close to fraud.
    I don’t pay for this “service”, but I really wonder why no one of you, who some people here call “whiners”, do not go to court. Maybe your loss was not significant, but if you believe that the TOS is waterproof for American laws, you’re probably wrong. If you lost a few-thousand USD you should find a lawyer and get them back.
    Maybe a few lawsuits would change LL’s philosophy.

  52. Lady Tairov says:

    I am rather new and just starting purchasing items today. The systems subtracted money for tats I was purchasing but then crashed before they could load. I went back and bought them again and it actually worked that time. I went and purchased a pair of shoes but now they only show up as an object. I don’t give up easily so then I went and made a more expensive purchase buying a full avatar…well that was a mistake. The system crashed again and I ended up with a pair of shoes and avatar objects but nothing I can actually wear. I wonder if I ever want to make another purchase. It seems like a big waste of time and money. From the thread I have just read it looks like this is going to be an ongoing challenge.

  53. CC says:

    they dont care about data or inventory losses they cant even post a bloody warning when they have had many reports of issues from people that would save us all the losses or cut down on many if we knew they have issues POST THE ISSUE WARNINGS IN GAME FOR CRYING OUT LOUD tell us to be careful not to rez even when u get ONE warning so sick of the lindens THEY DO NOT HEAR US THEY DO NOT LISTEN

  54. Wanja Lubitsch says:

    Lady Tairov, tthe object you have in your inventory is probably the box the purchased items are packed in. Got to a place where you are allowed to rez objects, rez and open them. Your shoes should be in the shoebox.

  55. nomoresecrets says:

    seems all the LL-staff need to hve a visit by rl-police… wonder where all the things going. lindens, expensive stuff…. AND NO!!! ITS NOT NORMAL TO LOSE THINGS FROM A COMPUTER. it sounds criminal!!

    since 1984 am in computers, never before i saw things like at LL.

  56. Alisha says:

    Ahh,

    That would explain why I lost a few hours of customer guided custom work Saturday night.

    As I was building in My sim, which alone costs 295 usd a month, surely a kind blue box asking my to “save please”, is not out of the question.
    Saving and moving, verses crashing and losing…

    *Stumbles off into the dark, trying to feel Her way*

    Alisha

  57. misslove85 Short says:

    I’ve lost a animantions dance from Sine Wave Island, Sine Wave Island (130, 124, 305) called “Release”. I paid 250 lindens. I was in Animations Island when i lost it. Please help me for return me the animations dance. Thanks you very much.

  58. nomoresecrets says:

    miss , the lindens steal 30k linden from my account, and a huddle with dances worth 14k …. dont think the thieves will give back!

    yes, am very angry. and believe me…. this will have consequents for all the staff !!

  59. Proxima Saenz says:

    *sings*

    Linden Lab.. cant you see the problems.. wake up.. listen to your Residents.. Give the stolen Objects and Money back..


    I lost many things too… but somehow I got them back a few weeks later.. maybe a Linden was in a good mood.. or it was just luck ^_^
    Anyway, seriously Linden Lab.. what the heck are you guys doing??

  60. Nita says:

    If inventory items were stored on our hard drives, it would be our responsibility to backup and save these items. However, they are not, and we don’t have a choice in the matter (at least, not that i know of). We spend real money on SL and store our inventory on SL servers. SL should have backups of our inventory so that we do not experience data loss. Even a 30-day backup would suffice. If people lose money they spend here, then they may stop spending.

  61. Bucky Barkley says:

    @38 >> Is LL staffed by young, enthusiastic, but not very experienced coders? Where are the software architects who have built systems like this before?

    No, LL has some pretty decent hardcore people.

    The root of the problem is twofold.

    Cory was not a good choice for CTO. He was the skilled contributor that should be akin to an Apple Fellow. I dont see that he was an effective leader on the inside, judging from the problems we have seen over the last year and what got released. Nor was he an effective communicator with the residents, based on the infrequent and technically plagued town halls, and the very sporadic postings to the residents in this blog. Being the Alpha Geek doesn’t mean you can run the organization.

    The other root of the problem is the Tao of Linden – specifically the “pick your own work” sentiment. Stack that up against the problems we have seen over the last 18 months. Does it seem to be something to tout?

    The real solution is to have a true leader as CTO, to ditch the Tao of Linden for engineering, and to do some housecleaning. If there is some tech staff who insist on doing their own projects instead of fixing some crucial stuff now, then they need to go somewhere else. There are too many eyes on this stage to be running things in such a naive / idealistic fashion.

  62. Hamilton Linden says:

    @1 This problem is not related to the web problems, like the 503 errors.
    @3, @44 When this problem occurred one of two things typically happened. The most common was that the Region wouldn’t restart and we immediately rolled back to the previous saved state, which was usually less than an hour old. Otherwise, if the Resident requested a rollback via a customer ticket due to thousands of items being lost we rolled back.
    @5 No this is was not a bad caching problem, it was a hardware problem in a router.
    @15 This problem did not cause inventory loss, only items that existed in your Region.
    @29 The lost objects in your Region will not reappear unless you request a rollback of your Region assuming it was affected. If you see many items vanish from your land please file a ticket with support immediately.
    @34 This is not related to login

  63. Ann Otoole says:

    fyi whatever holds all the textures we paid L$10 each to upload is borked. hope you got a backup and are planning to do an overlay restore or refund a lot of L$10 upload charge fees.

  64. nomoresecrets says:

    Hamilton… you need to have backups. in the LL-tos you say its our Stuff!! and u didnt secure them… you childs need a lesson!!!

    what your doing to your CUSTOMERS!!!!!!!!!!!!!! isnt funny anymore!

    u need to learn what customers can do!

  65. .xo says:

    life lessons:
    1. all investments involve risk.
    2. expect the worst when you sit down at a computer.

    k ?
    now stfu, plz.

  66. Lina Pussycat says:

    @68 …. Seriously now? They can roll back and your complaining that they need backups ? What exactly do you think a rollback is? Its a backup of the server state at a diff time period in case issues happen like this.

  67. NIKOLAI KIDD says:

    This issue brings me to a question I have always had..If you own an Island Estate, with tons of personal assets or even just your personal items, why isnt there a way to download and store thenm to your home computer? Terabyte Drives are cheap now, and could easily do a asset backup daily if desired, like a traditional backup only adding new items. This would solve a lot of issues, which are not even solved by a roll back. I for one was in [process of getting lots of inventory ready to buils an outdoor complex, and had also spent hours updating all my rental boxes after the last update killed all the scripts.
    I had terraformed the entire island, and also purchased items to make machinima at great L$ expense. I would have NOT been happy had my region been one of the ones rolled back.
    Hey, I would even be willing to pay for the download, oike a picture charges..it would be a lot cheaper than replacing lost inventory!!!

  68. Ravanne Sullivan says:

    @69
    They can roll a sim back to an earlier state but they do not have that ability with the asset servers where your inventory actually resides, hence no backup of your inventory. Even a sim rollback will not restore an object that has disappeared from the asset server.

    Backup also applies to redunancy, something that seems to be missing in LL’s network design as single points of failure are causing major problems.

  69. Cael Merryman says:

    Get real people. There will be no backups. If you had a backup with a seamless reintegration, you would also have a way of possibly moving inventory items from one avatar to another – the non-transferable kind. The Lindens have made probably more than a hundred dollars on repurchases of lost items from their mistakes and another couple of hundred off of the inability to transfer items to a new av when I decided to change. That’s off me alone. Figure it out, do the math, and realize that there will be no backup – unless you pay them for it.

    The structure of SL, from the inability to rename an avatar (changing prime keys is not the issue it was in 1990, if it is a prime) to other chosen rules, is all geared to current members repurchasing items whenever possible. I doubt they could stay in business if members tomorrow weren’t being forced into repurchasing items for an almost innumerable list of reasons. I know of no other situation like this one.

  70. Youri Ashton says:

    Textures fail to rezz at all, avi’s not rezzing, voice and walking only pretty much works. I have a lot of work to do, i really cant use this :*( Please lindens, i beg you to fix it!

  71. Ravanne Sullivan says:

    66 Hamilton Linden

    You say not related like it means something.

    All we as endusers see is the lose of inventory, whether that is from router issues, database issues or mice eating it we lose things that we have paid for. And L$ do have cost, we either have to buy them with real world money or earn them in some way but they do have cost to us and losing something we buy with then costs us.

    What ever the problems cause, if it looks like a duck, and it sounds like a duck and it tastes like a duck as far as we are concerned it is a duck and just because determine it’s a loon and not a mallard, we don’t care, its a duck, FIX IT!

  72. Kira says:

    Very cool, now if i could only get a sim restart/roll pack in pachylia to get rid of all these ghosted prims on my land . that i deleted but our still using my prim count.

  73. Hiro says:

    it is not RESOLVED!!! Please check it out!

  74. Sffan2 losangeles says:

    RC 1.19.0 is back up and we’re required to use it now.

  75. Michael Timeless says:

    Hamilton
    I am serious about this. I don’t think it is an unrealistic expectation to ask a simple question and get a simple answer. How can I backup my data? If you can’t keep track of it then how can I as a customer backup my own data? Who do I have to ask for this? Can we have some kind of procedure?

    I am tired of having the Linden Labs apologists tell us how great this system is and take up half the posts telling us how the problem is fixed. Why not require real “in-world” names on these posts so we can separate the LL employees from the paying customers.

    Mike

  76. Wanja Lubitsch says:

    Sffan2 losangeles Says:
    “RC 1.19.0 is back up and we’re required to use it now.”

    No, we are not. It is a release *candidate*, not a required update.
    Why do you say that?

  77. U M says:

    If thisis the case then why do I still see some islands with objects thrown 700 meters in the airs still. This is not fixed.

  78. The problems are many. (1) they were loosing inventory for 2 weeks and they did not notify anyone about it so we could provide some minimal recovery procedures (no new create, copy all copyable to backup avatars, etc) (2) the lose of assets from an asset server are not recovered by a rollback… (3) they provide no repayment for the USD lost in inventory that is gone. (4) in the year I have been here they have “fixed” problems… but the “fix” cause more new problems than what was “fixed” and since it caused the new problems too, it fixed nothing. (5) linden labs have been using the population of second life as it’s ONLY QA team for all the time I have been here. (6) they are going ahead with NEW GARBAGE PILED ON TOP OF OLD BROKEN GARBAGE AND IT JUST GETS WORSE. I spent almost $20,000 USD last year running My island and I am looking with anticipation at the open sim and open grid code being created elsewhere. I have a sim on My machine and it has an island….. it is only a very short while before we will be able to run our own grids connected to the central grid (or another grid)…. Lindens I have been with you for over a year, and I have NEVER tried to make a living off of second life because it would be a suicide wish. I am still on a class 4 server and $195 a month but when you decide to not let grandfathered in islands continue at that rate I will be gone…. You have MAJOR problems and in the last year they have NOT been addressed. I called the concierge line and they told Me they did not have any information why My sim was restarted 5 different times in that roll back restart fiasco… If you don’t even bother to tell your help desk/concierge people what they should tell us, then you have simply dropped the ball again and again and again. And yes, I am angry at the “all is well in LLworld” attitude that I get from those who could say something but don’t. Primary import STABILITY… I dont care if y0ou add one new feature for the next year if you will simply provide real working fixes to the hundreds of existing severity 1 problems and not just band aids that “fix” something and cause 5 new things to break. This will probably get purged… first time I have actually let go here and written what I felt for the last year.

  79. Vicki Harding says:

    Ok, Linden Labs have definately decided NOT to hire competent coders or technicians. While the almightly dollar is king here, you are slowely and not so slowley loosing the very very people you want to have a part of the world. HIRE COMPENTENT CODERS AND TECHNICIANS TO RESOVE THE PROLBEMS AND STOP SCREWING AROUND WITH NEW AND UNTRIED GADGETS!!! COMON GET A GRIP FOLKS. I CAN DO BETTER THEN THAT!!! GET RID OF VOICE, HAVOC 4 AND BUSINESSESS WHO DICTATE YOUR MEDIUM. AND FOR HEVEN SAKES YOU ARE A VIRTUAL WORLD, GET RID OF THE GESTOPO!!!!

  80. Cait Lykin says:

    @78 Michael Timeless – “How can I backup my data?” Simple answer. You can’t. And unless you create your own grid and content, or find a grid where all content is full-permission, don’t expect that ability any time soon.

    Any tool, procedure, or method capable of backing up your inventory locally and restoring it to the grid is also capable of copying no-copy objects by uploading them multiple times, and of transferring no-transfer objects by sending the local files to someone else for uploading. And that’s all before other users try manipulating the locally-saved backups, like hacking permission tags, changing Creator names, and taking ownership of property that does not even belong to them. Naturally, content creators would be at a massive loss if this were allowed.

  81. Action Jackson says:

    Boy, am I glad that I never paid for anything on SL. Surviving on freebies and money trees has worked out just fine. No wasted investments, or regrets when the next big virtual environment takes over.

  82. katerinakirax says:

    This may be the wrong place to ask about this, but I figured what the heck, we’re sort of discussing inventory.

    I figure forewarned is forearmed so I’m trying to clean out and back up my inventory. The fact that I can’t move things to my alt in folders is quite frustrating along with the number of no transfer or no copy items I picked up along the way.

    I thought perhaps some of the crashing issues I’ve been having might be related to an abundance of inventory – but my 24K in objects pales in comparison to the 110K I saw mentioned above. But I am determined to minimize the bloat anyway.

    What I’d like to know is if anyone knows offhand is an item an item an item? What I mean to say, as far as the asset servers are concerned is a prim shoe the same as a texture the same as a boxed house or are they all different sizes? Is it going to help me more to delete 500 free textures or to get rid of all my excess shoes?

  83. Tegg B says:

    Chavi Skomoroklov Says:
    “You know, reading the posts here above and in many other reactions, i get the feeling of frustrations and anger. Is there any way Linden will ever notice that clients here are PAYING clients?? ”

    Hmm well 10% are paying perhaps, most are freeriders or campbot goldfarmers withdrawing from SL

  84. Tegg B says:

    Sffan2 losangeles Says: RC 1.19.0 is back up and we’re required to use it now.

    No, you’re not required to use it, it’ only a beta viewer.

    U M Says: If this is the case then why do I still see some islands with objects thrown 700 meters in the airs still. This is not fixed.

    Most of those objects have up been there for at least 12 months, I lost a forklift somewhere over Andraca like thet, so let me know what altitude it’s at if you find it.

  85. Trella McMahon says:

    This was not fixed Sat., and it is not fixed now. If this is related to our inventory loss. I scratch my head and wonder, just where is the rest of my hard worked for copyright creations, as well hard RL $ paid for inventory and just when if ever can we expect it to return?

  86. Bad Bobbysocks says:

    I have no idea what the problem is but the last week or so has been crud I make items pick them up and they go no mod,drop them.. full mod unlink it, check, re link and on it goes,things refuse to rez or take multiple attempts only to log in next time to find dozens our record so far was items appearing a couple of days after WTH?? stale money transfers a friends list populated by hippo’s until you check one of the tick boxes and it magically loads, Fitting prims items why do I lose the z axis arrow on my right leg? the list goes on and on. I did see an inworld message not to use inventory but had customers logging in just after trying to buy items..they didn’t see it. I love SL and the time I spend here creating content but this is getting crazy

  87. Bibi Book says:

    It is really great to hear this three weeks after it started. Just after reading it I logged in and visited my test cinema for some testing to discover an obvious loss of objects. Nothing really important..
    But even if it is something important: after such a long time one cannot do any rollback. These info should have been here on the very first day already. And there should have been a notice not to build without taking in inventory until it is solved.

  88. U M says:

    No this one island just had their objects, structures etc blownup to over 700m

  89. Totem Flow says:

    *Goes to reopen support ticket for the items I lost*

    Expects to get fobbed off yet again….

  90. Tasrill Sieyes says:

    Cait Lykin

    “Any tool, procedure, or method capable of backing up your inventory locally and restoring it to the grid is also capable of copying no-copy objects by uploading them multiple times, and of transferring no-transfer objects by sending the local files to someone else for uploading. And that’s all before other users try manipulating the locally-saved backups, like hacking permission tags, changing Creator names, and taking ownership of property that does not even belong to them. Naturally, content creators would be at a massive loss if this were allowed.”

    Content creators already have to deal with this. I had a friend get all her cloths riped from sl and reuploaded full prem and sold by a scam artist. The decedents of copybot can take everything but scripts and copy them mod or no mod. The thing you fear has been living and plaguing content creators for months now as many skin makers and cloths makers just give up making things. So where will local backup make this worse? All I can see is it at least making sure the we content creators don’t keep losing the crap we made from sl cause we forgot to make and spread copys of it across 7 alts.

  91. Cristopher Lefavre says:

    Doing a rollback on a subsystem alone will of course generate data integrity problems. In the case of region rollback: Loss of no copy items that is “moved” from the inventory to a region.

    But I am sure that internally in the asset server, rezing a no-copy item to a region is just a flag setting and a pointer /coordinates to the region. The object will be in the asset server still owned by whoever owns it.

    A very simple solution would be for all rezzed non-copy objects to show up in the inventory, just marked as “rezzed”, grayed out or something. This way, the user could just “take” the grayed item and get it back. The asset server would delete the reference to the region, so if the object was not lost due to region rollback it would dissapear anyway. So, no copy would still be not copy, but it will not be “lost”.

    I am sure this is what LL does when you enter a support ticket, complaining about your lost 6000L bed. But what about all the minor items you cant remember you ever had?

    And in a lot of situations, it would be very nice to be able to get back your objects without having to go where they are!

    Just a suggestion; thanks for making this the great world it is!

  92. Michael Timeless says:

    #83 Cait

    Thanks and I understand the pain most of the vendors feel. There are just too many ways to allow a backup without violating the integrity of intellectual property to believe that I can’t make a backup. At some point, considering the amount of real cash that is being lost that someone will release a hack to allow the backups, probably by selling it on the Internet. It would be cheaper for vendors and the Lindens in the long run to provide this ability. Having bought Thinc boxes and other storage devices to protect my non copy stuff and having multiple avi’s it is still frustrating to lose about $1200 worth of product over the last four months. It has stopped the purchase of at least one sim from me and I’m sure for other folks as well. I would think that the loss of revenue alone would be enough to come up with something more supportive.

    Post #94 is correct that this would be a good fix.

    Mike

  93. I do understand the very talented content creators wanting to protect their things so they can make money, but……
    if you do a risk analysis you realize your “job” on sl is smoke and mirrors because copybots (and most anyone who can spell Open GL) can steal whatever you create. Most people are honest though. So content creators really should not creat anything of intellectual value on secondlife because secondlife is not a secure platform.
    Now, the content creators are also customers and something we have not addressed is the “fair use” policy which customers have a right to. I do not want to argue if content creators have a right to their property… they do… and/but… when I buy a video tape, I am allowed to make a backup copy of that tape to have to use if and only if the primary copy was lost or destroyed…. I can’t sell the copy or even use it at the same time the original is used but I CAN LEGALLY make a backup. That is part of fair use. Linden Labs in their effort to attract content creators have made the decision that customers CANNOT have FAIR USE of what they buy. They do NOT allow Me to backup what I have legally purchased and own. Since they have denied Me “FAIR USE” on what I have purchase, and have told Me they “back up all My data so I cant loose it” then they are fiscally responsible for My losses because of their inability to provide backup and recovery of My items which they said they would. “FAIR USE” and “intellectual property” do NOT conflict but simply complement each other so both the artist AND the consumer are protected. LL have eliminated the possibility for Me to have “FAIR USE” of the 37,000+ objects I have purchased over the last year. I am certain that there is a class of creators and users who have been victimized by this policy of Linden Labs. There are several ways to allow consumers to have fair use of what they buy in secondlife. Apparently until a lawsuit forces linden labs to respond with a solution equitable to both consumers and creators they will continue to do “other things” which they deem “more important” than paying customers being able to have use of things we buy in second life. Does it truly take a class action lawsuit to get your attention linden labs? I really DO LOVE what secondlife CAN BE but the day to day failures of management priorities makes second life at best second rate…. Can a linden respond to the priority list that they currently have? please? or do we have to take other avenues to get your attention? forcing you to do the right thing should NOT be required… second life COULD BE first rate but currently it is just a toy that doesn’t work well or often or right… The list of things to fix should start with “PROVIDE USERS WITH THE ABILITY TO BACK UP THEIR OWN DATA” — I DON’T CARE HOW — BUT DO NOTHING ELSE UNTIL THAT IS DONE. please…

    and yes #94 does look like a good start and a workable “fix”.

  94. sirhc deSantis says:

    great – “Unable to create requested object. Object is missing from database”. here we go again

  95. coventina dalgleish says:

    It appears someone takes notes on past problems but when creating the upgrades no one takes the time to read them to avoid repeating the mistakes of the past.

    As far as off game back ups there is no way to protect the inventory from hacking, thus duplication of product and many other aberrations.

    Lets face it if all persons were honest we would not have to endure griefers, copy bots, and the many other those who consider ruining the experience a pleasure.

    Linden Lab does care how this game is developed but appears to be totally submerged in the maze of spaghetti code that has accumulated over the past 6 years. Patch after patch compounding many of the issues.

    I hold no expectations of other virtual world games showing on the scene very soon as this is the development platform. Those who have observed the problems here have undoubtedly experimented and found it daunting.

    As for Havok 4 lets hope this solves many of the problems as Havok 1c is ancient history and has a limit on just how much it can handle.

    Voice on the other hand still has a severe load on the system. I do not condemn voice as it has many uses. What I do question is the entry statement that it would be managed by out of game servers. Recently it has become apparent that it is not. When voice was introduced the freeze lags began. When voice is off or down freeze lags appear much less of the time. It also seems that at that time they introduced new garbage clean up routines making it difficult to ascertain which system contributes to this problem.

    Asset server upgrades has been a disaster. What was touted as an improvement has been nothing but a fiasco. There used to be minor problems and some lag but the massive data loss of the last month is Unconceivable. I have no doubt they did not intend this to develop in this manner, but then the road to hell is paved with good intentions.

    It seems to an observer it is time to re-evaluate the main program engine and consider well thought out changes weather this means major changes or not. It is time to give us a stable operating platform on which to build.

  96. U M says:

    @98 THANK YOU for say this
    “Voice on the other hand still has a severe load on the system. I do not condemn voice as it has many uses. What I do question is the entry statement that it would be managed by out of game servers. Recently it has become apparent that it is not. When voice was introduced the freeze lags began. When voice is off or down freeze lags appear much less of the time. It also seems that at that time they introduced new garbage clean up routines making it difficult to ascertain which system contributes to this problem.”

    FINALLY someone else speaking out on this issue…….It does seem to cause alot of problems with some users i noticed these past few months. Whats worse i from what i heard from people now the newest reease has VOICE on when you update the old client to the new client. WHAT THE **********………..More badly done PR about Voice like like owning a 30 kilo TV instead of a Flat screen tv……. Shakeshead

    VERY GOOD POST!!!!!!!!!!!!!!!!

  97. Pinkie Piek says:

    What is going on with the trafficreports???? it has not been refreshed for 3 days now????????
    Can’t even see how my traffic is doing anymore.

    Would be nice if that will be resolved

    thanks,Pinkie

  98. Pingback: Region Data Corruption / Loss Problem « Official Linden Blog

  99. Pingback: MMODump.com » Blog Archive » [Resolved] Region Data Corruption / Loss Problem

  100. Noisey Lane says:

    Everybody, Lindens included, take a big breath and say…….. OMMMMM.
    … or OMG!
    Like, you’re telling me this HOW many weeks later?
    The task of hunting through inventory trying to work out what is still there or NOT is just too daunting.
    It’s ok if technology let’s you down but pah-leeze let us know early.
    I won’t buy, sell, create or add to the beauty of SL during this time.
    Knowing I might not know when this is happening, makes me want to hold back and not add to the wonderment that is SL.
    So I think it’s important that the genuine concerns of the voices shared in this blog are heard.
    (1) What is happening to lost inventory? – one way or the other.
    (2) Ditto re: lost objects in-world.
    (3) $ reimbursment*
    *I didn’t lose any money during the ‘first plague of 08’ so my question is on behalf of previous blog posters only.
    Big ups to those asking about TP’s & Group Chat.
    The basics need to be delt with. Conceptually they work. Reality, virtual or otherwise, only some of the time.
    I have not been able to initiate a successful Group Chat since… forever, TP is 50/50.
    Users require blow by blow updates. We can tell when things are not happening correctly. Hours later, we get the blog post.
    The basic principal of SL is (to paraphrase): “we’re all in this together” so bring us together when it happens.
    This IMO is a biggie. Show stopper as they say.
    I’m (fairly) confident you’re working on better ways to back up personal data (read investments) and while I love windlight and dazzle (converge these two soon), let’s not lose focus on the core issues.
    No matter how cool it looks, if it don’t do stuff, it’s history.

  101. Pingback: [Resolved] Region Data Corruption / Loss Problem : DiarioSL

  102. shesek says:

    shesek blah blah shesek

Comments are closed.