Seti is down again

[The Grinch]

Homepage are still online, right know!
Database-Server offline ...

[msattler]

I am hoping that, given that magnitude of the downtime and the fact that things never had time to recover after last week's outage, the boyz might get things sorted before coming back online and forgo the 3 day outage this week.  Hopefully that would give things time to recover whilst they are in the lab and able to monitor the servers should problems arise.

This, of course, is assuming that no critical damage has been sustained to the servers from the AC failure and subsequent overheating.  But there is also the possibility of hidden damage or partial damage that can rear it's head after such an event.

Wishing the crew the best of luck in sorting things this week.

Meow meow.

[JohnDK]

Talking about damage,  I assume they have backups but could it ever be so bad that stuff like user stats and credits is lost?

[efmer (fred)]

Talking about damage,  I assume they have backups but could it ever be so bad that stuff like user stats and credits is lost?

Depends where they keep the data. If it's all in the same location/building definitely.

[perryjay]

Monday Update: The air conditioning in our server closet failed on Saturday, causing several of our important systems to overheat. Most servers and services were shut down and will remain down as we continue to wait for the air conditioning to be fixed, which will be Tuesday at the earliest.

They changed the message on the front page. It's gonna be close but I might just make it if they get it fixed early enough.

[Cosmic_Ocean]

Homepage doesn't respond, and cricket is flatlined.  5:15pm PDT.. if they're not still in the lab, I would venture to guess at this point that we roll right into the scheduled downtime for the week.  Just a theory.

[perryjay]

Glad I saw and copied the new message before it went away again. Hard to tell what they will do about the regular outage after this one though.

[glennaxl]

With the 3-day outage, here's what I don't understand - working/developing on a production environment? That's bad practice. Isn't seti beta suppose to handle that; I guess its just for frontend(science app). There must be something wrong with their methodology.

[arkayn]

With the 3-day outage, here's what I don't understand - working/developing on a production environment? That's bad practice. Isn't seti beta suppose to handle that; I guess its just for frontend(science app). There must be something wrong with their methodology.

Beta runs off the same servers as SETI, they have the downtime so they can analyze the data that has come in in the last 10 years. They tried it with all the servers up and it just crashed the systems from the strain.

[IrishFBall32]

If I recall, the main webserver is actually in the same closet as the other servers, but you're correct that it is fed from the campus ISP instead of the separate high bandwidth one for S@H

[msattler]

Monday Update: The air conditioning in our server closet failed on Saturday, causing several of our important systems to overheat. Most servers and services were shut down and will remain down as we continue to wait for the air conditioning to be fixed, which will be Tuesday at the earliest.

They changed the message on the front page. It's gonna be close but I might just make it if they get it fixed early enough.

Thanks for saving that notice while I was gone to work.

Bummer........wonder if they couldn't get anybody in to look at it or if they are waiting on parts......
Sigh.......maybe tomorrow.

[perryjay]

Well, all my work completed overnight. Got 4 ready to report that uploaded just before we went down the rest waiting to upload. Should be interesting to see what they are going to do today.   

[Claggy]

I ran out of ATI Astropulse tasks a couple of hours ago, now down to my last 6 Cuda tasks on Seti Main, still got 20 Normal AR Cuda tasks on Beta, and 20 Cuda VLARS, they'll keep me going for almost 2 days, 

Claggy

Edit: The website is up again, and according to the Status page, the download servers are online, and cricket shows some uploads and downloads too, (but not for very long)

[hiamps]

Sure glad I got my 10 day cache, should still have a couple days left...Hope they didn't lose much.

[msattler]

Sure glad I got my 10 day cache, should still have a couple days left...Hope they didn't lose much.

Still got some left here as well......
The home page is back up, with the usual Tuesday-Thursday outage message.
Wondering now if that message is automated, or if they really intend to be down until Friday.

Meow, Meow, Meeeeeouch.