Seti@Home optimized science apps and information
Optimized Seti@Home apps => Discussion Forum => Topic started by: Urs Echternacht on 08 Oct 2008, 07:09:09 am
-
I think atm there is an unannounced outage at berkeley happening. Can't reach seti@home, beta, boinc, boinc alpha or the ssl pages. Hope it is nothing serious.
-
Can reach message boards here Urs , is that not accessible to you?
-
Confirm access from here :)
And server status of SETI project just fine.
-
Still can't reach any of SSL's internet addresses. Will try pinging and tracing.
Update: Ping works, but trace does not.
Update: The closest i can reach is http://astro.berkeley.edu/facilities/labs.html
-
Message boards and download server OK for access from the UK, but the upload server has been listed as 'disabled' for about 4 hours. That means someone switched it off manually somewhere between 1 am and 2 am local time: presumably in response to some detected problem they felt they could wait until daylight to fix properly.
-
From UTC 02:00 to 15:00 no upload and reporting from Germany.
SETI@home homepage not available.
Then everything was fine.. I uploaded and reported everything.
[EDIT:Tested not the homepage]
Now [UTC 19:12] the SETI@home homepage not available..
Berkeley don't want no longer german members? ;)
Others have problems also?
-
Same for me. Still can't reach any webpages at the SSL in Berkeley. Uploading and reporting is working now. Could somebody who can reach the webpages please tell them that they cut off their second biggest resource of volunteers.
Thanks Sutaru, so i know now that i'm not alone with that problem
-
how far does the traceroute get Urs?
-
Sorry, Jason, my ISP does not allow tracerouting anymore (just found out today). But pinging berkeley web servers does work.
-
Huh, that's a bit rough!, LoL, you can have a connection but no diagnostics if you get a problem ...
-
traceroute for setiathome.berkeley.edu:
Host-Name IP Zeit 1 Zeit 2
....
2 217.0.118.202 217.0.118.202 72.178ms
3 87.186.251.18 87.186.251.18 72.185ms
4 194.25.6.41 194.25.6.41 166.525ms
5 62.156.128.110 62.156.128.110 167.349ms
6 sl-st20-ash-5-0-0.sprintlink.net 144.232.3.76 166.255ms
7 dcp-brdr-02.inet.qwest.net 63.146.26.81 170.635ms
8 dcx-core-02.inet.qwest.net 205.171.251.37 167.238ms
9 los-core-01.inet.qwest.net 67.14.22.10 234.200ms
10 no reply *
11 63.147.28.182 63.147.28.182 234.302ms
12 dc-svl-isp--lax-isp-t1.cenic.net 137.164.40.220 242.714ms
13 inet-ucb--svl-isp.cenic.net 137.164.24.106 251.645ms
14 g3-19.inr-201-eva.Berkeley.EDU 128.32.0.58 245.580ms
15 g6-1.inr-230-spr.Berkeley.EDU 128.32.255.110 248.273ms <-- some router at berkeley
16 no reply *
....
31 no reply *
-
On the internet their seem to be no other "blocked" (if it is a blockade) webpages, europe, korea, japan, hongkong, taiwan, australia, even other pages in california, especially at the berkeley campus, i can reach. It is only the SSL that is gone.
-
Mind if I pass on that traceroute info to NC forum there speedimic?
-
no problem, Jason.
-
Is there somewhere in the inet a homepage from this I can test the access then to Berkeley?
That I can see where the problem is?
Like speedimic done..
-
It is only the SSL that is gone.
the Cricket grapher (fragment1.berkeley.edu/....) is also gone.
EDIT: at least it's alive:
traceroute to fragment1.berkeley.edu (128.32.206.221), 30 hops max, 40 byte packets
2 217.0.118.202 (217.0.118.202) 44.707 ms 47.445 ms 50.305 ms
3 87.186.251.22 (87.186.251.22) 53.272 ms 56.127 ms 59.230 ms
4 194.25.6.41 (194.25.6.41) 156.318 ms 159.195 ms 162.087 ms
5 62.156.128.110 (62.156.128.110) 165.796 ms 168.179 ms 170.673 ms
6 sl-st20-ash-5-0-0.sprintlink.net (144.232.3.76) 175.634 ms 166.042 ms 168.218 ms
7 dcp-brdr-02.inet.qwest.net (63.146.26.81) 168.752 ms 139.538 ms 139.167 ms
8 dcx-core-02.inet.qwest.net (205.171.251.37) 141.664 ms 144.960 ms 147.722 ms
9 los-core-02.inet.qwest.net (67.14.22.18) 225.189 ms 227.718 ms 230.888 ms
10 los-edge-01.inet.qwest.net (205.171.32.38) 224.822 ms 228.018 ms 230.851 ms
11 63.147.28.182 (63.147.28.182) 233.712 ms 236.081 ms 239.527 ms
12 dc-svl-isp--lax-isp-t2.cenic.net (137.164.40.222) 264.997 ms 265.036 ms 265.353 ms
13 inet-ucb--svl-isp.cenic.net (137.164.24.106) 223.482 ms 222.944 ms 224.541 ms
14 g3-19.inr-201-eva.Berkeley.EDU (128.32.0.58) 220.457 ms 218.754 ms 219.650 ms
15 t1-4.inr-210-srb.Berkeley.EDU (128.32.255.149) 220.355 ms 222.962 ms 226.260 ms
16 t1-2.inr-211-srb.Berkeley.EDU (128.32.255.45) 225.440 ms 228.035 ms 230.970 ms
17 fragment1.Berkeley.EDU (128.32.206.221) 237.272 ms 240.095 ms 242.074 ms
/EDIT
-
no problem, Jason.
Done. [Edit: Looks like something else is going on with server issues, so I guess they'll see my post when they finish putting out those fires ... and lunch ...]
-
just found this: Latimer Hall Network Outage (http://ucbsystems.org/)
well, it's another subnet, but you never know...
-
Is there somewhere in the inet a homepage from this I can test the access then to Berkeley?
That I can see where the problem is?
Like speedimic done..
Try http://www.traceroute.org/
-
Can you get there if you go here?:
http://128.32.18.150
-
Can you get there if you go here?:
http://128.32.18.150
Nothing happens when i click that link, the browser waits forever for an answer.
Found some working service via traceroute.org (my ISP is not listed there, so this is from somewhere else in germany) :
trace from latency.hosteurope.de (AS20773) to 128.32.18.150
begun at Wed Oct 8 22:41:55 2008 CEST (Wed Oct 8 20:41:55 2008 UTC)
1 80.237.129.193 (80.237.129.193) 0.595 ms 0.235 ms 0.231 ms
2 ffm-b1-link.telia.net (213.248.102.157) 3.031 ms 2.968 ms 2.814 ms
3 ffm-bb2-link.telia.net (80.91.254.106) 2.907 ms 2.875 ms 2.867 ms
4 prs-bb2-link.telia.net (80.91.248.65) 12.737 ms prs-bb2-link.telia.net (80.91.249.47) 12.676 ms prs-bb2-link.telia.net (80.91.248.61) 12.749 ms
5 nyk-bb2-link.telia.net (80.91.251.100) 84.518 ms 84.599 ms 84.562 ms
6 nyk-b5-link.telia.net (80.91.248.154) 97.036 ms 84.560 ms 84.571 ms
7 xe-10-1-0.edge1.NewYork1.Level3.net (4.68.110.81) 83.474 ms 82.834 ms 83.147 ms
8 vlan99.csw4.NewYork1.Level3.net (4.68.16.254) 95.562 ms 83.943 ms vlan89.csw3.NewYork1.Level3.net (4.68.16.190) 83.421 ms
9 ae-84-84.ebr4.NewYork1.Level3.net (4.69.134.121) 90.731 ms 83.573 ms ae-74-74.ebr4.NewYork1.Level3.net (4.69.134.117) 89.582 ms
10 ae-2.ebr4.SanJose1.Level3.net (4.69.135.185) 158.994 ms 156.389 ms 162.231 ms
11 ae-84-84.csw3.SanJose1.Level3.net (4.69.134.250) 160.348 ms 154.779 ms 162.335 ms
12 ae-31-89.car1.SanJose1.Level3.net (4.68.18.131) 152.384 ms 152.602 ms 152.611 ms
13 CENIC.car1.SanJose1.Level3.net (4.79.44.6) 152.502 ms 152.978 ms 152.577 ms
14 inet-ucb--svl-isp.cenic.net (137.164.24.106) 154.870 ms 154.554 ms 154.228 ms
15 g3-19.inr-201-eva.Berkeley.EDU (128.32.0.58) 268.663 ms 253.098 ms 203.452 ms
16 g6-1.inr-230-spr.Berkeley.EDU (128.32.255.110) 153.803 ms 154.431 ms 154.113 ms
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
-
Yeah, so definitely router / link problem rather than funky DNS ... Everything's gone really quiet on the forum, so maybe you're not the only ones effected.
-
Can you get there if you go here?:
http://128.32.18.150
DNS is not the problem.
Tracerouting doesn't give us a clue either. It get's stuck at inr-230 from wherever I try (even the Cogent-router at Los Angeles), but from the US there is no problem to reach seti. inr-230 simply blocks traces.
SOLUTION:
use a proxy for the time beeing.
http://www.anonymitychecker.com/page1.html
tested some... this one 64.66.192.61 works.
-
LoL, maybe you can drop into my lonely thread then ;D
-
Have tried, but the connection via proxy is simply too slow for my taste (and in my neck there sits a little security advisory that says: "You better wait that out!").
-
Have tried, but the connection via proxy is simply too slow for my taste (and in my neck there sits a little security advisory that says: "You better wait that out!").
yep, reminds me of surfing via notebook + mobile phone (not UMTS) somewhere in the Schwarzwald... ;D
-
Seems to be ok now.
-
Ah good, the magic internet elves do it again.
-
Do you mean the proxy connection or the SSL web-pages ? SSL is still unreachable from here.
And the traceroute from hosteurope.de still stops at the same ip.
add: give me a second, will check a win pc.
-
Ah good, the magic internet elves do it again.
Must be Microsoft elves 5.0...
It's ok on my Windows notebook (no proxy, any browser), but still stuck on my Linux-box (no proxy, Firefox).
Now THAT is really strange!
-
Sorry, no web-pages are loaded over here, not on two different linux versions or two different windows variants. Still have to wait.
-
as I said; REALLY strange.
I'll go to bed now and dream the solution...
gute nacht.
-
SSL ?? :-\
I can't reach:
http://boinc.berkeley.edu
http://setiathome.berkeley.edu
:'(
The problem is posted at Berkeley?
[technical thread?]
Or maybe PM to the Berkeley-crew?
-
I'm in office now and there is no simple way to reach SSL (Space Science Laboratory).
I'm behind a Linux proxy here, so that might be the problem... I'll go to another network segment to get around the proxy later and look if windows does the job...
-
I can't reach Berkeley..
WinXP and IE.
I feel alone now.. :'(
;)
There are some news because of the prob?
Berkeley-crew know because of the prob and/or some post/s in the forum?
-
I'm situated in the UK. I can contact Berkeley (both SETI and BOINC) with no problems: the SETI server page is showing everything running except the usual couple of splitters.
No, there's no problem with Berkeley. There clearly is some problem with the internet connection between Germany and Berkeley, but I doubt it is under the control of the SETI staff.
-
Yeah it's been fine here in Australia too. It's really weird if these people can get up/downloads & pings through but no web pages ...
-
A possible clue: I have one machine (only) which is failing to download new SETI work from Berkeley. Uploads and scheduler contacts (and general browser work) are fine. The problem is only with Vista/v5.10.13: XP/v5.10.13, and XP/v6.2.18, are both fine.
I turned on debug options in cc_config.xml, and received:
09/10/2008 10:59:34||[file_xfer_debug] PERS_FILE_XFER::start_xfer(): URL: http://boinc2.ssl.berkeley.edu/sah/download_fanout/2b0/19au08aa.25729.25021.10.8.171
09/10/2008 10:59:34||[http_debug] [ID#2] info: About to connect() to boinc2.ssl.berkeley.edu port 80 (#0)
09/10/2008 10:59:34||[http_debug] [ID#2] info: Trying 208.68.240.13...
09/10/2008 10:59:34||[http_debug] [ID#2] info: Connected to boinc2.ssl.berkeley.edu (208.68.240.13) port 80 (#0)
09/10/2008 10:59:34||[http_debug] [ID#2] Sent header to server: GET /sah/download_fanout/2b0/19au08aa.25729.25021.10.8.171 HTTP/1.1
User-Agent: BOINC client (windows_intelx86 5.10.13)
Host: boinc2.ssl.berkeley.edu
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/x-www-form-urlencoded
09/10/2008 10:59:34||[http_debug] [ID#2] Received header from server: HTTP/1.0 503 Service Unavailable
09/10/2008 10:59:34||[http_debug] [ID#2] Received header from server: Content-Type: text/html
09/10/2008 10:59:34||[http_debug] [ID#2] Received header from server: Content-Length: 53
09/10/2008 10:59:34||[http_xfer_debug] HTTP: wrote 53 bytes
09/10/2008 10:59:34||[http_debug] [ID#2] info: Expire cleared
09/10/2008 10:59:34||[http_debug] [ID#2] info: Closing connection #0
09/10/2008 10:59:35||[file_xfer_debug] FILE_XFER_SET::poll(): http op done; retval -184
09/10/2008 10:59:35||[file_xfer_debug] PERS_FILE_XFER::poll(): file transfer status -184
And then, while I was typing that.....
09/10/2008 11:07:49||[file_xfer_debug] PERS_FILE_XFER::start_xfer(): URL: http://boinc2.ssl.berkeley.edu/sah/download_fanout/1fa/19au08aa.25729.25021.10.8.189
09/10/2008 11:07:49||[http_debug] [ID#4] info: About to connect() to boinc2.ssl.berkeley.edu port 80 (#0)
09/10/2008 11:07:49||[http_debug] [ID#4] info: Trying 208.68.240.18...
09/10/2008 11:07:49||[http_debug] [ID#4] info: Connected to boinc2.ssl.berkeley.edu (208.68.240.18) port 80 (#0)
09/10/2008 11:07:49||[http_debug] [ID#4] Sent header to server: GET /sah/download_fanout/1fa/19au08aa.25729.25021.10.8.189 HTTP/1.1
User-Agent: BOINC client (windows_intelx86 5.10.13)
Host: boinc2.ssl.berkeley.edu
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/x-www-form-urlencoded
09/10/2008 11:07:50||[http_debug] [ID#4] Received header from server: HTTP/1.1 200 OK
09/10/2008 11:07:50||[http_debug] [ID#4] Received header from server: Date: Thu, 09 Oct 2008 10:07:48 GMT
09/10/2008 11:07:50||[http_debug] [ID#4] Received header from server: Server: Apache/2.2.9 (Fedora)
09/10/2008 11:07:50||[http_debug] [ID#4] Received header from server: Last-Modified: Thu, 09 Oct 2008 06:01:09 GMT
09/10/2008 11:07:50||[http_debug] [ID#4] Received header from server: ETag: "11a9b789-5ba99-458cbc2581340"
09/10/2008 11:07:50||[http_debug] [ID#4] Received header from server: Accept-Ranges: bytes
09/10/2008 11:07:50||[http_debug] [ID#4] Received header from server: Content-Length: 375449
09/10/2008 11:07:50||[http_debug] [ID#4] Received header from server: Connection: close
09/10/2008 11:07:50||[http_debug] [ID#4] Received header from server: Content-Type: text/plain; charset=UTF-8
and a couple of them downloaded OK.
Do German ISPs use eclusively HTTP/1.0? Is it under anyone's direct control?
Edit - note the two different IP addresses:
208.68.240.13 is trying - and failing - to use HTTP/1.0 when the GET specifies HTTP/1.1
208.68.240.18 uses HTTP/1.1 as directed
Further edit:
I put 208.68.240.18 boinc2.ssl.berkeley.edu into a HOSTS file, flushed DNS, and restarted BOINC. All downloads completed at the first attempt.
-
Hmmm, I think Urs mentioned that a web page request sits doing nothing rather than getting back a 503 service unavailable. If they were getting that I'd suspect something 'closed for repair or maintenance'. Don't know about locale specific variants, but HTTP 1.0 & 1.1 are both application layer so only the client and server endpoints should be able to tell the difference...
My next random guess is someone (Matt?) fiddling with QoS prioritisation somewhere, so some overloaded piece of equipment on a specific path might simply be dropping the requests... Perhaps the offsite storage or similar could be via the same link...
If it's something weird like that then it should come good as soon as whatever congestion clears, but I woudln't count on it being anything like that situation. Just a guess, which doesn't help matters.
-
I just talked to a fellow admin located in Nürnberg.
He can get to setiathome.berkeley.edu without any problem.
The difference we figured out is that he is going via N-IX and I go via DE-CIX.
So the problem most likely is ISP dependent.
Edit:
just tried out what Richard tested.
I opened the two round robin IPs of boinc2.ssl.berkeley.edu with Firefox:
http://208.68.240.18/sah/download_fanout/2b0/ --> gives me the directory listing
http://208.68.240.13/sah/download_fanout/2b0/ --> The service is not available. Please try again later.
Further Edit:
It does not matter if I use HTTP 1.0 or 1.1.
-
It'll be interesting to find out how the blockage is so specific. (if you ever find out that is)
-
I've sent the HTTP debug output in a PM to Matt and Eric. Hopefully they can figure it out when they get into the lab in 4 or 5 hours time.
-
ok, did some more testing...
Richard's up/download problem has got nothing to do with our http problem.
boinc2.ssl.berkeley.edu is in a completely different network (208.68.something... (Hurricane Electric ??)) and works
setiathome.berkeley.edu is in 128.32.something... (Campus network) and doesn't work
fragment1.berkeley.edu (the Cricket grapher) is in 128.32.something... (Campus network) and doesn't work
that's why up/downloads work, but the website doesn't.
Taking that "different ISP- Different status" thing into account, I'd say it's some kind of peering problem here in Germany (especially Telekom / DE-CIX).
-
If i use 208.68.240.18 port80 as a http proxy in Firefox on linux i can access setiathome.berkeley.edu. Do i use direct internet connection as usual no contact is made.
And instead of boinc.berkeley.edu i got the attached testpage with that as a proxy.
[attachment deleted by admin]
-
If i use 208.68.240.18 port80 as a http proxy in Firefox on linux i can access setiathome.berkeley.edu. Do i use direct internet connection as usual no contact is made.
Already thought about that when I relized that boinc2.ssl.berkeley.edu is a proxy server but first, I'm alredy behind a proxy/firewall, and 2nd don't know if it's ok for berkeley if we use their proxy...
-
It's now also a topic in our SETI.Germany-forum (http://board.setigermany.de/showthread.php?p=71799#post71799)..
It's look like only some internet provider have problems..
I'm at the german Telekom doughter Congstar.
-
so Telekom for Sutaru Tsureku and myself.
Urs, what's your provider?
-
A German user with t-online has reported the same symptoms (using the Einstein message boards, which are obviously still OK for him).
-
I read the different threads and just called t-online, i tell them there must be a problem on there site because the website is reachable over other providers. They tried it and also can not reach seti. So they would forward it to the right department and check out what is wrong.
I think we have to wait until they find something.
I will post any news.
-
http://setiathome.berkeley.edu/forum_thread.php?id=49711&nowrap=true#816312
-
What can i read there ? For me seti is dead in the water, can't reach any website.
-
At the moment i can connect to setisites, have no answer from t-online at the moment. Is it working on your site also?
-
yep, green lights everywhere.
;D ;D ;D
still like to know what the problem was....
-
just called t-online again and they said, that they had a problem with a dns server and send every request to seit to nowhere. I hope it is working for all crunchers right now.
-
Maybe ET doesn't want to be found :o !
-
http://setiathome.berkeley.edu/forum_thread.php?id=49711&nowrap=true#816353
-
Maybe ET doesn't want to be found :o !
That's it! Some bad alien fired the quantum-flux-whatever-gun on some routers. :o
-
@ crazyrabbit1
Thanks for your call! :)
I'm back in the SETI@home-forum.. :D
But it's little strange.. it was only the SETI@home-homepage? :-\
Or they cutted the cable to USA? :o ;)
-
My final guess on this issue is that it was a coincidence of two bad things happening at the same time at two different ends of the world:
1. A server at berkeley had a blip, lost its NFS mount and that corrupted the httpd service by accident (see one of Matt's post in seti@home's forum).
2. A DNS server of german Telekom (which is not my ISP but my ISP uses their infrastructure) tried to sync its services (the routes to seti) right after that issue 1. and was holding a nonsense route to seti from that moment on.
Please take that as a layman's explanation combined out of all the postings that were written over the last two days about this issue.
-
My final guess:
http://setiathome.berkeley.edu/forum_thread.php?id=49711&nowrap=true#816415
-
Hello, :(
since yesterday evening I can't log-in in my SETI@home account at Berkeley.
I get the message:
"HTTP 500 Interner Serverfehler"
I can see the Berkeley-homepage and read the forum.. but can't log-in in my account.
I'm the only one? :-\
-
Hello, :(
since yesterday evening I can't log-in in my SETI@home account at Berkeley.
I get the message:
"HTTP 500 Interner Serverfehler"
I can see the Berkeley-homepage and read the forum.. but can't log-in in my account.
I'm the only one? :-\
No, you're not alone. Someone did report this in seti@home Technical News Forum. So, i think THEY know. ;)
-
I have had no problem, but I never log out anyway.
-
I can't contact the SETI@home-homepage.. :(
http://setiathome.berkeley.edu
The Boinc-side is available..
http://boinc.berkeley.edu
From Germany..
How it's looking with you?
-
Down at the moment, from here in Oz
-
Very, Very Slow here in the U.K, I can manage to open the forums and status page, but it takes several minutes to happen.
Claggy
-
berkeley.edu/newcricket/grapher.. (http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=%2Frouter-interfaces%2Finr-250%2Fgigabitethernet2_3;ranges=d%3Aw;view=UcastPackets)
It's looking like they have probs.. unplanned outage..?
-
The status page is showing all green, except for a couple of MB splitters (that's normal),
so any fixing hasn't started yet.
Claggy
-
It is very slow here in Arizona as well, any fixes will most likely wait until tomorrow.
-
My uploads go through ok but I can't contact the scheduler.
-
The database has been shut down, at least according the front page.
The upload and download servers are still running but the scheduler is disabled.
-
And again.. no homepage available..
http://setiathome.berkeley.edu
(http://fragment1.berkeley.edu/newcricket/mini-graph.cgi?type=png;target=%2Frouter-interfaces%2Finr-250%2Fgigabitethernet2_3;inst=0;dslist=ifInUcastPackets%2CifOutUcastPackets;range=151200;rand=423)
Too much traffic @ Berkeley.. :(
How it's looking @ you?
-
It was probably a temporary thing. It's all right now.
Edit: But I have problems uploading.
-
Yes.. now I can reach again the Berkeley-homepage.. ;D
And I got some new WUs.. *thumb up*
..but will see how long it will go well.. ;)