SETI MB CUDA for Linux

[pp]

Are these machines dedicated crunchers or you use them also as your desktops?

For the moment it's a dedicated cruncher but it's supposed to be my regular desktop computer. I have still to fit some more disks, serial I/O cards and deal with the air flow inside the box. It just breaks my heart to stop it from crunching to do that
/PP

[sunu]

It just breaks my heart to stop it from crunching to do that

Yes, I have the same feelings.

[riofl]

whoah. seems like things are moving fast that 8 gpu machine must be able to cook a turkey placed behind it or at least heat a small home! those cards are entirely too close. each one will cascade more heat into the next one until the end one must run well over 130c! that thing would have to run in an ambient temp environment of near 0c to keep all those puppies cool. with all this changing technology and the imminent gt300 release my project may well get delayed into the 2nd quarter of the year to allow some re-design.

btw for those talking about boinc versions, i have been running 6.9.0 compiled from source and it has worked fine for multiple gpus... the scheduler is a bit odd in that it will refuse to pickup up work units until it is down to less than a  few hundred then it goes into panic mode and tries to continuously request them until it gets its quantity back up, rather than begin asking for more when the queue reaches around 50% which seems the sensible place to refill.

[sunu]

whoah. seems like things are moving fast that 8 gpu machine must be able to cook a turkey placed behind it or at least heat a small home! those cards are entirely too close. each one will cascade more heat into the next one until the end one must run well over 130c! that thing would have to run in an ambient temp environment of near 0c to keep all those puppies cool.

That's why I'm saying that for multi multi-gpu installations it's better a water-cooling setup.

btw for those talking about boinc versions, i have been running 6.9.0 compiled from source and it has worked fine for multiple gpus... the scheduler is a bit odd in that it will refuse to pickup up work units until it is down to less than a  few hundred then it goes into panic mode and tries to continuously request them until it gets its quantity back up, rather than begin asking for more when the queue reaches around 50% which seems the sensible place to refill.

As I said to Tye, you could try 6.6.40 or 6.10.11, they should have proper multi-gpu support.

[b0b3r]

btw for those talking about boinc versions, i have been running 6.9.0 compiled from source and it has worked fine for multiple gpus... the scheduler is a bit odd in that it will refuse to pickup up work units until it is down to less than a  few hundred then it goes into panic mode and tries to continuously request them until it gets its quantity back up, rather than begin asking for more when the queue reaches around 50% which seems the sensible place to refill.

You can avoid this by setting proper flops value in app_info.xml. This vale have impact on "result duration correction factor" which is used by scheduler process to guide your client in which manner it should ask for results.

[pp]

I ran 6.9.0 too for a while and I think it suffered from the on_frac bug. The branch was never tagged and was only available on trunk for a while. 6.10.x has fixed that bug but probably introduced a few more...

[riofl]

ill try upgrading boinc this week.. doesn't seem to be any rush with the server probs seti is having. most of my servers have run out of work anyway

water cooling makes some sense but do they make plates that cover all the chips that get contact from the stock cooler? jeeze on my tesla and gtx285  it is almost every chip that has a thermal pad attached with only the gpu actually getting any thermal grease. with the varying depths of the different chips the thermal plate would have to be custom made for each card like the stock plate is. do they actually make custom ones for each card? i have had thoughts of water cooling for some time especialy since my case is made for it but the seeming complication of cooling the gpus properly has held me back.

[Urs Echternacht]

GPUs under water.

@riofl: is that ^ e.g. what you are looking for ?

[riofl]

sure is! thank you!

now all i have to do is figure out what cooling systems are decent / good / great and compare prices and performance. being in florida i have a lot of higher ambient temps to contend with so the radiator will have to be very efficient. my only true concern is ultra high reliability. the systems run 24/7 and my workstation which is the biggest heat generator is an absolute must to remain up at all costs (every minute it is down i cannot work).

should be an interesting adventure .. never delved into water cooling before, always used good fans and matched air flow for best heat transfers and that always worked fine for me... this will be different.

(hmm maybe a freon based system is worth looking into? )

[riofl]

well, i know one thing i will not be getting anytime soon. pcie expansion cabinet. those puppies are massively expensive! i could buy a 295 and a 285 for the price of the cheapest one i found! cant see it.. no reason for that kind of price gouging except for the fact its a 'niche' and they can do it. damn greed. doubt there is more than $50 actual  mfg cost in there.

was thinking of getting a 4 device expansion cabinet to deal with heat and power separately and save the added expenses of big mobos.

if i could find the individual interfaces i could convert an old minitower.

[sunu]

You could also consider ready for water cooling cards like this or this or, for even easier water-cooling, plug 'n play, self-contained water-cooled cards like these.

[riofl]

You could also consider ready for water cooling cards like this or this or, for even easier water-cooling, plug 'n play, self-contained water-cooled cards like these.

hmmm adds $200+ to the price of every card plus in the case of the nice idea self contained unit, finding spots to put each of the 120mm fan/cooler units..  (i really don't like having things hanging out of the case all over the place)

wondering how air would do.. my 285 runs less than 65c full load so with a bit of tinkering i would think i could keep a 295 below 75c total  with 100% fan plus plenty of ambient air flowing all around it which is still well within factory specs. dunno. i have no clue what a properly running 295 runs at for temps. i have only ever seen 'problem' temps which are off the charts in msg boards.

i can see the need for water cooling when packing those babies together but i think i can get away without water on the standalone replacement for my tesla. plus i read water cooling results all over the place from 30c cooler to less than 5c difference between water and air. i definitely dont like the idea of spending 500 to 700 extra for proper water cooling for my system only to find it isnt much better than i had with air (especially with next year's project having to spend 200-300 extra per card.. would be cheaper to build 2 smaller air units to get the same gross results).

if it wasnt for the condensation problem i would just spend a few hundred on a top or front mount air conditioner or a window unit with a duct feed dedicated to the case.. nothing is gonna overheat when the ambient internal case temp is 1 to 3c

[riofl]

heh... might as well go for broke and get this case to put it all in

absolutely the greatest 'style' ive ever seen!


http://www.ttlevel10.com/

[sunu]

i can see the need for water cooling when packing those babies together but i think i can get away without water on the standalone replacement for my tesla.

Of course, I'm talking about watercooling only for your multi multi-gpu project. For a single card you can do without it.

heh... might as well go for broke and get this case to put it all in

absolutely the greatest 'style' ive ever seen!


http://www.ttlevel10.com/

I don't think cooling-wise it'll have many big advantages over "standard" cases.

[riofl]

i can see the need for water cooling when packing those babies together but i think i can get away without water on the standalone replacement for my tesla.

Of course, I'm talking about watercooling only for your multi multi-gpu project. For a single card you can do without it.

heh... might as well go for broke and get this case to put it all in

absolutely the greatest 'style' ive ever seen!


http://www.ttlevel10.com/

I don't think cooling-wise it'll have many big advantages over "standard" cases.

the only advantage i see is that it separates the heat generators into separate individually cooled compartments which reduces the overall heating load to have to deal with.

i can make anything work cool enough i think, i just LOVE how it looks