Modified SETI MB CUDA + opt AP package for full GPU utilization

[koschi]

Just a FYI. Whatever I tried, flops, avg_ncpus, maxncpus in app_info.xml in order to force 3+1 in a quad didn't have any effect. The only thing that seems to work is

<avg_ncpus>1.3400</avg_ncpus>
<max_ncpus>1.3400</max_ncpus>

in AP section of app_info.xml. But that possibly means that If there is no MB work, only 3 cores will be used. Whatever I did in MB section seemed irrelevant.

Are you running SETI only?
I'm running some more projects and would like to run a 3+1 configuration, but whatever I set avg_ncpus/max_ncpus to (tried 0.04, 1, 1.5, 2), its doesn't work. When set to 2, no CUDA process is started at all. With all values >= 1 the given nice value is 19, not 10...

Is there detailed documentation of the app_info stuff somewhere that explains all tags?

I guess I have to use a cc_config.xml with ncpus 3 if there is no solution via app_info, right?

[sunu]

I'm running astropulse+multibeam. Because of linux client's full core use we have to use 3+1. So I put those values in the AP section of app_info.xml. That did the trick. Whatever was in the MB section didn't seem to matter. Playing with cc_config.xml didn't have the desired effect. Please see above another post from me (reply #242) about cc_config.xml.

[koschi]

This machine where I'm trying to set up MB CUDA and AP is running POEM, LHC and GPUGRID as well.
With ncpus=3 in cc_config I get the desired result (as long as I have MB WUs), but thats far away from being perfect, if there is no GPUGRID task (which happens from time to time) and it doesn't get work from SETI, one core is idle...

I've read your post regarding the AP settings (found it via google, hehe), but I'm a little confused, I doubt that it works when more projects are attached... Right now I don't get any AP WUs, only had some MB units earlier today, so I cant say if it runs MB and AP in parallel or not yet...

edit:

My goal is to run CPU units (POEM, SETI AP, LHC) and GPU units (SETI & GPUGRID) at the same time with the best utilization (better more then less, the scheduler can stand it...)

[sunu]

As it is right now I think the problem lies with boinc client which cannot handle very well cpu- and gpu-bound tasks simultaneously.

Your situation is more complicated with all those projects that you want to run. I don't know whether the other projects use an app_info.xml file too, but if they do, you can put these two rows
<avg_ncpus>1.3400</avg_ncpus>
<max_ncpus>1.3400</max_ncpus>
on their respective CPU apps sections.

That way you will always run 3+1 except when there is no GPU task where you'll have 1 core idle. Then you can put ncpus 5 to cc_config.xml for some time until you get some GPU tasks to crunch again.

Regarding more is better than 1 core empty I think it is wrong. These optimized apps have special optimization for cache usage and when you have two apps, especially from different projects, sharing a core you will have degraded performance overall with all these cache misses or whatever. On the other hand you will always have some seti cuda or GPUGRID tasks to crunch more or less. On those rare circumstances that you will not, you can use the cc_config.xml trick.

[koschi]

I changed the AP ncpus from 0.0400 to 1.34 in SETIs app_info.xml...
But as long as there is no AP WU running this is not effective I think. My app_info should be OK, but I didn't get a single new WU since this afternoon. Had some MBs this morning (with the new attached app_info.xml) but since then nothing...

So unfortunately I don't have a single SETI unit right now, all clients since 6.3.21 kind of sucks, it doesn't get better :-( In almost every client there is some work fetch problem...
Regarding sharing cores, the problem is we don't know what the CUDA linux app is actually doing while consuming one full core for nothing... When running GPUGRID some months ago, when the app was still consuming half a core for polling the GPU app, there was no performance problem, when running 4+1 rather then 3+1 under Linux... Right now it runs flawlessly in 2+1 or 4+1 mode on my hosts without any difference to 3+1 or 1+1.

edit:

now I got it, sorry, reading an understandig...
My other projects don't have app_infos, so there is nothing to edit :-(




[attachment deleted by admin]

[sunu]

Right now there are zero AP workunits. We wait for new data from Arecibo.

Since the other projects don't have an app_info.xml, you'll have to play with cc_config.xml.

For SETI you can play with app_info.xml. The file you've attached will run 4+1 AP+MB without any modification to cc_config.xml.

[Raistmer]

Regarding more is better than 1 core empty I think it is wrong. These optimized apps have special optimization for cache usage and when you have two apps, especially from different projects, sharing a core you will have degraded performance overall with all these cache misses or whatever. On the other hand you will always have some seti cuda or GPUGRID tasks to crunch more or less. On those rare circumstances that you will not, you can use the cc_config.xml trick.

Hm, you could give some more trust to my words I'm  aware about AK v8 optimizations and participate in AK v8 benchmarking pretty long time already. And being one of direct developers of AP opt app too, btw, surely aware about its interaction with cache.Core switching will give much less performance degradation then just leaving full core idle. I don't know what is done in Linux build that it requires 100% CPU usage (and even not sure that it's so indeed), but this thread about Windows app, moreover, about modification that allows to use whole cores. And you give suggestion to leave one core free ?? If you need free core, don't use this mod at first, this mod designed to use ALL cores AND GPU.

[chelski]

Thanks Raistmer for the wonderful V7 app.  Have been running it fine for about 1 week now on a "weakling" 9600GSO and apparently its thoughput is about 1.5x my Core2Duo E6550 running AK optimised aps.
Just noticed there are some glitches when the app completes one unit and change to another one, on some occasions, ~0.8x per day will see that task stuck there for hours at 0:00:01 CPU times (in my case the CUDA apps neeeds about 30 secs CPU time to prepare the WU for CUDA) for hours and only way to restart it is to restart the client, at which point that WU will normally work out fine.  Tried limiting CPU usage to 95% ( to leave a few % for CUDA WU switching) but somehow it doesn't work.  It is just a small inconvenience that some babysitting for BOINC is required (i generally check 2-3 times a day to restart if required) but any suggestion for other workaround would be great.  And again thanks for the great effort on the CUDA app!

[Raistmer]

Thanks Raistmer for the wonderful V7 app.  Have been running it fine for about 1 week now on a "weakling" 9600GSO and apparently its thoughput is about 1.5x my Core2Duo E6550 running AK optimised aps.
Just noticed there are some glitches when the app completes one unit and change to another one, on some occasions, ~0.8x per day will see that task stuck there for hours at 0:00:01 CPU times (in my case the CUDA apps neeeds about 30 secs CPU time to prepare the WU for CUDA) for hours and only way to restart it is to restart the client, at which point that WU will normally work out fine.  Tried limiting CPU usage to 95% ( to leave a few % for CUDA WU switching) but somehow it doesn't work.  It is just a small inconvenience that some babysitting for BOINC is required (i generally check 2-3 times a day to restart if required) but any suggestion for other workaround would be great.  And again thanks for the great effort on the CUDA app!

Sorry, can't suggest anything but to use some monitoring tool like Maik's script or smth like that to restart hung tasks. Unfortunately (in this context) my own 9600GSO seems doesn't have such problems so I can't reproduce this behavior.... If you can catch hang tasks and attach them here I could try to re-run it on my host in standalone mode - maybe I meet that hang too? ...

[seeker98tw]

Hi
computer:
AMD Phenom(tm) 9650 Quad-Core Processor
and 9600GT Graph-CARD

Can I use 'package_V7' ? Is it working ?

thanks

[Raistmer]

Hi
computer:
AMD Phenom(tm) 9650 Quad-Core Processor
and 9600GT Graph-CARD

Can I use 'package_V7' ? Is it working ?

thanks

Yes, it should work. But you could use V8 too (you have single GPU card). It will work faster for AMD probably.

[seeker98tw]

Thanks

I will test 'AK V8 + CUDA MB team work mod' at night.

I uses your optimized Apps in this month, it compute quickly. very great

[Slawek]

Hi,

Any 64 bit version CUDA ?

[Raistmer]

Hi,

Any 64 bit version CUDA ?

current x86 will go on x64 OS too.
Don't expect any noticeable speed improve from x64 CUDA cause most computations performs on GPU, not on CPU so can't feel bitness of CPU part.

[alexanderhuzar]

My 9600GSO working hard too 58C (~44 in idle state). It has very big cooler  (Asus card)
Thinking is it time to try some overclocking for GPU?....

A word of advice: please run ATI tool to "SCAN FOR ARTIFACTS" for at least 8 hours if you are going to use the GPU or you run the risk of sending bad results if you overclock or don't maintain/clean/check temps on your computer regularly.