Seti@Home optimized science apps and information

Optimized Seti@Home apps => Windows => GPU crunching => Topic started by: Raistmer on 20 Dec 2008, 05:18:24 pm

Title: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 20 Dec 2008, 05:18:24 pm
0) Please, watch carefully what results your CUDA-enabled host returns and stop using CUDA SETI MB till new version will be available if you see excessive invalid results rate.
1) This package attached to this post.
Targed hosts: Windows x86, SSE3 support for AP, CUDA support for MB.

2) It consist of modified SETI MB CUDA and current SSE3 opt SETI AP binaries with corresponding app_info.xml file
3) Modification that I have done increases CUDA worker thread priority in SETI MB CUDA that allows more fully GPU usage while keeping all CPU cores busy too. That is, using of this build can increase total performance of your host for BOINC tasks.
4) MB binaries based on CUDA MB sources from Berkeley's repository (with small modification), opt AP is just repacking of current Lunatics opt AP release (SSE3 build).
5) It's not "official" Lunatics release so you could blame only me (or yourself, or BOINC bugs and so on and so forth) for any issues you encounter.
6) For best CPU and GPU usage I recommend to set number of processors available for BOINC to real_number_of_cores+1. This will mitigate current BOINC bug with CPU+CUDA scheduling and will allow fully load CPU and GPU.
Here is example of minimal cc_config.xml file you need (it also attached to this pos - example for quad)
<cc_config>
<options>
<ncpus>enter_number_of_cores+1_value_here</ncpus>
</options>
</cc_config>

 You should put it in BOINCdata folder (folder that contains projects subfolder).

7) Installation instructions are the same as for any opt app: stop BOINC, decompress all files in archive into SETI project directory, restart BOINC.

Notes for current build:
Ok, new bug fxes available - so new modded app available too :)
VLAR bug still with us although most advanced GPUs (fastest ones) could be free from it already - worth to test. Midrange like my own 9600GSO still suffer from it, especially under Vista.
Again, it's wort to try under XP - there are chances it will work under XP better.

Current bug fixes fight mostly with different overflows. Actually, they should eliminate overflows at all. So, please, report any overflow you will get if it not from VLAR and not from task was ran after driver crash w/o OS reboot.

ADDON: You need 180.48 or upper videodriver version to run this build successfully.

ADDON5 (for V7):
Here is new package version.
It contains the same CUDA part as before (equivalent to 6.08 stock with some mods) and latast opt AP SSE3 version (r103) )
This package will provide suboptimal performance in CUDA part versus V5b (because VLAR CUDA processing very inefficient now and V5b just abort VLAR tasks) but provides performance increase in CPU part because of opt AP rev103 increased performance.

ADDON6 (forV7 VLAR autokill mod):
And one more version.
Now changes in CUDA part of package. This build based on 6.08 sourses but with VLAR auto-kill mod enabled. It sould provide best overall performance at cost of reissuing VLAR task to another host. NOT for using on SETI beta project.


[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilize
Post by: tfp on 20 Dec 2008, 05:55:39 pm
So it's a rebuild of the CUDA app so that the base thread is at a higher priority then 1 but not real time correct?  I've tried running the CUDA app at very high priority and I remember some amount of system lag from the forced priority.

Has anyone figured out if it is possible to run both the CUDA app and AK 8.0 app at the same time or is it only possible to run MB app + CUDA app?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 20 Dec 2008, 06:03:31 pm
1) Yes, it's rebuild CUDA MB with increased worker thread priorty. Yes, it' not real time. It's still lower than priority of all normal processes and their threads, it's just higher than priority (1) of all other BOINC project worker threads.

2) No, combining AK8 with CUDA MB still not possible. It will require merging CUDA part into AK8 opt app and doing some synchronization between different instnces of new AK8.
This will be done eventually I hope. My current package is just temporary solution of course.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 20 Dec 2008, 09:56:13 pm
Hi,
What BOINC version needed ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: tfp on 20 Dec 2008, 10:07:37 pm
I think on the seti forums they were saying to upgrade to 6.4.5 for the CUDA app.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 20 Dec 2008, 10:31:32 pm
Ok working on 6.4.5
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Geek@Play on 21 Dec 2008, 01:08:40 am
 Thanks for your work on this Raistmer.   ;D

When installed on my system with a 4 day cache the system immediately goes into EDF mode and estimated time to completion is all messed up.  This happens with the stock version also so is not your fault.  Question is how does one deal with the messed up times and cache?  Just ignore it?  Will it eventually go out of EDF mode and completion times become more realistic if I continue to run it?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 21 Dec 2008, 02:56:00 am
HI

Working

http://setiathome.berkeley.edu/result.php?resultid=1096327006

http://setiathome.berkeley.edu/result.php?resultid=1096305038

etc..

now im very happy :)

This ( time ) is CPU not real time.. real time is about 1 hours.

Boinc dont computing GPU time work on WU

Ok

Ty guys :)

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Dec 2008, 04:19:27 am
Thanks for your work on this Raistmer.   ;D

When installed on my system with a 4 day cache the system immediately goes into EDF mode and estimated time to completion is all messed up.  This happens with the stock version also so is not your fault.  Question is how does one deal with the messed up times and cache?  Just ignore it?  Will it eventually go out of EDF mode and completion times become more realistic if I continue to run it?
Yes, just ignore. My own host had same problem but now it seems it came to some realistic time predictions and all 5 tasks work in usual mode.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Dec 2008, 04:20:31 am

now im very happy :)


Fine! :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 21 Dec 2008, 04:47:49 am
My GPU is working ok. 6 hours

http://setiathome.berkeley.edu/results.php?hostid=4708858&offset=40 and all time working hard
GPU 62 C temp.:)

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Dec 2008, 04:59:29 am
My 9600GSO working hard too :) 58C (~44 in idle state). It has very big cooler  (Asus card) :)
Thinking is it time to try some overclocking for GPU?.... ;)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Devaster on 21 Dec 2008, 05:36:00 am
overclocked from 700/170 to 720/1800 and ok (9600GT)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Dec 2008, 05:44:11 am
overclocked from 700/170 to 720/1800 and ok (9600GT)
Asus SmartDoctor shows 3 numbers:
Engine: 600
Shader: 1700
Memory(DDR): 1800

And it has various OCing modes: each freq can be changed or some HyperDrive mode can be enabled.
What allowable temperature for GPU? Have I some reserve or 58C(136F) is near to limit ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Devaster on 21 Dec 2008, 06:19:16 am
temp 63 celsius stable at any freq
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 21 Dec 2008, 07:22:28 am
time to OC my SP unit ;P

or buy faster GPU like 260 or 280


Now BOINC computing AP  :o to
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Dec 2008, 07:27:59 am
temp 63 celsius stable at any freq
Fine, thanx. Collecting runtime statistics now then will OC :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 21 Dec 2008, 07:32:23 am
i need info for my 8600GT what shader clock max for this card ?  :-\ and safe temp.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Dec 2008, 07:38:52 am
i need info for my 8600GT what shader clock max for this card ?  :-\ and safe temp.
It's seems now time for searching some GPU OCing forums :) I need that info too :) About temperature - look at Devaster's post.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: MAOJC on 21 Dec 2008, 09:20:11 am
Has anyone tried multiple cuda enabled cards plugged into a single system yet?  Like 3 or 4 PCI cards in a single system.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Dec 2008, 09:40:09 am
Has anyone tried multiple cuda enabled cards plugged into a single system yet?  Like 3 or 4 PCI cards in a single system.
Yes, I read at least two such reports already. Look om main forum thread with same name as this one - there is nice picture (snapshot) of BOINC manager state for syc system (2 cores and 2 GPU all working together on SETI AP + CUDA MB)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: MAOJC on 21 Dec 2008, 11:01:09 am
Has anyone tried multiple cuda enabled cards plugged into a single system yet?  Like 3 or 4 PCI cards in a single system.
Yes, I read at least two such reports already. Look om main forum thread with same name as this one - there is nice picture (snapshot) of BOINC manager state for syc system (2 cores and 2 GPU all working together on SETI AP + CUDA MB)


Nice, so it looks like a nice 4 slot PCI Express board with 4 GPU would be a crunching monster,.

Now what about GPU benchies,  4 9800 GT can be had off Newegg for about $100 each, or 980 Gtx for $169 or better yet GTX260 for about $229 each
Add a MSI 4 slot MB for $129, a nice quad core AMD for $119 and you would be in the super computing business.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Dec 2008, 11:09:04 am
Aha, and liquid nitrogen or helium better for cooling system ;D
Although for "Ultimate cruncher" I could chose i7-based CPU instead.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Gecko_R7 on 21 Dec 2008, 11:26:47 am
Aha, and liquid nitrogen or helium better for cooling system ;D
Although for "Ultimate cruncher" I could chose i7-based CPU instead.

3 X the new GTX295.  6 GPUs.  296W per card @ full load.  soo...900w just for 3 cards.

Curious if there is a standard power supply that can actually run 3 of these beasts on a 1366 board???
Takes (3) 6 pin + (3) 8-pin power connectors.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: arkayn on 21 Dec 2008, 12:49:58 pm
Aha, and liquid nitrogen or helium better for cooling system ;D
Although for "Ultimate cruncher" I could chose i7-based CPU instead.

It would make a nice room warmer for cold winter months though.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Haos on 21 Dec 2008, 09:29:29 pm
Is there any way to combine AK for CPU cores with CUDA GPU app? I rarely get AP workunits, so AP/ MB CUDA often would leave my cpu cores idle...
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Dec 2008, 11:56:38 pm
Is there any way to combine AK for CPU cores with CUDA GPU app? I rarely get AP workunits, so AP/ MB CUDA often would leave my cpu cores idle...
No way still. Try to use SETI/Einstein combo for this moment.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Archangel999 on 22 Dec 2008, 09:38:41 am
pfu i try 178.28 180.60 180.84 driver from nvidia X64 windows xp but just crunch some work units with the gpu cuda and then all units goes error
what is wrong here

use the modifed seti mb cuda + opt ap

i try and boinc 6.4.5 and then 6.5.0 same problem
and even the boinc tell me the unit is success but when i go to the web site i see there is an error in all units
v_BaseLineSmooth (no other)
            v_GetPowerSpectrum 0.00016 0.00000
                   v_ChirpData 0.01016 0.00000
                  v_Transpose4 0.00387 0.00000
               FPU opt folding 0.00170 0.00000
Cuda error 'cudaMalloc((void**) &dev_cx_DataArray' in file 'd:/BTR/seticuda/seti_boinc/client/cuda/cudaAcceleration.cu' in line 234 : out of memory.

</stderr_txt>
]]>


all unit is that problem
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: The Grinch on 22 Dec 2008, 11:04:12 am
I got on x68 BOINC (6.4.5 and 6.5.0) same error!
On my x64 machine i have CUDA don't get work ...
I think: it's very BETA

The only projekt thats work fine is folding@home (for ATI and nvidia)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 22 Dec 2008, 11:13:36 am
Yes, it's very beta indeed.
If it can't work on your host it's better to refrain from SETI CUDA app till next, more stable release. And continue with AK8 opt app for now.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: The Grinch on 23 Dec 2008, 01:27:00 am
And continue with AK8 opt app for now.
Yes, i'm back on AK8, since there will be a stable one.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Bluesilvergreen on 23 Dec 2008, 06:08:27 am
@Raistmer : Great Work!

But, how can I manage to run 4 + 1 with BOINC 6.4.5

I run Einstein and SETI Beta, but only 3 core's of my Q6600 running einstein and 1 core is almost idle, with the CUDA (8800GT) running.

Is it possible to run all 4 core's on einstein and the GPU on SETI Beta?


Anyway my system hangs from time to time for about 1 or 2 seconds. Should I increase the Memory-Bandwidth or try another CUDA-driver? When I stop the CUDA-App it all works fine without any hang up.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Archangel999 on 23 Dec 2008, 08:41:41 am
hi there yes my systemhangs too at exatly at the 15 second it is cuda problem or nvidia driver problem ...
The GPU need's cpu power so it can't do it for now

Best Regards
D.Draganov
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Radiohead on 23 Dec 2008, 11:38:43 am
This modified app WORKED ON 8800GTS 320MB!!!!!!!!

Standart SETI CUDA & GPUGRID:
12/11/2008 11:02:27 AM|SETI@home Beta Test|Scheduler request succeeded: got 0 new tasks
12/11/2008 11:02:27 AM|SETI@home Beta Test|Message from server: No work sent
12/11/2008 11:02:27 AM|SETI@home Beta Test|Message from server: SETI@home Enhanced is not available for your type of computer.


BEST!
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 23 Dec 2008, 02:25:35 pm
1) About 4+1 mode. Look at my original post in this thread (btr, any modifications will be posted there, so it's wise to check head post time to time):
"
7) For best CPU and GPU usage I recommend to set number of processors available for BOINC to real_number_of_cores+1. This will mitigate current BOINC bug with CPU+CUDA scheduling and will allow fully load CPU and GPU.
"
2) 15 seconds problem still unsolved AFAIK. CUDA itself is not very mature technology still. For example, no CUDA kernel can't run more than 3 seconds continuosly or CUDA will crash. Maybe it's the case here.

3) it's definitely not my merit - to allow 8800GTS to tun :) Maybe it's consequence of app_info.xml use.... Maybe it's worth to recommend using of my package for other 8800 owners too...
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Richard Haselgrove on 23 Dec 2008, 03:07:30 pm
Note that there was a server-side modification on the SETI Main project (and presumably Beta too, though I haven't seen any direct reports) about 21:30 UTC, Monday 22 December.

The minimum GPU memory requirement was reduced from 256MB to 254MB. Many of the smaller cards were reporting 255MB, so were switched from 'just too small' to 'just big enough'. That may well account for some users getting CUDA work, when it was 'not available for your [type of] computer' before.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 23 Dec 2008, 03:18:07 pm
Note that there was a server-side modification on the SETI Main project (and presumably Beta too, though I haven't seen any direct reports) about 21:30 UTC, Monday 22 December.

The minimum GPU memory requirement was reduced from 256MB to 254MB. Many of the smaller cards were reporting 255MB, so were switched from 'just too small' to 'just big enough'. That may well account for some users getting CUDA work, when it was 'not available for your [type of] computer' before.
Sure, very possible.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 23 Dec 2008, 04:58:48 pm
Extended outage? Could be promising, ... or not  :-\
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 24 Dec 2008, 11:05:36 am
Okay, Pre-Release Boinc 6.5.0 *seems* to have fixed the +1 operation such that you don't need to set ncpus in cc_config anymore.  Well that's a start.

[..And base priority seems to be 'below normal' now too, might have fixed some of the pauses I was getting.  We'll see... That's two annoyances down!  ]

[Edit: Make that three! ... It seems that it's no longer suffering from oversubscription here either.  Will observe during new cuda task changeover to confirm.]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 24 Dec 2008, 01:41:26 pm
I tried 6.5 on x64. When boinc manager closes it closes only its windows. Process remains in memory and consumes full core (50% on duo).
I tried few times, after reboot after aborting process and so on - it always begins to consume CPU.
So I reverted to 6.4.5 that is official release now (at least for x64). It's pn Win2003 x64 host where I tried to run 2 CUDA- enabled GPUs simultaneously. 8500 ang 9400GT.
it seems only with 180.84 beta it will work w/o out of memory after each tak restart...
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 24 Dec 2008, 01:45:17 pm
LoL, crazy Boinc.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: lofty on 24 Dec 2008, 02:30:41 pm
Hi Raistmer.

Tried your files today and when I restarted Boinc, although it recognised my CUDA abilities (I have an NVIDIA 8800GTX) I got computational errors on all my WUs. I don't think the problem was with the files, I suspect the problem is user related  ::)

I have the latest version of Boinc (6.4.5) and after dloading your zip file I extracted the contents into my seti@home projects folder (C:\Documents and Settings\All Users\Application Data\BOINC\projects\setiathome.berkeley.edu, the default location). Have I done the right thing or was something within the zip file supposed to go to, perhaps, the Boinc installation folder? Sorry to be the newb.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 24 Dec 2008, 03:16:21 pm
You did right thing, but maybe it's time to suspend CUDA indeed.
Look here:

http://setiathome.berkeley.edu/forum_thread.php?id=50937
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 24 Dec 2008, 03:19:31 pm
To avoid further pollution of science database with wrong but passed validation results I suspend my mod distribution.
SETI CUDA should produce valid resutls or give computation error but not invalid "overflows" when go on large scale use. So, it should be repaired.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: lofty on 24 Dec 2008, 03:53:56 pm
This is sad news indeed. I guess I will also suspend any further download of WUs. False results are not good: Bad science is worse than no science. I guess I will finish the WUs I have then return to my old optimized non-CUDA apps for the time being. Thanks for your response.

Merry christmas to crunchers everywhere!
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 25 Dec 2008, 04:11:35 pm
Well, 6.06 sources differ form my own.
For example:

Mine:
          // TODO: Deallocate these at the end of the function
#ifdef USE_CUDA
         if(gSetiUseCudaDevice)
            cudaAcc_fftwf_plan_dft_1d(FftNum, FftLen, NumDataPoints);
   
From Berkeley's repository:

#ifdef USE_CUDA
            // Create FFT plan configuration
         if(gSetiUseCudaDevice)
            {
            if(cudaAcc_fftwf_plan_dft_1d(FftNum, FftLen, NumDataPoints))
                {
                    // If were here, something went wrong in the plan.
                    // Possibly ran out of video memory.  Destroy what we
                    // have and fallback to host CPU processing.
                    fprintf(stderr, "setiathome_CUDA: CUDA runtime ERROR in plan FFT. Falling back to HOST CPU processing...\n");
                    cudaAcc_free();
                    gSetiUseCudaDevice = 0;

                }
            }

#endif

Will rebuild mod with new sources and re-enable access to it for testing.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 25 Dec 2008, 06:10:30 pm
Ok, here is a new package, based on current (380) revision of Berkeley's SETI CUDA branch.


[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 25 Dec 2008, 06:27:03 pm
Does 6.06 fix the memory leak at least ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 25 Dec 2008, 06:42:29 pm
Does 6.06 fix the memory leak at least ?
I think it fixes nothing. But wanna check this for sure.
Joe's proposal about leak fixing still pending. I didn't check this in sources still but pretty sure that modification not included in sources by Berkeley.
rev 380 from 17 December ....

ADDON: now I will use this "up to date"mod to check broken results standalone too.
I didn't see any problems with PG* VHARs before, only with VLARs....

And first E-mailed sources differ from Berkeley's rep ones... So it's worth to try...
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 26 Dec 2008, 04:06:42 pm
Here is new build with logging of AR of overflowed tasks.
It will create and append later file r_debug.txt on C:\ and will write AR of overflowed task there.
Sure there will be "legal" overflows too, but any overflows not validated against CPU result worth to report.

Just replace CUDA MB executable from my last package with this one. Name remains same.

Logging mod is:

  if (e == RESULT_OVERFLOW) {
        fprintf(stderr, "SETI@Home Informational message -9 result_overflow\n");
        fprintf(stderr, "NOTE: The number of results detected exceeds the storage space allocated.\n");
      {//R: Logging of AR of overflowed task
         FILE* f=fopen("C:\\r_debug.txt","a");
         if(f){
            fprintf(f,"AR of overflowed task is\t%f\n",swi.angle_range);
            fclose(f);
         }
      }



[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: wndrbr3d on 27 Dec 2008, 01:34:42 pm
Why don't you have the debug file write to the BOINC project folder?

People running Vista won't have access to write to root unless they're running BOINC and it's child processes as administrator. This way everyone can get a debug, not just the XP users  ;D
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: popandbob on 27 Dec 2008, 02:17:29 pm
Also might be an idea to log the AR's of the wu's that don't overflow to see if there are any ranges that for whatever reason don't overflow...
Like for me 2.71 overflows 2.72 doesn't (for the most part)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 27 Dec 2008, 04:28:17 pm
Why don't you have the debug file write to the BOINC project folder?

People running Vista won't have access to write to root unless they're running BOINC and it's child processes as administrator. This way everyone can get a debug, not just the XP users  ;D
To lazy to figure out where to get SETI project directory on BOINC installation. Any ideas?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 27 Dec 2008, 04:29:13 pm
Also might be an idea to log the AR's of the wu's that don't overflow to see if there are any ranges that for whatever reason don't overflow...
Like for me 2.71 overflows 2.72 doesn't (for the most part)
Don't wanna be blaming for hard drive overloading ;)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Josef W. Segur on 27 Dec 2008, 10:35:25 pm
Why don't you have the debug file write to the BOINC project folder?

People running Vista won't have access to write to root unless they're running BOINC and it's child processes as administrator. This way everyone can get a debug, not just the XP users  ;D
To lazy to figure out where to get SETI project directory on BOINC installation. Any ideas?

The project_dir in the APP_INIT_DATA structure. The structure is defined in the BOINC app_ipc.h file, seti.h has a "extern APP_INIT_DATA app_init_data;" line, and part of initialization is to read the init_data.xml file in the slot directory and fill in the structure. For standalone testing it will probably be a null string, then maybe just using the current directory might make sense.

That's assuming the app will have permission to write in the project directory, I'm not sure about that.
                                                                         Joe
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 28 Dec 2008, 04:23:40 am
Thanks again, Joe :)
Well, writing in current directory not suitable for BOINC runs (slot will be deleted). Will try to change to project directory and check on VIsta.
C:\r_debug.txt still doesn't exist here although host completed few VHAR tasks. SO, BOINC-summoned app can't access to C:\ indeed  :-\
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: efmer (fred) on 28 Dec 2008, 10:52:17 am
This works always as iit is the Windows Temp Directory GetTempPath(MAX_PATH,wTempPath);
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 28 Dec 2008, 11:28:05 am
Thanks here too and welcome on Lunatics forums :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 28 Dec 2008, 12:58:26 pm
Hi

What news in CUDA APP ? new version?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 28 Dec 2008, 01:07:25 pm
Hi

What news in CUDA APP ? new version?
Ok, will build one with log in temp folder. And available memory log at task startup. It will illustrate memory leak problem I hope. So, only cosmetic changes.
Nothing new in Berkeley's repository still.

Addon: here some cosmetic improved version
Now will be this string in stder:

Total GPU memory 402653184    free GPU memory 355926016

If the GPU memory value will decrease from task to task - it's better to reboot before CUDA failures begins.


[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 28 Dec 2008, 03:11:47 pm
Ok, here is new build that incorporates all prev mods + attempt to fight with possible memory leak on overflow exit.


[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: efmer (fred) on 30 Dec 2008, 04:42:52 am
2 work units in the low ar range
http://setiathome.berkeley.edu/result.php?resultid=1108406642
http://setiathome.berkeley.edu/result.php?resultid=1108406628
Succeeded but with error. No crash, I have more....
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 30 Dec 2008, 05:48:49 am
2 work units in the low ar range
http://setiathome.berkeley.edu/result.php?resultid=1108406642
http://setiathome.berkeley.edu/result.php?resultid=1108406628
Succeeded but with error. No crash, I have more....

Ok, try to keep WUs available (copy them in other place on HDD) to be able upload interesting results along with their tasks too (w/o task we can't reproduce error or compare with CPU app in standalone mode).
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: efmer (fred) on 30 Dec 2008, 07:47:49 am
Posted 2 tasks on the seti board
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 31 Dec 2008, 05:32:30 am
Hi.

AP only computing 1 core? i have 2 WU only 1 is computing.
 CUDA computing now to

i have x2 Atlhon
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 31 Dec 2008, 06:03:53 am
Hi.

AP only computing 1 core? i have 2 WU only 1 is computing.
 CUDA computing now to

i have x2 Atlhon
Look first post of thread for instructions how to utilize both cores.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 31 Dec 2008, 06:05:09 am
Ok working 2 cores.:)

Happy new year!:P
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: gerga on 01 Jan 2009, 08:02:19 am
7) For best CPU and GPU usage I recommend to set number of processors available for BOINC to real_number_of_cores+1. This will mitigate current BOINC bug with CPU+CUDA scheduling and will allow fully load CPU and GPU.
Hi Raistmer

How do you set "real_number_of_cores"?

Happy new year!!
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 01 Jan 2009, 08:07:49 am
7) For best CPU and GPU usage I recommend to set number of processors available for BOINC to real_number_of_cores+1. This will mitigate current BOINC bug with CPU+CUDA scheduling and will allow fully load CPU and GPU.
Hi Raistmer

How do you set "real_number_of_cores"?

Happy new year!!

Edit or create file cc_config.xml
<cc_config>
<options>
<ncpus>0</ncpus>
</options>
</cc_config>

Set needed value instead 0.
cc_config.xml should be located in BOINCData folder.

Happy New Year! :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: beeze on 02 Jan 2009, 09:54:12 pm
boinc 6.5.0 works right without using a cc_config file
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 03 Jan 2009, 02:44:40 pm
boinc 6.5.0 works right without using a cc_config file
It has more annoying bugs, I reverted to 6.4.5 after trying...
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 03 Jan 2009, 05:10:24 pm
Works for me too (XP32 w/SP3) , but I stopped testing the cuda apps and removed them as I have some work to do, and they were causing some instability on my machine.  I'll wait 'till they fix a few things I reckon.

[Edit: Placing copy of Stock 6.05 & 6.06 binaries for those that can;t get them themselves, downloaded from fanout. (attached) ]



[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 03 Jan 2009, 06:47:23 pm
[Edit: Placing copy of Stock 6.05 & 6.06 binaries for those that can;t get them themselves, downloaded from fanout. (attached) ]

Thanks alot. Now i have a reference-app for stand alone testing ;)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 03 Jan 2009, 07:55:57 pm

[Edit: Placing copy of Stock 6.05 & 6.06 binaries for those that can;t get them themselves, downloaded from fanout. (attached) ]


Thanks :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 04 Jan 2009, 03:46:53 pm
Hi in new year:D

http://setiathome.berkeley.edu/result.php?resultid=1110879221

http://setiathome.berkeley.edu/result.php?resultid=1110877344

and more.. in my acconut.

:) and new hardware :P

no problems etc :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Devaster on 06 Jan 2009, 08:27:59 am
Raistmer, put your mod version of cuda sources to the our svn please ....
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 06 Jan 2009, 08:47:13 am
Suggestion, if you want to do that, add it to the SaHv6 Repsoitory as a new folder next to Opt and Stock:

by checking out:
svn://sinbadsvn.dyndns.org/SaH_v6b/
then creating & adding a new folder like 'Stock_Cuda', or some such, with the contents.

Probably wise to commit unmodified stock first, and then the small updates you have so the changes are logged for posterity.

Jason
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 06 Jan 2009, 12:20:38 pm
Did you put version I recived by E-mail before Berkeley's SVN was available in sinbad's rep already or there is no CUDA MB at all ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 06 Jan 2009, 12:34:09 pm
Me? Nope, No Cuda code development here. I have no interest in the stock nvidia cuda code since it crashes my machine, feel free to install the sources in that stock sinbad repository though for mimo's access (& easy logged progress  etc...)., That is a separate machine primarily dedicated for that purpose.

Maybe I will look and see if I can see anything obvious in the code if you put it there.  Sorry, this machine I work on is fairly 'mission critical' for other work at the moment, so only relatively trusted builds (Like ours) can cut the mustard.  It runs MatLab, Ansys, Inventor and Altium Designer which are all demanding and use DirectX, so "fubarred" device drivers are not an option until certain projects are completed, so I simply haven't risked playing.

Jason
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 06 Jan 2009, 12:39:51 pm
Ok, don't think it's sources that crashed you PC, hope it's only binaries built from that sources ;D
Sources pretty harmless ;D ;D ;D
Will put what I have into CUDA branch.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 06 Jan 2009, 12:43:38 pm
LoL, Indeed, but the temptation with sources on the machine is to (try to) fix them, run them, then experience the nightmare. As mentioned before, I resist and leave that barrel of fun to you guys (for now   :P)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 06 Jan 2009, 12:58:05 pm
Oops, and what is it ?
svn://sinbadsvn.dyndns.org/SaH_v6b/Cuda_app/seti_boinc
is it CUDA app indeed ?

With separate BOINC...
Maybe it's that sources I recived from Eric initially ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 06 Jan 2009, 01:00:39 pm
I have no idea!  ;D ... Checking logs  ::)

[Edit: Ahh!, Yes I put 6.05 there ... Please update this one  ;D]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 06 Jan 2009, 01:02:38 pm
I have no idea!  ;D ... Checking logs  ::)

LoL :)
"
Revision: 58
Author: jason
Date: 15:19:28, 13 декабря 2008 г.
Message:
Initial Berkeley Cuda app sources v6.05
----
"

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 06 Jan 2009, 01:04:26 pm
Ok, I have modded rev 380 in some folder on HDD - wanna share it with this CUDA rep. Please, suggest how to change repositories most effective way w/o recreating another folder (wanna have CUDA Berkeley/sinbads together)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 06 Jan 2009, 01:10:46 pm
Okay:

  - Checkout old cuda_app folder (fresh checkout)
  - go into folder, select all non-hidden, and rightclick-->TortoiseSVN-->Delete  (Not windows delete) ... They will disappear, but properly interacting with SVN.
  - Copy the replacement  files in using RightClick+Drag, On Release select "Copy & Add Files to This SVN WC" (or similar it says anyway)
  - Commit with nice friendly log describing exactly all the bugs and how to fix them, and all your girlfriends phone numbers.

Easy enough?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 06 Jan 2009, 01:31:07 pm
Hehe :)
But it will be 2 separate folder with sources on my own HDD that way?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 06 Jan 2009, 06:32:18 pm
Not necessarily when done, we don't need to keep separate trees for that as they are sequential updates to the same project.  As you'd be effectively replacing the repository files with yours, no need to have 2 local copies when done either. You keep the repository version active for working on and archive/delete any other you're not working on.  The only need is for this in the repository.

Stockv6 cuda_app folder log would look like this:
  - First Revision = Berkeley 6.05 unmodified. (Obsolete but there for preservation)
  - Second revisiion = Berkeley 6.06 unmodified (Also obsoleted by your mods, but there if needed)
  - Third revision = Your 6.06

[Of course the revision numbers will be higher that 1,2 & 3, because of mixed in with Stockv6 non-cuda repository.]

So only one progressive Repository is needed but can fall back to any point later if you decide to, and could even create a branches subfolder if you later need two separate builds for some reason, but I can't see any need to do that,  unless you find your mods seriously went in the wrong direction, but even then that is better handled by reverting individual files.  Branching (which would make two separate lines to keep locallly) should only be needed if there are some reasons the builds need to develop in different directions then later hopefully find a way merge.  An example of this is the AK_v8 split off of SSE build because it broke SSE2+ speed.  Hopefully one day these become reintegrated.


 
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 06 Jan 2009, 07:37:25 pm
I mean I want to save possibility to update from Berkeley's repository and commit to sinbad's one - is it possible from single workcopy on HDD or not?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 06 Jan 2009, 07:58:45 pm
There is one way to make it a bit easier that I'll go into further down, though, the easiest way I've found to accomplish this, like when AstroPulse needed to be updated to version 5 code from 4.35, is to drop the Berkeley files over an already committed working copy.  All changes then show in "Check for Modifications" which you can then do the Ping-Pong merge game on a per file basis, to decide which changes are relevant to your build.

The other way would be to use the SVN 'Externals' feature.  Look up 'external repositories' in the TortoiseSVN help.  Basically it should allow you to grab different subfolders (say one named 'Berkeley') which would sync to Berkeley whenever you do an update, then you can use regular merge tools like 'Two-Tree-merge' to grab the changes you want.

Up to you which approach to use. Just avoid committing until you have it arranged the way you want, but commit before you try merging new code in from Berkeley and you should be alright.  Either way it is still some juggling, though both are effective.  My only aversion to Method #2 is that updates of the repository rely on two servers being operational at the same time.

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 06 Jan 2009, 08:46:54 pm
Can't just copy berkeley's over existing files - will rewrite svn data that way too.... so updated just separate files.
Incostintent line endings error again....
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 06 Jan 2009, 09:55:16 pm
Yeah, opening the changed files with Visual studio before trying to commit should fix that, though you'd have to see if the option is turned on in VS options to check/fix consistency on open.

I kindof like SVN's paranoia about line endings consistency, despite they should be set to native and not care.  It can be revealing to see which files have the mixture.

Note: Copy over the files, not the SVN hidden data directory and you'll be right. (It's hidden for that exact reason, you want to update your svn not corrupt it)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 07 Jan 2009, 06:30:25 am
Note: Copy over the files, not the SVN hidden data directory and you'll be right. (It's hidden for that exact reason, you want to update your svn not corrupt it)
Hm... I revert folder settings to not show hidden files but explorer tries to overwrite .svn folder anyway....  Why it keeps do that ?...
I try to copy via right click Copy/Paste with all folders in seti_boinc selected.

Will export files from berkeley's rep first to get rid from .svn folders, then copy over sinbad's rep. It should work better :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 07 Jan 2009, 06:43:49 am
At revision: 65
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 07 Jan 2009, 06:56:52 am
Ahh Yes! You are learning the hard juggling act I had to do to update the AstroPulse 4.35 to Berkeley v5 sources.  It takes much beer and unnatural patience.  8)  .  Will take a look see what you've done and see if I can suggest anything.

Jason
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Archangel999 on 08 Jan 2009, 05:17:07 am
  v_BaseLineSmooth (no other)
            v_GetPowerSpectrum 0.00016 0.00000
                   v_ChirpData 0.01060 0.00000
                  v_Transpose4 0.00415 0.00000
               FPU opt folding 0.00290 0.00000
Cuda error 'cudaMemcpy(&flags, dev_flag, sizeof(*dev_flag), cudaMemcpyDeviceToHost)' in file 'd:/BTR/seticuda/Berkeley_rep/client/cuda/cudaAcc_gaussfit.cu' in line 510 : unknown error.

</stderr_txt>
]]>


---------------------------------

  v_BaseLineSmooth (no other)
            v_GetPowerSpectrum 0.00016 0.00000
                   v_ChirpData 0.01154 0.00000
                  v_Transpose4 0.00495 0.00000
               FPU opt folding 0.00199 0.00000
Cuda error 'cudaAcc_summax32_kernel' in file 'd:/BTR/seticuda/Berkeley_rep/client/cuda/cudaAcc_summax.cu' in line 147 : unknown error.

</stderr_txt>
]]>

---------------------------------

             v_BaseLineSmooth (no other)
            v_GetPowerSpectrum 0.00016 0.00000
                   v_ChirpData 0.01156 0.00000
                  v_Transpose4 0.00444 0.00000
               FPU opt folding 0.00292 0.00000
Cuda error 'cudaMemset(dev_flag, 0, sizeof(*dev_flag))' in file 'd:/BTR/seticuda/Berkeley_rep/client/cuda/cudaAcc_gaussfit.cu' in line 499 : unknown error.

</stderr_txt>
]]>

-------------------------------

       v_BaseLineSmooth (no other)
            v_GetPowerSpectrum 0.00017 0.00000
                   v_ChirpData 0.01245 0.00000
                  v_Transpose4 0.00441 0.00000
               FPU opt folding 0.00194 0.00000
Cuda error 'cudaAcc_GetPowerSpectrum_kernel' in file 'd:/BTR/seticuda/Berkeley_rep/client/cuda/cudaAcc_PowerSpectrum.cu' in line 56 : unknown error.

</stderr_txt>
]]>

----------------------

and more errors this is with MB_6.06r380mod_CUDA.exe with Windows Xp x64 and 180.84 cuda driver and SEtI 6.05


Best Regards
D.Draganov
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Archangel999 on 09 Jan 2009, 12:42:53 pm
i'm waiting for new version or some update i cannot understand why cuda application won't start it still at 0.00% and then i have to suspend that wu and go to another i see that the raistmer application is backround running .... all 30 pack hade to suspend because they won't start 13 hour at 0,00%  :o .... but some done with no problem


Best Regards
D.Draganov
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 09 Jan 2009, 12:45:22 pm
did you tried this one? (http://lunatics.kwsn.net/gpu-crunching/vbscript-fights-cuda.0.html)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 09 Jan 2009, 12:53:36 pm
i'm waiting for new version or some update i cannot understand why cuda application won't start it still at 0.00% and then i have to suspend that wu and go to another i see that the raistmer application is backround running .... all 30 pack hade to suspend because they won't start 13 hour at 0,00%  :o .... but some done with no problem


Best Regards
D.Draganov
Check what AR they have, please. And try to use Maik's VB script, indeed, to restart hang task.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Archangel999 on 09 Jan 2009, 04:07:25 pm
Raimster work but wow a lot of wu are in the crash section :D :D
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 09 Jan 2009, 04:10:33 pm
Raimster work but wow a lot of wu are in the crash section :D :D
tasks come in packs, so whole low AR pack could be crashed (better - canceled before ;) ) at once.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 09 Jan 2009, 04:11:45 pm
Raimster work but wow a lot of wu are in the crash section :D :D

Oh Hell !

did u read that:

Be carful if you use that 'automated' WU-cleaning (process-terminating). Check your host from time to time.
Some WU's are causing your grafic driver to crash. The result are a lot of failing WU's in a row (noticed that this morning at my host).
The crash announces itself with around jumping pixels everywhere. Some users posted that already ...

if u notice that, stop seti and do a reboot :P
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Archangel999 on 09 Jan 2009, 04:22:50 pm
yea when i gotta sleep  ;D ;D
thanks anyway :)


Best Regards
D.Draganov
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 10 Jan 2009, 02:33:06 am
Here is new build based on rev396 Berkeley's sources with mods (as usual :) ).
Changes: triplet finding overflows bug-fix incorporated, CUDA fast math enabled.
So, it could be faster than prev builds and shoul do correctly more WUs than before. VLAR bug still with us  though :P
BEWARE, app name changed! So you need to edit app_info.xml or download V4a of my package from first post in this thread (attached to this post too)

ADDON: You need 180.48 or upper videodriver version to run this build successfully.

[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Alex Kraft on 10 Jan 2009, 03:23:14 am
Seem that this app´s running :)

Tested with two PC´s. Vista 64 bit. Nvidia 9800 GT and 9800GTX+
Boinc 6.5 and Nvidia Driver 181

But one Question i have. I wont let run both apps. I mean GPU and CPU Seti Apps. At the moment i let run Seti GPU and WCG.
What must i do in the settings?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Devaster on 10 Jan 2009, 05:05:08 am
bench attached

[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 10 Jan 2009, 05:22:05 am
Seem that this app´s running :)

Tested with two PC´s. Vista 64 bit. Nvidia 9800 GT and 9800GTX+
Boinc 6.5 and Nvidia Driver 181

But one Question i have. I wont let run both apps. I mean GPU and CPU Seti Apps. At the moment i let run Seti GPU and WCG.
What must i do in the settings?

You can run only SETI MB and SETI AP on CPU+GPU simultaneously now. There is no known option to run only SETI MB on both CPU and GPU w/o too much hand work (you need to change task headers starting/stopping apps and so on).
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 12 Jan 2009, 11:34:15 am
Hello all


What news ? New cuda APP ? ( where? ;P )

now im only computing AP ... not CUDA :( no new work for my CUDA engine :(

 ??? :'(
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 12 Jan 2009, 01:00:02 pm
Hello all


What news ? New cuda APP ? ( where? ;P )

now im only computing AP ... not CUDA :( no new work for my CUDA engine :(

 ??? :'(
Dont'know why you use only AP. There is SETI MB CUDA version available. This thread just about modification of that version.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 12 Jan 2009, 08:54:47 pm
Ok, new bug fxes available - so new modded app available too :)
VLAR bug still with us although most advanced GPUs (fastest ones) could be free from it already - worth to test. Midrange like my own 9600GSO still suffer from it, especially under Vista.
Again, it's worth to try under XP - there are chances it will work under XP better.

Current bug fixes fight mostly with different overflows. Actually, they should eliminate overflows at all. So, please, report any overflow you will get if it not from VLAR and not from task was ran after driver crash w/o OS reboot.



[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Richard Haselgrove on 13 Jan 2009, 09:28:43 am
I've seen one overflow with CUDA stock v6.07: http://setiweb.ssl.berkeley.edu/beta/workunit.php?wuid=1693470

That had 30 spikes at AR=0.378607 - but the CPU v6.03 validator found exactly the same thing, so I'm assuming it's genuine.

Edit - likewise http://setiweb.ssl.berkeley.edu/beta/workunit.php?wuid=1693468
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 13 Jan 2009, 10:42:00 am
Sure, nobody cancel "valid" overflows existence. Only "invalid" ones important o CUDA MB.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 13 Jan 2009, 12:21:04 pm
This build devoted to  all who still experiences videodriver crash on VLAR tasks. It should abort its own execution if VLAR with AR <0.14 is detected (temporary measure, of course).
All thanks go to Crunch3r & Maik  for this mod.


[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 13 Jan 2009, 12:24:31 pm
Let me know how it goes and I'll crank it up again if it now stops crashing drivers and doesn't require manual babysitting.  Once the heat here goes down anyway, I don't really fancy pumping more Watts into the room right at the moment 8)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 13 Jan 2009, 12:32:19 pm
Let me know how it goes and I'll crank it up again if it now stops crashing drivers and doesn't require manual babysitting.  Once the heat here goes down anyway, I don't really fancy pumping more Watts into the room right at the moment 8)
Hehe, yes, heating more needed here than at your place now ;D
It should go OK now, but will see (as usual)

ADDON: oops, I just going to post on main forums about update but project is closed for maintenance now... Time to start moan it seems ;D ;D ;D

ADDON2 - unfortunately I already deleted all VLARs from current cache load + it seems beta feeds only midrange AR tasks now (see Richard's posts). So online mod validation will take some more time.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Crunch3r on 13 Jan 2009, 12:58:38 pm
... Time to start moan it seems ;D ;D ;D



Moan ? I can do that a lot !  :P
Let me see, where should I start??

First cross-project parity and credit correction... is that what you had in mind ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 13 Jan 2009, 01:05:57 pm

First cross-project parity and credit correction... is that what you had in mind ?

No, I just joked about usual SETI forums moanings about any server outages even if they scheduled ones.
As I said in another thread all cobblestone value changes is really bad thing in my point of view too. Idea of interproject comparison is not so bad, but it should not be determining one IMHO...

ADDON: If cobblestone credits and RAC are overtopped to inter-project comparison idea they lost many advntages and become just measures of some artifical "absolute rating" between participants. I'm not sure that correct "absolute rating" between very different projects is possible at all.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Archangel999 on 13 Jan 2009, 03:42:01 pm
use mb_r396 mode with my windows xp x64 and all is ok :) no more crash and errors :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 13 Jan 2009, 03:43:27 pm
use mb_r396 mode with my windows xp x64 and all is ok :) no more crash and errors :)
396 is outdated now. Use 400 instead.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 13 Jan 2009, 03:44:11 pm
About Vista's driver restart:

The default timeout period in Windows Vista is 2 seconds.

http://www.microsoft.com:80/whdc/device/display/wddm_timeout.mspx
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: perryjay on 13 Jan 2009, 03:45:31 pm
Wait! Are you saying they are thinking about doing away with credits?  But I still don't have enough for my toaster!  :(
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Archangel999 on 13 Jan 2009, 04:27:14 pm
use mb_r396 mode with my windows xp x64 and all is ok :) no more crash and errors :)
396 is outdated now. Use 400 instead.

thanks but i will use 396 because it is ok for me ;)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 13 Jan 2009, 05:47:00 pm
use mb_r396 mode with my windows xp x64 and all is ok :) no more crash and errors :)
396 is outdated now. Use 400 instead.

thanks but i will use 396 because it is ok for me ;)
It will be Ok just until new bunch of tasks arraives :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Josef W. Segur on 13 Jan 2009, 06:21:25 pm
About Vista's driver restart:

The default timeout period in Windows Vista is 2 seconds.

http://www.microsoft.com:80/whdc/device/display/wddm_timeout.mspx

Yeah, and here's what David Anderson is thinking of doing:
Quote
From: David Anderson <davea@ssl.berkeley.edu>
Cc: boinc_alpha@ssl.berkeley.edu
To: BoincSpy Administrator <boinc_spy@telus.net>
Date: Sun, 11 Jan 2009 21:26:37 -0800
Subject: Re: [boinc_alpha] CUDA Blue Screen of death.
Message: 3

I've seen this also.
It's not related to driver version.
According to NVIDIA engineers,
if a GPU driver request doesn't complete in 2 seconds,
Windows assumes it's hung, and crashes.
Apparently the SETI@home/CUDA app does something that
sometimes takes > 2 secs on slow GPUs.

The NVIDIA people didn't have a fix or workaround for this.
So I'm going to change the server so that it won't send
CUDA jobs to GPUs slower than 60 GFLOPS
(I'm not sure this is the magic number,
but on my machine the GPU is 50 GFLOPS,
computed as clockRate * multiProcessorCount * 2,857).

-- David

From a comment somewhere in the BOINC sources, his board is a Quadro FX 3700.
                                                                          Joe
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 13 Jan 2009, 06:25:00 pm
Wrong decision IMHO.
1) only some range of tasks suffer from this trouble - so no need to restrict whole host from processing.
2) it possible to repair application or at least tofall back to CPU version on these affected tasks.

That is - no intervntion on server side is required, at least in form of restriction whole host from getting CUDA work.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 13 Jan 2009, 06:27:14 pm
This build devoted to  all who still experiences videodriver crash on VLAR tasks. It should abort its own execution if VLAR with AR <0.14 is detected (temporary measure, of course).
All thanks go to Crunch3r for this mod.

Hmm.... im wondering where this idea is from .... looking at my script - code --- well, a angle range check - routine ...
and then the idea poped up after crunch3r downloaded my script / took notice of the thread....  :P :P :P :P
a credit would have been nice
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 13 Jan 2009, 06:30:13 pm
:)
No prob
ADDON: done.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Richard Haselgrove on 13 Jan 2009, 07:22:31 pm

Hmm.... im wondering where this idea is from .... looking at my script - code --- well, a angle range check - routine ...
and then the idea poped up after crunch3r downloaded my script / took notice of the thread....  :P :P :P :P
a credit would have been nice


To be fair, I don't think anyone was trying to belittle your contribution. We've all had something to say on the subject, starting with Raistmer's thread on Beta (http://setiweb.ssl.berkeley.edu/beta/forum_thread.php?id=1443) and Alexander's batch file. Each contributer has drawn on and extended the work which has gone before.

So here's a suggestion for the next phase. Make the VLAR autokill threshhold a variable, and read it in from an XML file somewhere [I used to love .INI files when I was programming, but I digress]. That way, those of us who like living dangerously can test out ever-lower ARs, while the default value can stay at the nice safe (I would say over-cautious) 0.14 level. And we don't have to ask Crunch3r to compile a new version every day with the new daily threshhold hard-coded into the source.... :D
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 13 Jan 2009, 07:47:05 pm
To be fair, I don't think anyone was trying to belittle your contribution. We've all had something to say on the subject, starting with Raistmer's thread on Beta (http://setiweb.ssl.berkeley.edu/beta/forum_thread.php?id=1443) and Alexander's batch file. Each contributer has drawn on and extended the work which has gone before.

So thanks to Alexander too.

So here's a suggestion for the next phase. Make the VLAR autokill threshhold a variable, and read it in from an XML file somewhere...

There is no way in my scipt to terminate specified WU's (and i will not add it). This script is just watching the process (like you could do via WinTaskManager) and terminating this process if it is idle for 60 sec. The BM is restarting the same wu after this happend. You could do that by yourself by marking the task on WTM and terminate it.
The idea on my script is to do that work after you have filtered your cache and have enough work to do others things than watching the WTK ;)
With non-VLAR-taks that is working fine for me. After a terminate like this the task is running to 100% and at stderr you will see 'Restart at xx%'.
So there is no obstacle to crunch VLAR's. Just dont delete it if BM gets new tasks ;)
But no warranty that it is working if you use my script and crunch VLAR's. If you get a driver-crash all following WU's will maybe fail  :P
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 13 Jan 2009, 07:52:56 pm
like living dangerously can test out ever-lower ARs, while the default value can stay at the nice safe (I would say over-cautious) 0.14 level. And we don't have to ask Crunch3r to compile a new version every day with the new daily threshhold hard-coded into the source.... :D
Hehe, actually, it's my build again, no prob to rebuild it (just for sake of total and complete correctness ;D ) Idea with variable threshold is good but I personally have hope and trust that this annoing VLAR issue will be solved very soon. So actually do't want to put any more efforts in this - many different tasks on hold...
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 13 Jan 2009, 08:02:21 pm
btw.
I was testing the v5 400basic mod on a VLAR: 0.0112
No crash, laggy like hell and it took 25min to do 1%.Ii aborted it.
I think my card is still to slow for VLARs so i'll try the v5a. thanks for this ;)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 13 Jan 2009, 09:27:29 pm
Current bug fixes fight mostly with different overflows. Actually, they should eliminate overflows at all. So, please, report any overflow you will get if it not from VLAR and not from task was ran after driver crash w/o OS reboot.



v5a, driver 8120, no driver crash, next wu's in list running fine

Edit: 2 more -9 overflow ... did reboot ... next overflow ... the 3new ones all with Spike count:    30
still have 11 more wu's form this series in task list, lets see ...
summary 5 wu's now, all with AR: 2.579270, all from 16no08ab.25078.13571.14.8.x_x ... -series

it seems that this series picked up the iss or something like that ;)
all wingmans who returned results yet have same results ...


[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 04:18:15 am
Ok, I formulated it bad.
Rephrase: All CUDA MB overflows that didn't pass validation versus CPU wingman - because there is "usual" overflows and "non-usual" ones.
Surely we take interest only for "non-usual" ones for this moment.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Archangel999 on 14 Jan 2009, 04:43:03 am
use the new app and work better then the 396 but are more lagy but it crunch realy fast  i 'm having 8800gtx and use the new vbscript all working fine
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 04:47:44 am
There is known "feature" of BOINC 6.4.5 that now works more as bug - it sets process priority for CUDA application to "Normal" instead of "Idle"
It helps nothing because "Idle" priority of worker thread has the same value for "Normal" and "Idle" priorities of process (so called priority class) so thread priority should be adjusted, not process priority (not priority class). Setting process priority to "Normal" excessively increase control thread priority that caused sluggish OS behavior.

So for Don's (Geek@Play) request I did another cleanup for BOINC - now modded CUDA MB will adjust priority class as well. Hope it will give more smooth execution.
V5b attached.



[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Richard Haselgrove on 14 Jan 2009, 05:45:38 am

Yeah, and here's what David Anderson is thinking of doing:
Quote
...
The NVIDIA people didn't have a fix or workaround for this.
So I'm going to change the server so that it won't send
CUDA jobs to GPUs slower than 60 GFLOPS
(I'm not sure this is the magic number,
but on my machine the GPU is 50 GFLOPS,
computed as clockRate * multiProcessorCount * 2,857).

-- David

From a comment somewhere in the BOINC sources, his board is a Quadro FX 3700.
                                                                          Joe

That card has the same number of shaders (112) as my 9800GT (but costs six times as much - hehe).

I can't find a clock speed for it - but if I plug the shader clock for the 9800GT (1500 MHz) into the formula, I get 48 GFLOPS: seems like the right ballpark.

If they're going to set the limit 25% above a 9800GT, then GPU crunching is dead until the next generation of hardware.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: alollini on 14 Jan 2009, 05:52:08 am
I try with V5a ... that seems to work
however I have still only 4 tasks running, that may be better with 5

if the overall performance is better with real_number_of_cores + 1

where do i enter this setup ? in what file ? what line ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 06:08:05 am

Yeah, and here's what David Anderson is thinking of doing:
Quote
...
The NVIDIA people didn't have a fix or workaround for this.
So I'm going to change the server so that it won't send
CUDA jobs to GPUs slower than 60 GFLOPS
(I'm not sure this is the magic number,
but on my machine the GPU is 50 GFLOPS,
computed as clockRate * multiProcessorCount * 2,857).

-- David

From a comment somewhere in the BOINC sources, his board is a Quadro FX 3700.
                                                                          Joe

That card has the same number of shaders (112) as my 9800GT (but costs six times as much - hehe).

I can't find a clock speed for it - but if I plug the shader clock for the 9800GT (1500 MHz) into the formula, I get 48 GFLOPS: seems like the right ballpark.

If they're going to set the limit 25% above a 9800GT, then GPU crunching is dead until the next generation of hardware.
Sorry but I consider this approach as meaningless. We speak about doing SETI on Android platform, ON CELLPHONES (!!!) and throw away 50GFLOPs of power? Nonsense!
So, there will be custom builds that will work on all CUDA-enabled GPUs and will just abort VLARs as my current V5a/b do.  Much better if true bug-fix will be developed than such inefficient workaround... Even falling back to CPU processing approach much better than this.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 06:11:48 am
I try with V5a ... that seems to work
however I have still only 4 tasks running, that may be better with 5

if the overall performance is better with real_number_of_cores + 1

where do i enter this setup ? in what file ? what line ?
search for cc_config.xml file description. You need ncpus field.
I will add minimal cc_config.xml example to first post of this thread.
(done. You can take that sample and edit it if you have duo or single-core CPU instead of quad)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: alexeft on 14 Jan 2009, 06:15:51 am
Hi Reistmer, thanks for all the effort you put (along with Crunch3r)

I'm using 5b and seems to work fine, had no crashes yet.

What I notice is that it goes slow at the beginning of the WU and then accelerates rapidly. Why is that so?
Also, the timer is totally wrong.

Thanks again!
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Richard Haselgrove on 14 Jan 2009, 06:16:07 am

search for cc_config.xml file description. You need ncpus field.


http://boinc.berkeley.edu/wiki/Client_configuration
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 06:20:54 am
What I notice is that it goes slow at the beginning of the WU and then accelerates rapidly. Why is that so?

This behavior is not specific to V5b build AFAIK. At the beginning of task (and on each restart) some CPU pre-processing should be done (and GPU initial feeding by data) - so few seconds there will be no % done increase.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: alexeft on 14 Jan 2009, 06:29:28 am
No, I'm not talking about those few seconds.

In the beginning it goes at 0.05% per second.
In the middle it goes at 0.1% per second.
Toward the end it goes at 0.3% per second.

Not specific to 5b, all versions did that!!!
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 06:51:05 am
No, I'm not talking about those few seconds.

In the beginning it goes at 0.05% per second.
In the middle it goes at 0.1% per second.
Toward the end it goes at 0.3% per second.

Not specific to 5b, all versions did that!!!


Don't know - think it's algorithm dependent feature. Maybe someone else can better satisfy your curiosity.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: FloSoft on 14 Jan 2009, 07:13:51 am
Hi,

updated my cuda-app today to 5b and I get client errors: http://setiathome.berkeley.edu/result.php?resultid=1121446826

the "stock" cuda app works, but only says 2-3minutes cpu-time, but computes for 30minutes
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 14 Jan 2009, 07:22:07 am
updated my cuda-app today to 5b and I get client errors: http://setiathome.berkeley.edu/result.php?resultid=1121446826

Hmm.. Whats your gfx-driver version?

btw.
still have non-VLAR's that go stuck:
[li]14.01.2009 07:55:12 > 16no08ah.27587.13978.5.8.79_0 runtime 686s -> stuck ?
Restarted at 18.62 percent.
Flopcounter: 21582326705508.687000
Spike count:    0
Pulse count:    0
Triplet count:  0
Gaussian count: 0
called boinc_finish[/li][/list]
[li]14.01.2009 11:44:02 > 16no08ah.10277.3344.6.8.64_0 runtime 351s -> stuck ?
Restarted at 6.82 percent.
Flopcounter: 21586649651890.055000
Spike count:    7
Pulse count:    0
Triplet count:  0
Gaussian count: 0
called boinc_finish[/li][/list]
[/list][/size]Any suggestions?

[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: alollini on 14 Jan 2009, 07:57:43 am
that seems to work, I have 4 Astropulse at work and 1 seti enhanced cuda.

the 9600 GSO is warm ! (that is better than cold)

i hope the work units are done in a useful way (and not hidden errors).
my config is AMD quad 9550 + 9600 GSO driver 181.x win XP sp3 32 bits
peak power (12V power supply for sun use was 112w before cuda, now 147... 152w)
has anyone done a performance estimate of different gpu vs performance of one core ?

1122162365   395044125   14 Jan 2009 10:33:21 UTC   14 Jan 2009 12:36:15 UTC   Over   Success   Done   72.72   14.59   pending
1122162329   395044166   14 Jan 2009 10:33:21 UTC   14 Jan 2009 12:36:15 UTC   Over   Success   Done   73.05   14.59   pending
1122162326   395044109   14 Jan 2009 10:33:21 UTC   14 Jan 2009 12:36:15 UTC   Over   Success   Done   67.58   14.59   pending
1122162324   395044111   14 Jan 2009 10:33:21 UTC   14 Jan 2009 12:36:15 UTC   Over   Success   Done   71.41   14.59   pending
1122162322   395044103   14 Jan 2009 10:33:21 UTC   14 Jan 2009 12:36:15 UTC   Over   Success   Done   79.33   14.59   pending
1122162320   395044160   14 Jan 2009 10:33:21 UTC   14 Jan 2009 11:47:52 UTC   Over   Success   Done   379.63   14.62   pending
1121074263   384447812   13 Jan 2009 08:25:12 UTC   14 Jan 2009 11:47:52 UTC   Over   Success   Done   112.72   49.74   37.31
1121074214   394542483   13 Jan 2009 08:25:12 UTC   14 Jan 2009 11:56:27 UTC   Over   Success   Done   67.94   14.15   pending
1121074189   394542488   13 Jan 2009 08:25:12 UTC   14 Jan 2009 11:47:52 UTC   Over   Success   Done   68.89   14.15   pending
1121074187   394542493   13 Jan 2009 08:25:12 UTC   14 Jan 2009 11:47:52 UTC   Over   Success   Done   68.31   14.15   14.15
1121074186   394542482   13 Jan 2009 08:25:12 UTC   14 Jan 2009 11:47:52 UTC   Over   Success   Done   73.73   14.15   14.15

advice: paint your computer boxes in red, those will go faster  ;D
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 09:21:20 am
Hi,

updated my cuda-app today to 5b and I get client errors: http://setiathome.berkeley.edu/result.php?resultid=1121446826

the "stock" cuda app works, but only says 2-3minutes cpu-time, but computes for 30minutes
Update driver.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 09:26:34 am
Any suggestions?
Only one: Try to catch hang with stock 6.07 too and if it will hang - report to Q&A forum - it should be reported back to devs in this case.
And try to keep tasks too for standalone testing.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 14 Jan 2009, 10:11:50 am
Everytime a task gets stuck and my script notice that it will do a copy ;)
I've a bunch of them now ^^
Omgh , could someone attach stock app 6.0.7 please.
Ive a full cache ...
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 10:31:54 am
Everytime a task gets stuck and my script notice that it will do a copy ;)
I've a bunch of them now ^^
Omgh , could someone attach stock app 6.0.7 please.
Ive a full cache ...

Here:
http://setiathome.berkeley.edu/beta/forum_thread.php?id=1508&nowrap=true#36436
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Richard Haselgrove on 14 Jan 2009, 10:53:22 am
From a comment somewhere in the BOINC sources, his board is a Quadro FX 3700.
                                                                          Joe

That card has the same number of shaders (112) as my 9800GT (but costs six times as much - hehe).

I can't find a clock speed for it - but if I plug the shader clock for the 9800GT (1500 MHz) into the formula, I get 48 GFLOPS: seems like the right ballpark.

If they're going to set the limit 25% above a 9800GT, then GPU crunching is dead until the next generation of hardware.

I've been discussing CUDA-Z performance measurement off-line with Maik. Here is my card's report - it seems to be (still - just re-tested) just fast enough to pass SETI Beta's - and hence David Anderson's - presumed speed test: I get Beta v6.07 allocated automatically.

[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 14 Jan 2009, 10:59:36 am
I think you're looking at Integer arithmetic.  Take a look at single precision, which is what we use for the majority of computation. (You've got a little headroom)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 11:02:52 am
I like device to device memory copy speed, really :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Richard Haselgrove on 14 Jan 2009, 11:17:59 am
I think you're looking at Integer arithmetic.  Take a look at single precision, which is what we use for the majority of computation. (You've got a little headroom)

Yet Raistmer's 9600GSO was too slow. I was hoping a few ective users could post comparisons (maybe just the floats, not the whole graphic), and whether they can get work from Beta, so we could work out where the cutoff came. People are already asking questions on the SETI Main board.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 11:24:38 am
I think you're looking at Integer arithmetic.  Take a look at single precision, which is what we use for the majority of computation. (You've got a little headroom)

Yet Raistmer's 9600GSO was too slow. I was hoping a few ective users could post comparisons (maybe just the floats, not the whole graphic), and whether they can get work from Beta, so we could work out where the cutoff came. People are already asking questions on the SETI Main board.
Good idea, repost CUDA-Z link here, please... It will simplify comparison...

OOps, it shown on picture, sorry  ::)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Devaster on 14 Jan 2009, 11:26:21 am
here:


OS is WS2k8 64-bit

[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 14 Jan 2009, 11:27:22 am
hello all

i have new version APP

and this results

http://setiathome.berkeley.edu/results.php?hostid=4708858 :)

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 14 Jan 2009, 11:34:33 am
9600GT cuda-z-report (http://maik0978.homepage.t-online.de/CUDA-Z.html)

@slawek, your gfx-driver-version?
and what app-version are you using !? (Triplet count:  31 found at your results ...)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Richard Haselgrove on 14 Jan 2009, 11:35:03 am
here:

OS is WS2k8 64-bit

And can you get work from SETI at the moment, without an app_info.xml?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 11:35:18 am
My results:

CUDA-Z Report
=============
Version: 0.4.74
http://cuda-z.sourceforge.net/
OS Version: Windows x86 6.0.6001 Service Pack 1

Core Information
----------------
   Name: GeForce 9600 GSO
   Compute Capability: 1.1
   Clock Rate: 1700 MHz
   Multiprocessors: 12
   Warp Size: 32
   Regs Per Block: 8192
   Threads Per Block: 512
   Threads Dimentions: 512 x 512 x 64
   Grid Dimentions: 65535 x 65535 x 1

Memory Information
------------------
   Total Global: 384 MB
   Shared Per Block: 16 KB
   Pitch: 256 KB
   Total Constant: 64 KB
   Texture Alignment: 256
   GPU Overlap: No

Performance Information
-----------------------
Memory Copy
   Host Pinned to Device: 1574.49 MB/s
   Host Pageable to Device: 640.827 MB/s
   Device to Host Pinned: 1635.31 MB/s
   Device to Host Pageable: 598.338 MB/s
   Device to Device: 17237.8 MB/s
GPU Core Performance
   Single-precision Float: 318330 Mflop/s
   Double-precision Float: Not Supported
   32-bit Integer: 63389.4 Miop/s
   24-bit Integer: 318331 Miop/s

Generated: Wed Jan 14 19:33:38 2009


As you all know already :) My host can't get 6.07 on beta w/o app_info.xml
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 14 Jan 2009, 11:39:59 am
...
As you all know already :) My host can't get 6.07 on beta w/o app_info.xml
...
  Explain this slowly for those who weren't paying attention. (i.e. Me  :D)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Richard Haselgrove on 14 Jan 2009, 11:40:44 am

As you all know already :) My host can't get 6.07 on beta w/o app_info.xml


And yet both your float and int speeds are only about 5% lower than mine - I think we have our cut-off.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 11:43:09 am
...
As you all know already :) My host can't get 6.07 on beta w/o app_info.xml
...
  Explain this slowly for those who weren't paying attention. (i.e. Me  :D)

  ;D Ok :) Firstly, info special for you - there is new V5b build - it does VLAR autokill + decrease it's own priority class - you could try it - it should be much less laggy than all my prev builds.
And about 6.07 - see this conversations from this post:
http://setiathome.berkeley.edu/beta/forum_thread.php?id=1508&nowrap=true#36422
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 14 Jan 2009, 11:45:55 am
Runs okay on 9600GSO? , any more driver crashes?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 11:51:56 am
Runs okay on 9600GSO? , any more driver crashes?

I expect driver crashes from 6.07 stock but not from V5b build - it should autokill itself before GPU will be even touched.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Josef W. Segur on 14 Jan 2009, 11:53:50 am
I think you're looking at Integer arithmetic.  Take a look at single precision, which is what we use for the majority of computation. (You've got a little headroom)

Yet Raistmer's 9600GSO was too slow. I was hoping a few ective users could post comparisons (maybe just the floats, not the whole graphic), and whether they can get work from Beta, so we could work out where the cutoff came. People are already asking questions on the SETI Main board.

It was shown in David's message, clockRate * multiProcessorCount * 2,857 is how the Scheduler will estimate GPU speed. The clockRate and multiProcessorCount are reported by the host.
                                                                            Joe
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 11:57:39 am
Ok, 12*1700*2.85=58140 <60 000 Thanks, Joe :)

ADDON:
I hope this ridiculous measure will not lasts too long :) (even brand new gts280 or alike experience driver restart on VLAR.... )
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 14 Jan 2009, 12:00:48 pm
It was shown in David's message, clockRate * multiProcessorCount * 2,857 is how the Scheduler will estimate GPU speed. The clockRate and multiProcessorCount are reported by the host.
                                                                            Joe

Ah, so my 9600 GSO running stock , is 1350*12*2857 = 46,283,400   , No problems, can crank up the clock later if this thing proves stable at stock.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 14 Jan 2009, 12:01:35 pm
9600GT cuda-z-report (http://maik0978.homepage.t-online.de/CUDA-Z.html)

@slawek, your gfx-driver-version?
and what app-version are you using !? (Triplet count:  31 found at your results ...)


178.28

and APP version from Raistmer's_opt_package_V5
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 12:03:21 pm
9600GT cuda-z-report (http://maik0978.homepage.t-online.de/CUDA-Z.html)

@slawek, your gfx-driver-version?
and what app-version are you using !? (Triplet count:  31 found at your results ...)


178.28

and APP version from Raistmer's_opt_package_V5

Update driver to at least 180.48 and update my package to V5b

And this result recived not on my build:
http://setiathome.berkeley.edu/result.php?resultid=1121739787
My builds always have clear indication in stderr...

Moreover, I checked another last results reported by host you posted - this host doesn't use my build, chack your link and/or host.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 14 Jan 2009, 12:23:18 pm
Well Looks more promising here than before.  Will leave that overnight, then hammer the card with RthDrible/ATITool And O/C to see how much it will take without glitching (on stock fan cooler).

Jason
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 12:31:51 pm
Well Looks more promising here than before.  Will leave that overnight, then hammer the card with RthDrible/ATITool And O/C to see how much it will take without glitching (on stock fan cooler).

Jason

Hehe, it's not right season for OCing at your side it seems - today still aroung +40C ?
Here still about 0C :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 14 Jan 2009, 12:34:56 pm
Correct, But necessity is the mother of invention, and good Aussie innovation wins again.  I modified my portable swamp cooler to circulate cool water over its cooling pad, using a pond water pump I had, now the room temps are 10 degrees cooler  ;D.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 12:46:17 pm
Correct, But necessity is the mother of invention, and good Aussie innovation wins again.  I modified my portable swamp cooler to circulate cool water over its cooling pad, using a pond water pump I had, now the room temps are 10 degrees cooler  ;D.
Great :) Congatulations!
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 14 Jan 2009, 12:55:42 pm
Autokill kicked in here on first WU, and machine seems to be munching through Cuda & AP WUs quite happily.  Will leave things be for a while and see that all stays stable & not crash etc.  Great improvement.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 14 Jan 2009, 01:14:32 pm
9600GT cuda-z-report (http://maik0978.homepage.t-online.de/CUDA-Z.html)

@slawek, your gfx-driver-version?
and what app-version are you using !? (Triplet count:  31 found at your results ...)


178.28

and APP version from Raistmer's_opt_package_V5

Update driver to at least 180.48 and update my package to V5b

And this result recived not on my build:
http://setiathome.berkeley.edu/result.php?resultid=1121739787
My builds always have clear indication in stderr...

Moreover, I checked another last results reported by host you posted - this host doesn't use my build, chack your link and/or host.



i have this pack Raistmer's_opt_package_V5 and old pack is first your build

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 14 Jan 2009, 01:22:42 pm
i have this pack Raistmer's_opt_package_V5 and old pack is first your build

v5b is the lastest one ... (http://lunatics.kwsn.net/gpu-crunching/modified-seti-mb-cuda-opt-ap-package-for-full-gpu-utilize.msg12762.html#msg12762)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 01:49:23 pm

i have this pack Raistmer's_opt_package_V5 and old pack is first your build



So maybe you didn't install it correctly. All that I can say - the link you provided point on host where not my build installed.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 14 Jan 2009, 03:34:43 pm


So maybe you didn't install it correctly. All that I can say - the link you provided point on host where not my build installed.

Quote

Installation of the optimized app is simple.

Copy all files in the "Files to install" directory to your BOINC\projects\setiathome.berkeley.edu folder.
There is a file called "app_info.xml" included that will make BOINC use this app automatically.

To use it, exit and re-open the BOINC Manager if you're running it in single-user mode, or re-start the service if you're using service mode.

That's it, you're done! You can open your task manager to see that it's really the new optimized app crunching your WUs.

BOINC will tell you that it found app_info.xml and is using an "anonymous platform".

I always use this instruction.

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 03:54:52 pm
Is it your host?
http://setiathome.berkeley.edu/show_host_detail.php?hostid=4708858
If yes, try to reinstall app.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 14 Jan 2009, 04:02:20 pm
Is it your host?
http://setiathome.berkeley.edu/show_host_detail.php?hostid=4708858
If yes, try to reinstall app.


yes, and i update new your APP... and waiting for new CUDA WU
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 04:07:18 pm
Look for next lines in result's stderr:
"
Rise priority modification by Raistmer based on rev400 of SETI@home sources
Priority of worker thread rised successfully
Total GPU memory 402653184    free GPU memory 348585984
"
It's from V5 build I suppose. V5a/b will add some more.
W/o first line you run something not from me.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 14 Jan 2009, 04:15:04 pm
Look for next lines in result's stderr:
"
Rise priority modification by Raistmer based on rev400 of SETI@home sources
Priority of worker thread rised successfully
Total GPU memory 402653184    free GPU memory 348585984
"
It's from V5 build I suppose. V5a/b will add some more.
W/o first line you run something not from me.


new version drives 181.20 WHQL good to ?

I dont know why stderr is other. I have 100% its from Raistmer's_opt_package_"X" APP

X- newes version
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 14 Jan 2009, 04:27:55 pm
new version drives 181.20 WHQL good to ?
yes.
Quote
I dont know why stderr is other. I have 100% its from Raistmer's_opt_package_"X" APP

X- newes version
Well, only you can figure out why, perhaps.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Vyper on 14 Jan 2009, 05:37:40 pm
Just a report of a GTX 285 running Cuda-Z..

Kind Regards Vyper

[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 15 Jan 2009, 12:15:25 am
Hmmm, was processing fine while I was using it, then went for a nap and a cuda task froze at 0%.  In the meantime system slipped into EDF mode, but continued processing. [AP on CPU that is]. Had to restart Boinc to continue.

It wouldn't be because of monitor going to sleep would it? would be a bit silly if so.  Will see If I can reproduce the effect a bit later, but will set for monitor not to go into standby, and turn it off manually, leave it and see if I get another task freeze.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Vyper on 15 Jan 2009, 03:48:55 am
I'm afraid that it could be so simple.

I experienced same things as you and when i disabled monitor sleep alot of my cuda troubles went away. :)

//Vyper
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 15 Jan 2009, 04:10:23 am
Hmmm, was processing fine while I was using it, then went for a nap and a cuda task froze at 0%.  In the meantime system slipped into EDF mode, but continued processing. [AP on CPU that is]. Had to restart Boinc to continue.

It wouldn't be because of monitor going to sleep would it? would be a bit silly if so.  Will see If I can reproduce the effect a bit later, but will set for monitor not to go into standby, and turn it off manually, leave it and see if I get another task freeze.
give my script a chance ;)
still have freezing tasks at different runtimes ... without the script i would have same problems lik you ^^
Raistmer's v5b + script = well crunching on a 9600GT
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 15 Jan 2009, 04:32:36 am
give my script a chance ;)
still have freezing tasks at different runtimes ... without the script i would have same problems lik you ^^
Raistmer's v5b + script = well crunching on a 9600GT
Okay, point me to post with the latest version.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Devaster on 15 Jan 2009, 04:54:24 am
i am crunching good on 9600 too /with or without script .... 5b package from raistmer
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 15 Jan 2009, 04:56:44 am
Hi mimo, what is Clock speed ? pasted below?  I might need to boost mine a little, What your temps?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Devaster on 15 Jan 2009, 04:59:18 am
now see, but every day i add 25 MHz to shaders to find max....

[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 15 Jan 2009, 05:00:45 am
give my script a chance ;)
still have freezing tasks at different runtimes ... without the script i would have same problems lik you ^^
Raistmer's v5b + script = well crunching on a 9600GT
Okay, point me to post with the latest version.

[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 15 Jan 2009, 05:06:35 am
Cheers both.   I will add script first, then try underclock, then overclock.  My temps are higher because of climate here though test stable with ATI tool and RthDrible.  I am most prone to think low speed is the culprit (Nvidia choosing too big a granularity for the parallelistion), as in the past I have my 8600GT's run @100C no problems under stress (68 degrees fully loaded now).

[If underclock fixes it, then I blame temps, If Overclock Fixes it then I blame Nvidia's choice of target cards]

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Leopoldo on 15 Jan 2009, 05:32:05 am
give my script a chance ;)
still have freezing tasks at different runtimes ... without the script i would have same problems lik you ^^
Raistmer's v5b + script = well crunching on a 9600GT

Agreed. Thanks both!
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Richard Haselgrove on 15 Jan 2009, 07:19:02 am
Cheers both.   I will add script first, then try underclock, then overclock.  My temps are higher because of climate here though test stable with ATI tool and RthDrible.  I am most prone to think low speed is the culprit (Nvidia choosing too big a granularity for the parallelistion), as in the past I have my 8600GT's run @100C no problems under stress (68 degrees fully loaded now).

[If underclock fixes it, then I blame temps, If Overclock Fixes it then I blame Nvidia's choice of target cards]


I'm quite convinced it's a programming bug in the SETI application.

Have a look at my bug report (http://lunatics.kwsn.net/7-gpu-crunching/seti-cuda-mb.msg12855.html#msg12855) from last night.

Normally, my 9800GT processes a mid-range AR in 22-23 minutes, regular as clockwork. It's still running, still doing that this morning. This is with the Berkeley Beta v6.07, on official download - the card is rated as fast enough by NVidia to be entrusted with the CUDA app.

At VLAR - different story. Took 20 minutes to reach 4%, an hour to reach 35%. Then just idled (endless loop?) for three hours, didn't even checkpoint. I suspended the task, and re-started the BOINC daemon to get it to recognise the suspension - normal work restarted. No reboot, no driver crash.

Everything is in that upload - datapak, checkpoint, wisdom, client_state. See if you can find the bug before Eric K does.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 15 Jan 2009, 07:34:41 am
Nah, steering away from looking at cuda code for the time being, partly because I want to skip it and go straight to OpenCL when vendor SDKs are available.  It has plenty of heads looking at it and seems to be getting better gradually.  I am sure there are multiple issues at play, and one of them definitely seems to be speed. cranking up the 9600GSO   (along with the fan), and checking for artefacts in rthdribl and ati tool has me up to 68GFlops estimated using that formula, and things indeed seem a little better than stock 48 GFlops estimate.  Will see what I can get stably without producing too much heat or generating artefacts, and comment on what I find.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 15 Jan 2009, 08:30:57 am
Found card stability test that works for me, Will run 1 hour ati tool artefact scan.

card is 9600GSO,
Have settled for 56Gflop conservative OC ( using estimate formula of ClockRate X MultiProcessors X 2857 )
Clocks Obtained with ATi-Tool 'Find Max Mem' and 'Find max Core' separately, after reducing safety limits to keep in range.  No atrefacts yet, card nice and warm, 500frames per second in RthDrible Large, no pausing. (Then installed in RivaTuner afterward)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 15 Jan 2009, 10:17:01 am
Ok, here modded build with VLAR bug-fix incorporated.
It should do correctly all AR ranges (please,  post erros if they will be).
But be prepared to VLRT (very long run time) for VLAR tasks.
I'm not sure that it's worth to do VLAR on GPU at all with current algorithm. So, you always can reverse to V5b and leave VLARs to hosts with CPU-based opt apps.
But before doing this compare wall clock times for your own hosts  (!) I use pretty low-end GPU, your experience can (and will ;) ) be different.

ADDON: I didn't test this build much so if you experience some new errors - just revert back to V5b or switch to 6.07 and forthcoming 6.08 on beta site.


[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: perryjay on 15 Jan 2009, 10:43:02 am
Raistmer, I pulled the MB_400 out. is that ok or should I leave it in with this new build?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 15 Jan 2009, 10:49:41 am
Raistmer, I pulled the MB_400 out. is that ok or should I leave it in with this new build?
You should just install V6 package as usual.
It it will not work for you you can just reinstall V5b as usual. No additional moves needed.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: perryjay on 15 Jan 2009, 10:52:44 am
Ok, just wondered because BOINC manager bitched when it couldn't find it. Everything seems to be running ok though.

Have a good trip. See you when you get back.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 15 Jan 2009, 10:56:35 am
HI

http://c.wrzuta.pl/wi161/831bfd2400026520496f5b98/BOINC?type=i&key=j5W3rTpanB&ft=f   

and first WU done in new build http://setiathome.berkeley.edu/result.php?resultid=1123248779


now i delete old MB files ( CUDA )....
i waiting for next CUDA WU download meybe stderr now be correct....:/

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 15 Jan 2009, 11:19:05 am
and first WU done in new build http://setiathome.berkeley.edu/result.php?resultid=1123248779
That is still no result from a 'Raistmer-built'  ::)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 15 Jan 2009, 11:24:08 am
Im Stupid ..... i have 2 dir in Documents and settings... and all time computing on first Raistmer's_opt_package and change old BOINC dir...... :o

and all time no errors .. only 1 ... :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 15 Jan 2009, 11:45:55 am
Im Stupid ..... i have 2 dir in Documents and settings... and all time computing on first Raistmer's_opt_package and change old BOINC dir...... :o

and all time no errors .. only 1 ... :)
LoL. So, you helped yourself, as it was supposed :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 15 Jan 2009, 12:10:41 pm
since i have running MB_rev400mod_VLARkill_idle_CUDA.exe ... no stucking wu's / no BSOD's / no crash's .... and a full cache ^^
hmm ...dont want to switch to v6. (never change a running system, you know?  :-\)
now i have to wait around 24h that my chache is crunched to install v6 to test my underpowered 9600GT  ;D
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 15 Jan 2009, 12:24:20 pm
http://setiathome.berkeley.edu/result.php?resultid=1123310667

not working.. error :(
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 15 Jan 2009, 12:30:08 pm
http://setiathome.berkeley.edu/result.php?resultid=1123310667

not working.. error :(
Update driver.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 15 Jan 2009, 01:23:22 pm
See you all 20 January I hope, good luck with crunching!
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: perryjay on 15 Jan 2009, 07:10:37 pm
Ok, just an update. I finished my first VLAR (0.123395) using Raistmer's V-6 package. It started at 13:56 and ended at 16:26. 2.5 hours almost exactly. The tasks page shows 346.90 seconds.My wingman showed 7,493.92 seconds for his. I claimed 80.53 credits he claimed 69.08. Both validated, credit issued 69.08 still not bad for only 2.5 hours.

For those that haven't been paying attention, I'm running a Celeron C2D with a GeForce 8500GT graphics card
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Zoran Kirsic on 16 Jan 2009, 04:23:20 am
Secondary link for Raistmer_s_opt_package_V6.rar (http://rapidshare.de/files/41738955/Raistmer_s_opt_package_V6.rar.html/)

For the people who had problems with activation mail.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: efmer (fred) on 16 Jan 2009, 12:23:44 pm
This version looks promising. Did not see any driver crashes for some time. The previous version did crash twice. The program did but not the driver.
http://setiathome.berkeley.edu/result.php?resultid=1121391047 http://setiathome.berkeley.edu/result.php?resultid=1121077281
http://setiathome.berkeley.edu/result.php?resultid=1121077211 http://setiathome.berkeley.edu/result.php?resultid=1121077204
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 16 Jan 2009, 01:05:14 pm
i cant download any CUDA WU now:(

And i cant testing new build
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Devaster on 16 Jan 2009, 02:21:49 pm
yes, only app crashes ...
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 16 Jan 2009, 03:38:45 pm
new versions APP better performance to ? or only stability etc ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 16 Jan 2009, 09:33:16 pm
was testing v6 on my 9600GT:
 - AR: 0.11437
 - runtime: 2086s
 - no crash/BSOD/stuck
 - referece: non LAR/VLAR needs around 1500-1600 sec
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 17 Jan 2009, 08:18:23 am
Just to throw another observation into the mix, after munching through a few WUs now.  I seem to be claiming ~30% more credits than AKv8 wingmen. So far these have all been denied.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Maik on 17 Jan 2009, 08:29:32 am
Do you mean that?

Cuda owned by CPU ?!? (http://setiathome.berkeley.edu/forum_thread.php?id=51232)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 17 Jan 2009, 08:31:37 am
Yes, Maybe I'll have to find out if the flop [Actually fpop] counting standard is modified and build new AK_v8 builds...  :P

Posted in that thread:
Quote
It is 'counting' more flops [Well more correctly fpops anyway] for the same processing. This would not be tolerated in a third party app. I will have to determine if this is an intentional modification to the stock op count regime, so build new AK_v8 builds to match, if it continues.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: efmer (fred) on 17 Jan 2009, 08:58:02 am
Just to throw another observation into the mix, after munching through a few WUs now.  I seem to be claiming ~30% more credits than AKv8 wingmen. So far these have all been denied.
Maybe machine dependent I claim 13.83 the other non CUDA claims 16.58 so I claim a lot less....
But others are ok I claim 14.45 the other non CUDA claims 14.73.
I claim 59.82 the other 43.79
This one is interesting http://setiathome.berkeley.edu/workunit.php?wuid=394719892
It is a regular, a CUDA Raistmer and a stock CUDA. The Raistmer and Stock CUDA both claim more than a regular.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 17 Jan 2009, 09:00:40 am
Yeah, If the count is borked it likely will vary by angle range in discrepancy.  My concern is that it gives, for some people, a false indication of performance. [On the other hand, if it is on purpose, then it becomes the new standard, to which optimised apps must then comply]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Josef W. Segur on 17 Jan 2009, 07:02:43 pm
Yeah, If the count is borked it likely will vary by angle range in discrepancy.  My concern is that it gives, for some people, a false indication of performance. [On the other hand, if it is on purpose, then it becomes the new standard, to which optimised apps must then comply]

The undercount for triplets is recognized in a comment and is due to the CPU not knowing how many samples were above the triplet threshold in a PoT array. The overcount for gaussians is more complex, there's an overcount for the equivalent of getFixedPoT() which could easily be fixed, then difficulty in estimating how often the two data-dependent early outs are taken. A bit of statistical analysis on sufficient results should allow reasonable adjustments, though not exact. Probably the right target would be a small overclaim so variations seldom cause a lower claim than CPU and reduce the granted amount.
                                                                          Joe
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Radiohead on 18 Jan 2009, 05:47:11 am
What about ver 6.08 ?

http://setiweb.ssl.berkeley.edu/beta/forum_thread.php?id=1512
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 18 Jan 2009, 07:02:58 am
ok working new APP on my GPU http://setiathome.berkeley.edu/result.php?resultid=1125694703
Time to incrased " Computer is connected to the Internet about every "  :)

1) oo  :o i change cc_config and now computing 3 AP ins. 2  ( i have 2 core + GPU ) ?

2) now i see computing 2 AP and CUDA ) the same time... :o  http://www.wrzuta.pl/obraz/ue2eHIWu7I/b2
 
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 18 Jan 2009, 01:47:50 pm
Some WU doing my GPU ~10 min and some 50-59 min, why so long? : ::) CPU time is ~80-160 sec

This long time (50-59min ) is VLAR...hmm i think, but why to slow ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Richard Haselgrove on 18 Jan 2009, 05:36:05 pm
Some WU doing my GPU ~10 min and some 50-59 min, why so long?
The true VLARs do indeed still take vastly longer than all the others (91 minutes, versus 22-23 minutes for mid range on my 9800GT - haven't had enough shorties to get an idea of the minimum).

The CUDA app is still acknowledged as a "work in progress" by the Berkeley developers: this is the first build where they actually got it to run right through to the end, at any speed. They are aware of the abysmal performance, but we don't know how long it'll take to come up with the next version.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 20 Jan 2009, 12:40:48 am
BTW, if you feel current VLAR processing times too long for production app on main you could continue to use my V5b that will abotr VLARs and release task to be processed by another host possibly with CPU opt app (that will do processing much faster and effective). This is not perfect but some way of load balansing between CUDA/CPU in current conditions.

(I need some time to synch with threads now so sorry if this was proposed already).
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 20 Jan 2009, 05:15:39 pm
Hello ,

And this is my computing action 2 days http://setiathome.berkeley.edu/results.php?hostid=4708858

:)

650 RAC now :)

CUDA :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 21 Jan 2009, 12:05:47 pm
No errors from this build  , working good :)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Jan 2009, 01:25:16 pm
Fine. There is some validation problems remain, but probably they are pretty rare cases. Will see if beta testing will catch them again.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Jan 2009, 06:13:10 pm
Here is new package version.
It contains the same CUDA part as before (equivalent to 6.08 stock with some mods) and latast opt AP SSE3 version (r103) )
This package will provide suboptimal performance in CUDA part versus V5b (because VLAR CUDA processing very inefficient now and V5b just abort VLAR tasks) but provides performance increase in CPU part because of opt AP rev103 increased performance.


[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 21 Jan 2009, 06:50:14 pm
And one more version.
Now changes in CUDA part of package. This build based on 6.08 sourses but with VLAR auto-kill mod enabled. It should provide best overall performance at cost of reissuing VLAR task to another host. NOT for using on SETI beta project.

[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 21 Jan 2009, 08:47:59 pm
i have alot WU waiting  and ready for start and if i change new build i lost my cache WU ? i have 10 days in " Computer is connected to the Internet about every"
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 22 Jan 2009, 02:38:52 am
i have alot WU waiting  and ready for start and if i change new build i lost my cache WU ? i have 10 days in " Computer is connected to the Internet about every"
If you do right installation you should not lost any downloaded task.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 22 Jan 2009, 01:14:59 pm
All tasks OK :) computing now

no errors
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Devaster on 23 Jan 2009, 03:29:07 am
if some one want here is cufft and cudart  dlls from last CUDA 2.1 final :

stop boinc unpack and replace in boinc data dirs the old  dlls

[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: sunu on 23 Jan 2009, 10:19:37 am
Is there an info page that explains what exactly

<avg_ncpus>0.0400</avg_ncpus>
<max_ncpus>0.0400</max_ncpus>

or any other options in app_info.xml do?

I'm trying to force 3+1 in a quad and whatever I do in app_info.xml (eg 1.0000 instead of 0.0400) has no effect.

Maybe it is a boinc thing in linux?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: The Grinch on 23 Jan 2009, 10:48:56 am
I don't know if this will helps you

http://boinc.berkeley.edu/wiki/Anonymous_platform

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: sunu on 23 Jan 2009, 11:04:04 am
I don't know if this will helps you

http://boinc.berkeley.edu/wiki/Anonymous_platform


Thanks, that was what I was looking for. Unfortunately I think I've already used

<avg_ncpus>3.0000</avg_ncpus>
<max_ncpus>3.0000</max_ncpus>

for AP with not the desired effect. Maybe I'll try again.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 23 Jan 2009, 12:04:21 pm
use <ncpus>Ncores+1</ncpus> in cc_config.xml instead. IT works much better than fiddling with CPU shares.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: sunu on 23 Jan 2009, 12:09:42 pm
Raistmer here's the thing: I want to force 3+1 in a quad (not 4+1).

When I use <ncpus>4</ncpus> in cc_config.xml, boinc runs 4+1 when it is in normal mode and 3+1 in high priority mode.
When I use <ncpus>3</ncpus>, boinc runs 3+1 normal and 2+1 high priority.

All these are in boinc linux. Maybe it is a bug/feature?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 23 Jan 2009, 12:17:00 pm
then try 2. If you don't care about idle cores it should work.  :P
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 23 Jan 2009, 12:20:39 pm
Or you can try this: change the amount of cpus for the GPU app in the app_info file from 0.04, to 1  (or even 2  if needed)  then boinc can't allocate those cores to something else.  You might also want to check the flops setting in there is higher than for the CPU app (You can add #cpus (i.e. 1) and flops to CPU app section if it isn't there), so GPU app gets allocated first and hogs the required number of cores (x1)  prior to CPU apps being allocated(x remaining 3).

Jason
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: sunu on 23 Jan 2009, 12:28:58 pm
from 0.04, to 1  (or even 2  if needed)
Jason

I've never tried 2. I was stopping to 1  :)

New ideas, thanks!
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 23 Jan 2009, 12:31:05 pm
You could even try 1.04, if 2 cuts back too much,  that may play nicer with the newer CPU juggling with EDF mode.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: sunu on 23 Jan 2009, 12:35:34 pm
Thanks Jason.

Regarding your previous post, I didn't know the trick with flops. Can I put just an arbitrary number?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 23 Jan 2009, 12:44:03 pm
Yep, I gather it's just relative, but I would look how much GFLops in BoincView CPU app acheives, put that value (in Flops) for the CPU, then multiply by 2-5 (as appropriate for the card speed against CPU) for the GPU version.

Jason
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: sunu on 23 Jan 2009, 12:52:05 pm
I can't use boincview (linux). I've found some flop numbers in client_state.xml and I'll play with them.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 24 Jan 2009, 07:24:00 am
if some one want here is cufft and cudart  dlls from last CUDA 2.1 final :

stop boinc unpack and replace in boinc data dirs the old  dlls

Any imp. computing  ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: sunu on 24 Jan 2009, 09:36:58 am
Just a FYI. Whatever I tried, flops, avg_ncpus, maxncpus in app_info.xml in order to force 3+1 in a quad didn't have any effect. The only thing that seems to work is

<avg_ncpus>1.3400</avg_ncpus>
<max_ncpus>1.3400</max_ncpus>

in AP section of app_info.xml. But that possibly means that If there is no MB work, only 3 cores will be used. Whatever I did in MB section seemed irrelevant.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Jason G on 24 Jan 2009, 09:46:18 am
...
in AP section of app_info.xml. Whatever I did in MB section seemed irrelevant.

Ahhh, of course, I think in 6.4.5 or higher they might have made it ignore the Cuda CPU component! LoL. Madness  ;D
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: ceciltseng on 29 Jan 2009, 01:00:08 am
hi all ,
AP5.0 SSE4.1  + CUDA MB possible to use now?   ( 64 bit version is wonderful )
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 29 Jan 2009, 03:55:00 am
hi all ,
AP5.0 SSE4.1  + CUDA MB possible to use now?   ( 64 bit version is wonderful )
There is no SSE4.1 AP at all, did you mean AK_v8 SSE4.1 ? I found not improvement in speed for SSE4.1 over SSSE3x version. In multicore system SSSE3 performs better actually (but difference pretty slow anyway). So I don't plan to do separate SSE4.1 version. Also, there is no AP x64 build for now.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: koschi on 29 Jan 2009, 03:18:03 pm
Just a FYI. Whatever I tried, flops, avg_ncpus, maxncpus in app_info.xml in order to force 3+1 in a quad didn't have any effect. The only thing that seems to work is

<avg_ncpus>1.3400</avg_ncpus>
<max_ncpus>1.3400</max_ncpus>

in AP section of app_info.xml. But that possibly means that If there is no MB work, only 3 cores will be used. Whatever I did in MB section seemed irrelevant.

Are you running SETI only?
I'm running some more projects and would like to run a 3+1 configuration, but whatever I set avg_ncpus/max_ncpus to (tried 0.04, 1, 1.5, 2), its doesn't work. When set to 2, no CUDA process is started at all. With all values >= 1 the given nice value is 19, not 10...

Is there detailed documentation of the app_info stuff somewhere that explains all tags?

I guess I have to use a cc_config.xml with ncpus 3 if there is no solution via app_info, right?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: sunu on 29 Jan 2009, 04:56:15 pm
I'm running astropulse+multibeam. Because of linux client's full core use we have to use 3+1. So I put those values in the AP section of app_info.xml. That did the trick. Whatever was in the MB section didn't seem to matter. Playing with cc_config.xml didn't have the desired effect. Please see above another post from me (reply #242) about cc_config.xml.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: koschi on 29 Jan 2009, 08:14:06 pm
This machine where I'm trying to set up MB CUDA and AP is running POEM, LHC and GPUGRID as well.
With ncpus=3 in cc_config I get the desired result (as long as I have MB WUs), but thats far away from being perfect, if there is no GPUGRID task (which happens from time to time) and it doesn't get work from SETI, one core is idle...

I've read your post regarding the AP settings (found it via google, hehe), but I'm a little confused, I doubt that it works when more projects are attached... Right now I don't get any AP WUs, only had some MB units earlier today, so I cant say if it runs MB and AP in parallel or not yet...

edit:

My goal is to run CPU units (POEM, SETI AP, LHC) and GPU units (SETI & GPUGRID) at the same time with the best utilization (better more then less, the scheduler can stand it...)
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: sunu on 29 Jan 2009, 08:54:09 pm
As it is right now I think the problem lies with boinc client which cannot handle very well cpu- and gpu-bound tasks simultaneously.

Your situation is more complicated with all those projects that you want to run. I don't know whether the other projects use an app_info.xml file too, but if they do, you can put these two rows
<avg_ncpus>1.3400</avg_ncpus>
<max_ncpus>1.3400</max_ncpus>
on their respective CPU apps sections.

That way you will always run 3+1 except when there is no GPU task where you'll have 1 core idle. Then you can put ncpus 5 to cc_config.xml for some time until you get some GPU tasks to crunch again.

Regarding more is better than 1 core empty I think it is wrong. These optimized apps have special optimization for cache usage and when you have two apps, especially from different projects, sharing a core you will have degraded performance overall with all these cache misses or whatever. On the other hand you will always have some seti cuda or GPUGRID tasks to crunch more or less. On those rare circumstances that you will not, you can use the cc_config.xml trick.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: koschi on 29 Jan 2009, 10:12:58 pm
I changed the AP ncpus from 0.0400 to 1.34 in SETIs app_info.xml...
But as long as there is no AP WU running this is not effective I think. My app_info should be OK, but I didn't get a single new WU since this afternoon. Had some MBs this morning (with the new attached app_info.xml) but since then nothing...

So unfortunately I don't have a single SETI unit right now, all clients since 6.3.21 kind of sucks, it doesn't get better :-( In almost every client there is some work fetch problem...
Regarding sharing cores, the problem is we don't know what the CUDA linux app is actually doing while consuming one full core for nothing... When running GPUGRID some months ago, when the app was still consuming half a core for polling the GPU app, there was no performance problem, when running 4+1 rather then 3+1 under Linux... Right now it runs flawlessly in 2+1 or 4+1 mode on my hosts without any difference to 3+1 or 1+1.

edit:

now I got it, sorry, reading an understandig...
My other projects don't have app_infos, so there is nothing to edit :-(




[attachment deleted by admin]
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: sunu on 29 Jan 2009, 10:58:46 pm
Right now there are zero AP workunits. We wait for new data from Arecibo.

Since the other projects don't have an app_info.xml, you'll have to play with cc_config.xml.

For SETI you can play with app_info.xml. The file you've attached will run 4+1 AP+MB without any modification to cc_config.xml.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 30 Jan 2009, 07:21:30 am
Regarding more is better than 1 core empty I think it is wrong. These optimized apps have special optimization for cache usage and when you have two apps, especially from different projects, sharing a core you will have degraded performance overall with all these cache misses or whatever. On the other hand you will always have some seti cuda or GPUGRID tasks to crunch more or less. On those rare circumstances that you will not, you can use the cc_config.xml trick.

Hm, you could give some more trust to my words ;) I'm  aware about AK v8 optimizations and participate in AK v8 benchmarking pretty long time already. And being one of direct developers of AP opt app too, btw, surely aware about its interaction with cache.Core switching will give much less performance degradation then just leaving full core idle. I don't know what is done in Linux build that it requires 100% CPU usage (and even not sure that it's so indeed), but this thread about Windows app, moreover, about modification that allows to use whole cores. And you give suggestion to leave one core free ?? If you need free core, don't use this mod at first, this mod designed to use ALL cores AND GPU.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: chelski on 01 Feb 2009, 10:59:11 am
Thanks Raistmer for the wonderful V7 app.  Have been running it fine for about 1 week now on a "weakling" 9600GSO and apparently its thoughput is about 1.5x my Core2Duo E6550 running AK optimised aps.
Just noticed there are some glitches when the app completes one unit and change to another one, on some occasions, ~0.8x per day will see that task stuck there for hours at 0:00:01 CPU times (in my case the CUDA apps neeeds about 30 secs CPU time to prepare the WU for CUDA) for hours and only way to restart it is to restart the client, at which point that WU will normally work out fine.  Tried limiting CPU usage to 95% ( to leave a few % for CUDA WU switching) but somehow it doesn't work.  It is just a small inconvenience that some babysitting for BOINC is required (i generally check 2-3 times a day to restart if required) but any suggestion for other workaround would be great.  And again thanks for the great effort on the CUDA app!

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 01 Feb 2009, 11:04:51 am
Thanks Raistmer for the wonderful V7 app.  Have been running it fine for about 1 week now on a "weakling" 9600GSO and apparently its thoughput is about 1.5x my Core2Duo E6550 running AK optimised aps.
Just noticed there are some glitches when the app completes one unit and change to another one, on some occasions, ~0.8x per day will see that task stuck there for hours at 0:00:01 CPU times (in my case the CUDA apps neeeds about 30 secs CPU time to prepare the WU for CUDA) for hours and only way to restart it is to restart the client, at which point that WU will normally work out fine.  Tried limiting CPU usage to 95% ( to leave a few % for CUDA WU switching) but somehow it doesn't work.  It is just a small inconvenience that some babysitting for BOINC is required (i generally check 2-3 times a day to restart if required) but any suggestion for other workaround would be great.  And again thanks for the great effort on the CUDA app!



Sorry, can't suggest anything but to use some monitoring tool like Maik's script or smth like that to restart hung tasks. Unfortunately (in this context) my own 9600GSO seems doesn't have such problems so I can't reproduce this behavior.... If you can catch hang tasks and attach them here I could try to re-run it on my host in standalone mode - maybe I meet that hang too? ...
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: seeker98tw on 02 Feb 2009, 12:53:04 am
Hi
computer:
AMD Phenom(tm) 9650 Quad-Core Processor
and 9600GT Graph-CARD

Can I use 'package_V7' ? Is it working ?

thanks
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 02 Feb 2009, 02:15:46 am
Hi
computer:
AMD Phenom(tm) 9650 Quad-Core Processor
and 9600GT Graph-CARD

Can I use 'package_V7' ? Is it working ?

thanks
Yes, it should work. But you could use V8 too (you have single GPU card). It will work faster for AMD probably.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: seeker98tw on 02 Feb 2009, 02:44:12 am
Thanks

I will test 'AK V8 + CUDA MB team work mod' at night.

I uses your optimized Apps in this month, it compute quickly. very great
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Slawek on 05 Feb 2009, 08:09:35 am
Hi,

Any 64 bit version CUDA ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 05 Feb 2009, 08:19:02 am
Hi,

Any 64 bit version CUDA ?
current x86 will go on x64 OS too.
Don't expect any noticeable speed improve from x64 CUDA cause most computations performs on GPU, not on CPU so can't feel bitness of CPU part.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: alexanderhuzar on 07 Feb 2009, 08:00:26 pm
My 9600GSO working hard too :) 58C (~44 in idle state). It has very big cooler  (Asus card) :)
Thinking is it time to try some overclocking for GPU?.... ;)

A word of advice: please run ATI tool to "SCAN FOR ARTIFACTS" for at least 8 hours if you are going to use the GPU or you run the risk of sending bad results if you overclock or don't maintain/clean/check temps on your computer regularly.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: elec999 on 14 Feb 2009, 10:59:21 pm
I have a quad core Q6600 + 9800 GT, the gpu is running beutiful. But I cannot get my cpu cores to do any work. They are sleeping.
My cc file
- <cc_config>
- <options>
  <ncpus>5</ncpus>
  </options>
  </cc_config>
Can someone suggest please. The Q6600 are also excellent for seti.
Message when Boinc Runs
   Host   Project   Date   Message
   b3   ---   2/15/2009 00:46:43   Preferences limit # CPUs to 5
   b3   ---   2/15/2009 00:46:43   Preferences limit disk usage to 30.62GB
   b3   ---   2/15/2009 00:46:43   Preferences limit memory usage when idle to 1637.17MB
   b3   ---   2/15/2009 00:46:43   Preferences limit memory usage when active to 1023.23MB
   b3   ---   2/15/2009 00:46:43   Reading preferences override file
   b3   ---   2/15/2009 00:46:43   General prefs: no separate prefs for home; using your defaults
   b3   ---   2/15/2009 00:46:43   Computer location: home
   b3   ---   2/15/2009 00:46:43   General prefs: from SETI@home (last modified 02-Feb-2009 16:06:57)
   b3   SETI@home   2/15/2009 00:46:43   URL: http://setiathome.berkeley.edu/; Computer ID: 4737399; location: home; project prefs: default
   b3   ---   2/15/2009 00:46:43   CUDA device: GeForce 9800 GT (1024MB, est. 60GFLOPS)
   b3   ---   2/15/2009 00:46:43   Not using a proxy
   b3   ---   2/15/2009 00:46:43   Local time is UTC -5 hours
   b3   ---   2/15/2009 00:46:43   Disk: 38.28 GB total, 30.88 GB free
   b3   ---   2/15/2009 00:46:43   Memory: 2.00 GB physical, 3.35 GB virtual
   b3   ---   2/15/2009 00:46:43   OS: Microsoft Windows XP: Professional x86 Editon, Service Pack 3, (05.01.2600.00)
   b3   ---   2/15/2009 00:46:43   Processor features: fpu tsc pae nx sse sse2 mmx
   b3   ---   2/15/2009 00:46:43   Processor: 4 GenuineIntel Intel(R) Core(TM)2 Quad CPU    Q6600  @ 2.40GHz [x86 Family 6 Model 15 Stepping 11]
   b3   SETI@home   2/15/2009 00:46:42   Found app_info.xml; using anonymous platform
   b3   ---   2/15/2009 00:46:42   Running under account Administrator
   b3   ---   2/15/2009 00:46:42   Data directory: C:\Documents and Settings\All Users\Application Data\BOINC
   b3   ---   2/15/2009 00:46:42   Libraries: libcurl/7.19.2 OpenSSL/0.9.8i zlib/1.2.3
   b3   ---   2/15/2009 00:46:42   log flags: task, file_xfer, sched_ops
   b3   ---   2/15/2009 00:46:42   Starting BOINC client version 6.6.4 for windows_intelx86

Thank you
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 15 Feb 2009, 03:05:55 am
What applications do you use for crunching? Stock? Optimized ?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: elec999 on 17 Feb 2009, 09:41:48 pm
What applications do you use for crunching? Stock? Optimized ?
The Optimized cuda package found in this thread.
Thank you
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: bloort on 18 Feb 2009, 01:15:44 am
i'm experiencing the same issue as elec999, with 2 similar configurations, and being new to this, i suspect user error on setup.  So here's what i did.  Installed boink 6.4.5 (widgets version 2.8.7) fresh on machine one, and repair install on machine 2.  then unpacked Raistmer's_opt_package_V7.rar with overwrite  into project dir (C:\Documents and Settings\All Users\Application Data\BOINC\projects\setiathome.berkeley.edu\).  Then created a new file C:\Documents and Settings\All Users\Application Data\BOINC\cc_config.xml    with the line <ncpus>3</ncpus> for my dual core intel cpu with single nVidia 8800 GTX, and <ncpus>5</ncpus> for my other machine which is Quad core intel with dual nvidia 8800 GTX.  I fired up Boink and MAAAAAN those Work Units just flew...i figure about 18 minutes each...or less.  All three cards were happily crunching away at blistering speeds.  AMAZING speeds.  CPUs are asleep.  

Next, I scoured nVidia's site for a monitoring tool to watch the temps.  My single card setup runs about 82Celcius and i'm assuming the fan is idling at 60%.  (nvidia states that normal op specs for the 8800 GTX are up to 85C, and they've run them as high as 145C before the magic smoke left).  Just a note, that the default alarm level in the nvidia software is 127C)  This box is vented, but not as well as the other machine.  The dual card machine saw the top card running a bit hot and reaching the upper limit or 85, so i bumped the fan on the top card from 60% to 80% and top and bottom now run about 80C and 78C respectively.

Now one machine has the older nVidia system monitor with the grey panels, and the other machine has the newer with the slick transparent ones.  I've had the cuda processing seemingly grind to a halt overnight with the nvidia monitors left running.  Restarting boink brought it back to blistering speed, and with the monitoring software shut down, there doesn't appear to be any slowdowns yet.

now with my cpus essentially not being utilized, this works out Ok for me... i've even rendered video compliations through the cpu, while 2 gpu's cruch seti..and the results were perfect.

From my observation, GPU processing appears (i'm just estimating) to be about 16X faster than what the CPU can manage.  so at that score, it doesn't much matter other than in principle if the cpu's stay asleep...those units are flying by on GPUs alone.  but if i've done something in my setup that is wrong, and may be a common mistake..maybe here is where it can be corrected and clarified.

My hardware is a strange brew middle of the road pizza type from Dell.  They have  intel CPU, w nvidia chipset...which i find to be strange combo anyhow...so i'm wondering if the sleepy CPUs have something to do with that.  SLI works yes, but hyperthreading is not an available option in the bios although it's a feature in both CPUs, yet i see 4 and 2 cpus on each machine respectively, which i find confusing.

Well, i think that's about my story to date.  I type quite fast, yet 3 units have been processed by the GPUS while typing...and when i point the towers exhaust to vent under my desk...it keeps my toes warm.!!  Nice bonus for a northern canadian in mid winter!

Oh yes, a couple of observations which are just that, I don't know if they mean anything.

1. On the dual core single GPU, the boink manager progress shows a bar.  In the quad dual GPU it's numeric only.
2. not knowing what the Raistmer's_opt_package_V7_VLAR_kill was, I attempted to try that in the same methad as above to see if that would wake up the CPUs, and boink generated results as computational errors.  Actually I just overwrote the Raistmer's_opt_package_V7.  No doubt, my bad. :)

My machines are dell XPS 710 quad nvidia chipset and 2 x 8800 GTX , and the earlier Dell dimension 9150 dual core intel chipset and single 8800 GTX.  driver 1.81.22. no fancy overclocking for me.  just the fans. lol

So hopefully that will help, and if anyone needs more info i'll be lurking.  ty Hippy Dave.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 18 Feb 2009, 03:03:05 am
What applications do you use for crunching? Stock? Optimized ?
The Optimized cuda package found in this thread.
Thank you

Link on your host?
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 18 Feb 2009, 03:10:30 am
i'm experiencing the same issue as elec999, with 2 similar configurations, and being new to this, i suspect user error on setup.  So here's what i did.  Installed boink 6.4.5 (widgets version 2.8.7) fresh on machine one, and repair install on machine 2.  then unpacked Raistmer's_opt_package_V7.rar

With V7 pack your CPU will capable doing only AP SETI. And now servers send very small amount of AP tasks. Maybe it's worth to connect to another CPU-oriented project like Einstein@home.
On host with single GPU V8a should be used, not V7 one.

Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: bloort on 18 Feb 2009, 07:26:24 am
Yup...hold the press...there it is : 2 cudas and one astropulse running at the same time.  Absolsolutely fantastic.  I'm fascinated.  Cudos Raistmer, cudos and cudas!!
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Raistmer on 18 Feb 2009, 08:16:29 am
Yup...hold the press...there it is : 2 cudas and one astropulse running at the same time.  Absolsolutely fantastic.  I'm fascinated.  Cudos Raistmer, cudos and cudas!!
Actually, for dual core + dual GPU host it should be 2 CUDA MB + 2 AP tasks if all was installed correctly ;)

You have quad with 2 GPUs so it should be 4 AP tasks and 3 CUDA MB tasks.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: bloort on 18 Feb 2009, 07:46:41 pm
Yes I would assume that would be the case, but there was only one Astropulse unti available on my list.  I was stunned to find on the quad core that the unit was completed in 10 hours cpu time.  normally a unti would take days.  I'm still finding that cuda processing speed decays overnight.  After returning to the consoles after 12 hours, dual gpu processing which normally ticks at 0.100 or more, has slowed to 0.010..and for the first time the single gpu machine had stopped and was "waiting to start'.  Restarting boink brought everything up to speed again.  Both boinks are set to 'run always.  not sure why it's doing that, but i'll roll back the single gpu machine to 6.4.5 and report what happens.

Quad CPU and dual GPU should present 4 Astropulse and 2 cudas correct?  (just wondering if you made a typo)


Dual gpus on nvidia chipset, single gpu on intel chipset.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Stoffe on 19 Feb 2009, 01:54:25 am
Running the V7 app. my CPU-cores won't do any work, like Raistmer stated, but my problem is that my 9800gx2 will only run one core (and Boinc also reporting only 1 GPU core).
After reading Bloort's post about GPU-speed I get even more confused. It takes about 20 minutes for my GX2 to run one WU. Bloort talking about 8 minutes, on a 8800 GTX (witch is an older card than my 9800gx2).

What have I done wrong?
Using 181.22 drivers, SLI-mode disabled, Vista 32bit, Q6600.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: bloort on 19 Feb 2009, 05:10:39 am
Typo - Bloort Bad.  somewhere around 18 minutes give or take.  original post edited.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Stoffe on 19 Feb 2009, 05:21:12 am
Typo - Bloort Bad.  somewhere around 18 minutes give or take.  original post edited.
Thanks! You saved me further worries.  ;D
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Stoffe on 19 Feb 2009, 05:15:28 pm
Clocked my WUs now. Running 4 an hour on each GPU-core. I think I should be satisfied with that...  8)
Now, I'm only hoping for a miracle to make the AP WUs finish in a few hours a piece in stead of the current, close to ten.  :(
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: bloort on 21 Feb 2009, 09:31:55 am
you know, on the quad, i finally had a couple of AP units going at the same time.  they were finished in 8.  that was cool.  What kind of temps does your GPU when running.  Mine is doind 82, and nvidia monitor says it's managing the fan...seems to be.  And as far as I know 85 is upper limit...so it's within spec right?  temp keeps my toes warm lately, but don't want too much.

I've experienced some slowdowns (or possible reversions back onto the CPU) a few times that i've noticed.  I think for now i'm going to blame that on microsoft remote desktop access interfering with the GPU and me being lazy and not grabbing the other keyboard off to the side.
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: efmer (fred) on 21 Feb 2009, 09:59:33 am
you know, on the quad, i finally had a couple of AP units going at the same time.  they were finished in 8.  that was cool.  What kind of temps does your GPU when running.  Mine is doind 82, and nvidia monitor says it's managing the fan...seems to be.  And as far as I know 85 is upper limit...so it's within spec right?  temp keeps my toes warm lately, but don't want too much.

I've experienced some slowdowns (or possible reversions back onto the CPU) a few times that i've noticed.  I think for now i'm going to blame that on microsoft remote desktop access interfering with the GPU and me being lazy and not grabbing the other keyboard off to the side.
Don't worry 80-90 should be average. http://www.tomshardware.co.uk/forum/page-253599_15_0.html
My machine (8800GTX) is well cooled and goes up to 72C (65-72) running SETI. When I query the card with my program, the max temp is 127. But that is probably the shutdown temperature.
 
Title: Re: Modified SETI MB CUDA + opt AP package for full GPU utilization
Post by: Stoffe on 21 Feb 2009, 10:23:53 am
you know, on the quad, i finally had a couple of AP units going at the same time.  they were finished in 8.  that was cool.  What kind of temps does your GPU when running.  Mine is doind 82, and nvidia monitor says it's managing the fan...seems to be.  And as far as I know 85 is upper limit...so it's within spec right?  temp keeps my toes warm lately, but don't want too much.

I've experienced some slowdowns (or possible reversions back onto the CPU) a few times that i've noticed.  I think for now i'm going to blame that on microsoft remote desktop access interfering with the GPU and me being lazy and not grabbing the other keyboard off to the side.
My GPUs around 80-85c. I do not see that as a problem. Gaming Crysis Warhead will make it top out about 105c...