Seti@Home optimized science apps and information
Optimized Seti@Home apps => Windows => GPU crunching => Topic started by: Sutaru Tsureku on 18 Jul 2009, 12:06:53 pm
-
I used the Installer (unchecked MB/AP, checked only CUDA) and installed to a temporary - not to the setiathome.berkeley.edu - folder.
Then I took the CUDA_V12_app and the 3 .dll's .
And on my GPU cruncher work this app.. ;)
4 x OCed GTX260-216
http://setiathome.berkeley.edu/show_host_detail.php?hostid=4789793
Hmm.. it's look like the CPU support time is like before with the latest CUDA_V11_app.
Current I can't look to the real wall clock time, because the tasks overview of the PCs are disabled at SETI@home.
In the description I read:
Build with smaller pulse find limit (2048), hoped to exhibit less issues on most nVidia Cards.
What does this mean?
Which GPUs are meant?
GTX260-216 ?
The CUDA_V12_app should be faster than the CUDA_V11_app ?
The CPU preparation time of the CUDA WUs are the same.
So the GPU calculation time is faster?
Thanks! :)
-
...
In the description I read:
Build with smaller pulse find limit (2048), hoped to exhibit less issues on most nVidia Cards.
What does this mean?
Which GPUs are meant?
GTX260-216 ?
It turmed out to be all GPU cards tested so far. If you didn't get any display problems & laginess before, then it doesn't apply to you, no change willl be obvious. [Edit: Of course if you have no display plugged into the card, you wouldn't see any display problems either ;D]
The CUDA_V12_app should be faster than the CUDA_V11_app ?
By some small amount only, depending on machine/card. That may or may not apply to your card, since it is already fast ;), and some small change in time might be hard to notice/measure.
The CPU preparation time of the CUDA WUs are the same.
So the GPU calculation time is faster?
No, the preparation time should be shorter than V11. If not, then your CPU is too fast for you to tell the difference also. :D
So the changes may or may not have any effect at all on your card/system, which I would place in the category of 'ninja bastard' hardware, that will just go fast anyway. They are mostly directed at those of us whose machines used to become unusable for normal use.
-
Build with smaller pulse find limit (2048), hoped to exhibit less issues on most nVidia Cards.
It's part of internal app tuning leaked into public area :)
Only really slow (like 8300 if such card exists :) ) GPUs could have some problems with changes that governed by this value.
And yes, V12 could be slightly faster than V11. Mostly in VLAR area though. But VLAR processing too slow still so I recommend not to use CUDA MB apps to process VLAR tasks. Use rebranding instead.
-
I switched from a V11 wich was working fine to V12, and my videos started freezing, it reminded me of the CPU optis a long time ago. How is that possible ?
I restored my backup and all is OK.
This was some weeks ago, I wil try the new nvidia/CUDA drivers and make another attempt at V12.
-
I switched from a V11 wich was working fine to V12, and my videos started freezing, it reminded me of the CPU optis a long time ago. How is that possible ?
I restored my backup and all is OK.
This was some weeks ago, I wil try the new nvidia/CUDA drivers and make another attempt at V12.
then stay with V11, just update CUDA RT DLLs to 2.3
-
OK, I can do it step by step. It's a little bit confusing. The days of the simple optis are long gone ;D
-
OK, I can do it step by step. It's a little bit confusing. The days of the simple optis are long gone ;D
Did you use installer or downloaded V12 from some other source?
-
@ Lord Asmodeus
DL the Installer for your OS here:
http://lunatics.kwsn.net/index.php?module=Downloads;catd=9
If you installed, the CUDA app work fine.
Min. nVIDIA_driver_185.85
If you would like to have a little booster take the newest nVIDIA_driver_190.38 and the CUDA_V2.3 .dll's.. ~ 30 % faster.
http://lunatics.kwsn.net/12-gpu-crunching/latest-nvidiadriver-and-cudaversion.0.html
-
Because of the priority of the CUDA_V12_app.
IIRC, in past with all stock the priority of the apps:
CPU tasks 'low'
GPU tasks 'lower than normal'
And this all the time.
Now with the opt._CUDA_app:
CPU tasks 'low'
GPU tasks 'low'*
[* at the CUDA WU start this priority is for ~ 5 sec. to 'lower than normal'. After down to 'low'.]
The priority is shown wrong in TaskManager and the opt._CUDA_app is all the time like the stock_CUDA_app at 'lower than normal'?
If, why the priority isn't shown correct all the time with the opt._CUDA_app?
Thanks! :)
-
The priority is shown wrong in TaskManager and the CUDA WU is all the time like the stock app at 'lower than normal'?
If, why the priority isn't shown correct all the time with the opt. CUDA app?
Hi Sutaru, the "Process Priority" shown by task manager is controlled/set by Boinc when it launches the app. What is manipulated inside the GPU app is the priority of the internal 'worker thread', which apparently was set too low in the original code. To see/manipulate individual thread priorities, you would need to use 'Process Explorer' or similar tool. My understanding, though it's Raistmer's mod so he can explain his reasoning better, is the worker thread is put at a slightly higher priority than before, which ensures certain types of stalls & such won't occur as often, since kernel operations & transfers can be time sensitive, so should be allowed to complete before interruptions from the controlling thread. The need for this may or may not be reduced in the future.
-
Uhh.. I understand you with my ' poor ' school english.. :-[ ;) ..like this.. that the opt._CUDA_app have little higher priority than the normal 'lower than normal' in TaskManager ?
Something between 'lower than normal' and 'normal' ?
Thanks! :)
-
Yes, I used the installer.
I just did it again, and had the same result.
Videos seams to "slideshow" a little, in fact even the screen in general seems impaired, with the mouse slowing, etc.
When I restore my backup of the C:\Documents and Settings\All Users\Application Data\BOINC folder, it doesn't work ! Does the installer do something elsewhere ? boinc.exe takes 25% of the CPU (a full core) and no app seems to launch, but it screws WU (CUDA & CPU) one by one. I have to uninstall BOINC, reinstall it, then it works. I restored again at this point, to get back the screwed WU.
I tried the new drivers and new CUDA dll, they work fine.
config : Xeon 3210, GTX280, XP32, latest BOINC
-
Yes, I used the installer.
I just did it again, and had the same result.
Videos seams to "slideshow" a little, in fact even the screen in general seems impaired, with the mouse slowing, etc.
When I restore my backup of the C:\Documents and Settings\All Users\Application Data\BOINC folder, it doesn't work ! Does the installer do something elsewhere ? boinc.exe takes 25% of the CPU (a full core) and no app seems to launch, but it screws WU (CUDA & CPU) one by one. I have to uninstall BOINC, reinstall it, then it works. I restored again at this point, to get back the screwed WU.
I tried the new drivers and new CUDA dll, they work fine.
config : Xeon 3210, GTX280, XP32, latest BOINC
Sorry, I'm a bit lost as to where you're up to with this. Is it working fine? or something trashing the boinc folder/ seti tasks, or freezing happening?
Installer does nothing outside the setiathome project folder deeper in the folder you mention, (unless you tell it to install outside)
-
Well V11 is working fine, V12 not. And when I went from V12 back to V11 (deleting all the BOINC folder, restoring rared archive), it didn't work, I had to reinstall BOINC on top of it.
-
Ahhh, I see. Thanks for explaining. Yeah, it probably just would've worked better to just use the uninstaller via add/remove programs to put the old app back, sacrificing the few WUs. For normal Boinc backups I'm fairly certain there is some elaborate mucking around to to restore from a Boinc backup that will work if there was network activity disabled while. But I've never had to try it myself, so can't advise there. In any case, glad you have something that works with your config.
Jason
-
I didn't know there was an uninstaller. Does it restore the previous optis .exe ? I saw a backup folder. It wouldn't have cost me WU, because they were only screwed when I used my "old style" method of restoring. And V12 didn't screw the WU, it was just rendering my computer unusable.
-
Ahh , yes the uninstaller, is in add/remove programs via control panel, and uses that backup folder to put back previous app, DLL's and app_info. It was tested a few times by me, though I don't know know if many people have used that feature, so it is still a cross-your-fingers approach anyway. BTW: yes we are working on improving the usability and runtime performance at Low angle ranges, and have largely isolated the offending code portions. It is challenging, and will take some time yet, but very gradual progress is being made.
-
Is there a way to make a fresh new install (of BOINC, projects, optis) without losing WUs ?
I noticed that since a month or so my RAC has dropped, my GTX280 isn't so freaking fast as before. Also, boinc.exe is using lots and lots of CPU time, I don't think it is normal (on my other computer, identical except without CUDA device, I don't see this).
I already stopped getting new WUs but I have a big cache (~1700WUs).
-
Your problem is big cache. Search for this in SETI main forums, it was discussed in details.
BOINC can't do correct scheduling of huge task lists. It will consume more and more CPU time, up to completely take whole CPU core for its own needs. It's no way can be considered as correct behavior of framework that only needed to send and recive tasks (roughly speaking) but there is no bugfix available AFAIK, only cache reduce :-\
-
OK that explains one thing. But my videocard used to make more RAC than the CPU (roughly 4500 for the CPU and 5500 for the GPU), and now it is clearly not the case (6700 total), do you have an idea about that ?
What cache size do you suggest ?
-
Explanation not very complex:
app needs CPU to feed GPU with data. CPU busy by boinc.exe, this process has higher priority than app so app starve. GPU stays idle. So, low total RAC.
I don't know value that will fit to your config. Try to reduce cache until boinc.exe will almost not eat CPU.
-
ATM I don't ask any WU, I still got 1250 to go. I'm not really conviced by your idea, my GPU seems to work all the time, I can tell by the temp of the water (watercooled PC). Moreover I have a quad core so I don't see why a boinc.exe taking 100% of one core would affect the GPU.
When I got this card some 3 months ago, it was doing the WUs in around 10 minutes, and now it's twice that, I wonder why.
-
ATM I don't ask any WU, I still got 1250 to go. I'm not really conviced by your idea, my GPU seems to work all the time, I can tell by the temp of the water (watercooled PC). Moreover I have a quad core so I don't see why a boinc.exe taking 100% of one core would affect the GPU.
When I got this card some 3 months ago, it was doing the WUs in around 10 minutes, and now it's twice that, I wonder why.
There was a change made at Seti which roughly doubled the length of MB WU's............to increase the science a bit, but also to try to tame the bandwidth as well.
But don't fret.......if you check your completed results, you should see that the awarded credits have roughly doubled too.
Probably not a problem with your setup or card.
-
ATM I don't ask any WU, I still got 1250 to go. I'm not really conviced by your idea, my GPU seems to work all the time, I can tell by the temp of the water (watercooled PC). Moreover I have a quad core so I don't see why a boinc.exe taking 100% of one core would affect the GPU.
When I got this card some 3 months ago, it was doing the WUs in around 10 minutes, and now it's twice that, I wonder why.
Well, look at msattler post, did you accounted that task changed indeed?
About BOINC-related things, you can simply check if it applies to your config or not - just leave host running few hours w/o reboots and BOINC restarts than launch task manager and see how many CPU time was consumed by boinc.exe + boincmgr.exe processes. If only few seconds - well, your BOINC behaves good. If minutes - you should decrease cache probably.
-
Well the WUs my CPU crunched were not taking double the time so I didn't think it was a change like that. But now they are.
A few days ago I asked again some WUs (my GPU had nothing left), after having set the cache to one day, and I received hundreds and hundreds of them ! Now I restarted the "not asking WUs" part, and used the rebranding utility to compensate the lack of GPU WUs. The old script wasn't working anymore, but the exe is fine. With less than 1000 WU boinc.exe takes almost no CPU time, so around 4-5 days of cache should be the sweet spot. But I want to start from fresh so I'll finish up what I have and then reinstall.
-
Is there a way to make a fresh new install (of BOINC, projects, optis) without losing WUs ?
I noticed that since a month or so my RAC has dropped, my GTX280 isn't so freaking fast as before. Also, boinc.exe is using lots and lots of CPU time, I don't think it is normal (on my other computer, identical except without CUDA device, I don't see this).
I already stopped getting new WUs but I have a big cache (~1700WUs).
Keep BOINC in the taskbar when not in use. This will take away the heavy BOINC load on your machine.
What happens is, that BOINC and the manager exchange all the WU about every 1-2 seconds.
When you have a lot of WU you will notice this.
I have about 1500-2000 WU on my machines, no problems there. I use my own BOINC manager replacement, that's a bit faster.
-
Keep BOINC in the taskbar when not in use. This will take away the heavy BOINC load on your machine.
Isn't it a true statement that it is not necessary to run BOINC Manager (BM) at all? As I see the EXIT page by right-clicking BM in taskbar, it is possible to stop BM altogether while continuing to process, upload, download WU. If BM is stopped my processors and GPU continue at 100%. BM can be started when needed, then put away for future use.
-
Keep BOINC in the taskbar when not in use. This will take away the heavy BOINC load on your machine.
Isn't it a true statement that it is not necessary to run BOINC Manager (BM) at all? As I see the EXIT page by right-clicking BM in taskbar, it is possible to stop BM altogether while continuing to process, upload, download WU. If BM is stopped my processors and GPU continue at 100%. BM can be started when needed, then put away for future use.
Just close the manager with the [X] and it hides in the taskbar. It no longer uses extra time. You can also close the manager, if you leave the applications running. The manager is only the visual shell, the BOINC core client keeps on running.
-
BOINC runs in service mode. I usually just launch the manager once a day or less, to check if all is right. I don't see why letting it run would make boinc.exe (not the manager) use less CPU time. Anyway this problem is "solved" by asking less WUs. Since the other day, my RAC has continuously raised, I don't really understand why. boinc.exe used to use like 12h of 1 core per day (one eighth of the CPU) which doesn't add up with the raise I see. Even my other computer see a raise and I have not touched it for ages (server use), so I guess it has to do with the SETI crew.
My last WUs will be finished in a few minutes, I will then completely uninstall BOINC and reinstall, we'll see if V12 is OK after that.
-
Well, it is still no good.
I removed all traces of BOINC, rebooted, reinstalled, optimized with the auto installer. All seemed OK, except maybe a few glitches during the day, surfing and multitasking as usual. Sometimes the mouse slowed a bit.
Then comes the night and the watching of movies or episodes, and it stutters like hell. I have only one GPU WU in RAM and plenty of video RAM (1GB). No VLAR.
It was exactly MB_6.08_CUDA_V12_VLARKill_FPLim2048_test.exe
Back to MB_6.08_mod_CUDA_V11_VLARKill_refined.exe which works flawlessly.
-
[snip]
Anyway this problem is "solved" by asking less WUs. Since the other day, my RAC has continuously raised, I don't really understand why. boinc.exe used to use like 12h of 1 core per day (one eighth of the CPU) which doesn't add up with the raise I see. Even my other computer see a raise and I have not touched it for ages (server use), so I guess it has to do with the SETI crew.
[snip]
My RAC has a bit more than doubled since 30 July after making sure I had all the latest KWSN software configured on the PC's. And shortened my queue. Maybe the project had something to do with it as well.
I saw posts from Raistmer, Joe Segur etc about shortening the queue but did not see any info on what is the "right" queue size. Mine is now 3 days' work. What is about "the right size" for quad cores with CUDA video?
-
Completed, validation inconclusive
same WU (http://setiathome.berkeley.edu/workunit.php?wuid=498186381)
1.claimed credit: 31.47
-> mod: V12 non VLAR Kill
-> Flopcounter: 11572163551292.662000
-> Triplet count: 5
-> GPU: GTX275
2.claimed credit 0.12
-> mod: V12 VLAR Kill
-> Flopcounter: 40177900787.577545
-> Spike count: 30
-> GPU: GT9400
greets
Maik
-
Don't trust the 9400GT's host (http://setiathome.berkeley.edu/results.php?hostid=3020611&offset=0&show_names=0&state=4), it has 42 invalid WU's completed, and loads of 'Completed, validation inconclusive', a Reboot probably needed.
Claggy
-
Don't trust the 9400GT, it has 42 invalid WU's completed, loads of 'Completed, validation inconclusive', a Reboot probably needed.
Claggy
my 9400GT requires regular OS reboots :(
With last drivers it starts to do CPU fallback. Anower weird manifestations was before... crap card :(
About correct queue size - it should be determined experimentally.
look at task manager. ig boinc.exe takes no more than 1-2% of CPU - all ok.
-
About correct queue size - it should be determined experimentally.
look at task manager. ig boinc.exe takes no more than 1-2% of CPU - all ok.
Raistmer - Excellent thank you. None of my PC are above 0.2CPU. The PC I write this on, Q6600 and ASUS 9800GTX, it is 0.02CPU. And far as I know I've never had to reboot just because of the 9800GTX, dead reliable.
Regards
Art
-
[snip]
Anyway this problem is "solved" by asking less WUs. Since the other day, my RAC has continuously raised, I don't really understand why. boinc.exe used to use like 12h of 1 core per day (one eighth of the CPU) which doesn't add up with the raise I see. Even my other computer see a raise and I have not touched it for ages (server use), so I guess it has to do with the SETI crew.
[snip]
My RAC has a bit more than doubled since 30 July after making sure I had all the latest KWSN software configured on the PC's. And shortened my queue. Maybe the project had something to do with it as well.
I saw posts from Raistmer, Joe Segur etc about shortening the queue but did not see any info on what is the "right" queue size. Mine is now 3 days' work. What is about "the right size" for quad cores with CUDA video?
I haven't found it yet. It's at 2+1 for the moment. SETI being offline for several days at times doesn't help. Moreover, I don't understand how BOINC decides to ask new WUs, it defies logic, sometimes there is a dozen WU left and it won't ask, other times there is hundreds and it keeps asking 'em. It's quite frustrating, so now I don't even open the manager anymore, I just run reschedule once or twice a day, putting 72% of the WUs on the GPU.
My RAC also took a big jump recently, maybe a change in the credit attribution ?
-
[snip]
Anyway this problem is "solved" by asking less WUs. Since the other day, my RAC has continuously raised, I don't really understand why. boinc.exe used to use like 12h of 1 core per day (one eighth of the CPU) which doesn't add up with the raise I see. Even my other computer see a raise and I have not touched it for ages (server use), so I guess it has to do with the SETI crew.
[snip]
My RAC has a bit more than doubled since 30 July after making sure I had all the latest KWSN software configured on the PC's. And shortened my queue. Maybe the project had something to do with it as well.
I saw posts from Raistmer, Joe Segur etc about shortening the queue but did not see any info on what is the "right" queue size. Mine is now 3 days' work. What is about "the right size" for quad cores with CUDA video?
I haven't found it yet. It's at 2+1 for the moment. SETI being offline for several days at times doesn't help. Moreover, I don't understand how BOINC decides to ask new WUs, it defies logic, sometimes there is a dozen WU left and it won't ask, other times there is hundreds and it keeps asking 'em. It's quite frustrating, so now I don't even open the manager anymore, I just run reschedule once or twice a day, putting 72% of the WUs on the GPU.
My RAC also took a big jump recently, maybe a change in the credit attribution ?
Been on travel for a bit. Fast forward one month from my last post. My RAC evened out at 30,000 after the August changes. Maybe with some help from Rescheduler as I've had very few computation errors. The few I did have were due to the default 4 hour reschedule, which is not frequent enough for the newer Nvidia GPUs. One hour on a 275 works well, 2 hours on a 9800GTX. SETI reached 30K despite adding in 10 percent each for Milkyway and Einstein, reducing SETI to 80% on the 275. Only trouble with the 275, it is very noisy at 100% (MSI card) with the dual fans at full speed.
-
I'm little bit curious.. ;)
Why is the priority of the opt._CUDA_6.08_V12_app at 'lower than normal' and not at 'normal'?
Because of the boinc.exe and the System activity peaks I don't crunch on the CPU on my GPU cruncher (4x OCed GTX260-216).
Everytime this both progs have activity CPU and GPU tasks would be involved.
If the CUDA tasks would have higher priority ('normal') then only the CPU tasks ('low') would be involved if other progs would have activity.
And only if all CPU tasks are stopped then also GPU tasks are involved.
For example if my 4 GPUs would have a new CUDA start (CPU preparation) and the BOINC client have also activity (all 5 progs have 'normal') my system would crash?
AMD Quad-Core CPU.
http://setiathome.berkeley.edu/show_host_detail.php?hostid=4789793
Or maybe after the CPU preparation time change the 'lower than normal' to 'normal' for CUDA tasks?
With the stock current priorities it's not well to crunch also on my CPU.
The GPU calculation time would increase x3 or something because of the big boinc.exe/System activities/peaks.
Only a 3 day WU cache!
For max. GPU performance it would be well to have 'normal' priority of the CUDA tasks all the time. (Also in the CPU preparaton time ;))
-
Over in the development area I have an AMD X2 6000 with a 9800GT, an X2 6000 with a 8400GS. Even though both X2 6000's have a 1 meg L2 there are problems piping information when Cuda is running (Cache overrun). Currently I do not have a Cuda card in my 9550 with an L3 Cache to see how that would run (warranty issues).
It gets worse, doing AP/MB and Cuda you end up with a Large Cache overrun (can you say VLAR). I have found that using an integer based (non FFT) project on the CPU and Cuda Seti on the GPU I get the best balance. So during the discussion of events over the months, the priorities were set.
Now with the Hybrid ATI for AP Raistmer is once again playing with priorties. This does not mention a Hybrid Cuda that we have no work to test with.
Then you "add" until recently, Boinc would congest itself for users with Large Caches. You are just adding CPU overhead. Everything has to fit inside the box. That is the OS, the RAM, the CPU(s) and GPU(s). What Users do not realize is that you only have X amount of resources. It is easy to overrun those resources.
That said the "priorities" to the CPU, can make it easier to overrun the resources.
Regards
-
Why is the priority of the opt._CUDA_6.08_V12_app at 'lower than normal' and not at 'normal'?
Because of the boinc.exe and the System activity peaks I don't crunch on the CPU on my GPU cruncher (4x OCed GTX260-216).
Everytime this both progs have activity CPU and GPU tasks would be involved.
Because BOINC's science apps were designed to use remaining idle CPU/GPU cycles (with GPU it not tru now, it mostly you should just stopp GPU app when doing something GPU-intensive).
So, they priority should be lower than normal to not to disturb other user's applications.
Correct question is:
Why BOINC daemon and manager priorities are normal ones and not below normal. Cause I found that on notebook BOINC, even with CPU-only apps, disturbs video player.
So I had to disable BOINC computations completely to watch movie. It's not good, even it's bad actually.
-
Hello opt. crew!
I 'helped(?)' Jon in the SETI@home NC subforum to install opt. apps.
Core i7 920 & 2x GTX295 (old 2x PCB), Win7 64bit.
But after installation he have CUDA probs/errors.
Please if you have time, have a small look here: 'RAC with 2 gtx295 (http://setiathome.berkeley.edu/forum_thread.php?id=58380)'
IIRC, nVIDIA_driver_191.x and all stock and everything was well.
The probs started here.. 'Message 962334 (http://setiathome.berkeley.edu/forum_thread.php?id=58380&nowrap=true#962334)'.
In my messages you can find some interesting infos about his system.. but now - I'm out of ideas.. :-\
Thanks! :)
-
You mean when you told him to install the VLAR_kill application, without explaining what it does, how it works, and the requirement for a user of 'anonymous platform' applications to manage and maintain their own science app from that point forward?
Most of his errors are -6 "Bad workunit header". It's a VLAR WU. VLAR kills it. That's what it does.
Task 1478354101 (http://setiathome.berkeley.edu/result.php?resultid=1478354101) is interesting. -6 error, but no VLAR_kill message. Raistmer?
There are also a number of "Incorrect function. (0x1) - exit code 1 (0x1)", after what would appear to be full-term runtimes, but with none of the standard data in stderr_txt. Anyone?
-
Yes, most errors just VLAR rejections. But there are some that very similar to my own troubles with 9400GT in dual-GPU config on Core2 Duo host.
Same "0" available memory readings time to time, same "unknown error".
For my own host it was only one solution - to remove 9400GT from it and leave 9600GSO only. It works perfect now.
Also, 9400GT works just perfect in Q9450 host.
What the reasons for such behavior?
I see 3 possibilities:
1) overheating.
2) system underpowered
3) system PCI-E bus overloaded and brings corruption to bus transfers.
Check them.
last one could be checked by using bandwidth sample from nVidia's CUDA or OpenCL samples.
-
Task 1478354101 (http://setiathome.berkeley.edu/result.php?resultid=1478354101) is interesting. -6 error, but no VLAR_kill message. Raistmer?
Perhaps task was aborted in especially rough way and stderr buffer was no flushed into file.
I'm more concerned with such errors:
http://setiathome.berkeley.edu/result.php?resultid=1478354023
-
Yes, most errors just VLAR rejections. But there are some that very similar to my own troubles with 9400GT in dual-GPU config on Core2 Duo host.
Same "0" available memory readings time to time, same "unknown error".
For my own host it was only one solution - to remove 9400GT from it and leave 9600GSO only. It works perfect now.
Also, 9400GT works just perfect in Q9450 host.
The system has 2 old 2 pcb 295's probably the worst cards that nVidia made. They are not suitable for CUDA work, one maybe but two together get way way too hot.
I had my experience with them, and not good. The newer 1 PCB version or the 295 are ok though, quite a different design.
And to top thing off he uses Win 7 with drivers I couldn't get to work properly with my 2 295. Too many driver crashes.
So this system is asking for trouble and he is sometimes OC them as well. So not the best testbed.
-
Thanks! :)
I thought this would be also interesting.. 'Message 962882 (http://setiathome.berkeley.edu/forum_thread.php?id=58380&nowrap=true#962882)'.
Only ~ 200 MB free GPU RAM? All 2 GTX295 (all 4 GPUs), Win7 prob?
What would be the easiest way?
Install WinXP?
Just curious.. 'Message 962794 (http://setiathome.berkeley.edu/forum_thread.php?id=58380&nowrap=true#962794)', the same WinXP on more PCs? True or lie? :-\
This would be the easiest/cheapest way for Jon..
BTW. My browser need ~ 10+ sec. to find this site before he can load the site.
Is there a connection prob?
In past it was much faster/immediately..
-
@Raistmer how much futher should I carry the experiment. This host has been running for quite a while and the bulk of the errors are -12's a -1 and vlarkilled.
usability of video is good... Running Aqua on CPU
http://setiathome.berkeley.edu/show_host_detail.php?hostid=5133086
Shortly I can switch on Seti to the CPU's if needed.
-
do yuo run V12b? On single GPU? at any moment :)
It's for multi GPU and even there advantages still questionable.
-
do yuo run V12b? On single GPU? at any moment :)
It's for multi GPU and even there advantages still questionable.
Actually I like the affinty feature esp when on an AMD. In the past I played with setting affinity and saw increased progress but was not able to set for 24/7 to set affinity on each task. Even Crunch3r knew that if task could get locked to a specifice CPU then on a dual CPU machine one runs full tilt and the other handles most of the OS stuff. So to that extent that part is a success.
Here is a LAR http://setiathome.berkeley.edu/result.php?resultid=1478568634 0.21 AR that it just completed (2210.4 sec).
Now if we could get Sataru to test it (live), we would have a better idea of its merit. It will not hurt his RAC and after a month live it is "safe."
-
In a multi-GPU, multi core system there could be two possibilities: 1) all GPU tasks assigned to a single core, 2) every GPU gets its own core. Have you considered these regarding affinity?
-
In a multi-GPU, multi core system there could be two possibilities: 1) all GPU tasks assigned to a single core, 2) every GPU gets its own core. Have you considered these regarding affinity?
It has been studied many times and can work to the advantage of the system administrator. Over the years there have been many utilities that you could down load that would "lock on launch" an application to a specific CPU. So while CPU 0 is handling Ring 0 stuff, launch the Database compression on CPU1 with affinity lock. So in this first experiment Raistmers intent was in machines with multi cores and multi GPU's that the GPU tasks were assigned to a specific core. Looking at Boinc and how things are ran the Apps migrate back and forth depending on what "might happen" at any given instant. With the advent of multicores Crunch3r attempted to convince the Boinc Devs that "affinity locking" might be a very good idea. He even produced a boinc core or two to prove it.
So to actually fully prove it all Lunatics apps would have to be CPU/core aware (and a table setup to read what is where) . Core 0 takes care of the OS and other stuff the Users plays with. Other cores real or virtual then are assigned to a specfic app, they run clean uninteruppted.
in Nix you can assign certain things to CPU's which has been there for ages... That was done on a Dual Pent Pro under slackware (then it was only 19 1.44 floppies to load).
-
Thanks Pappa, I know all these, more or less. I'm asking how Raistmer implemented the affinity in this app, all cuda tasks in a fixed core or distributes them among all available?
-
Thanks Pappa, I know all these, more or less. I'm asking how Raistmer implemented the affinity in this app, all cuda tasks in a fixed core or distributes them among all available?
From what I read all GPUs would use one core. But that was few pages ago...
We wait for Raistmer ;D
For nostaliga it is team member #5 before it went to Nix
http://seticlassic.ssl.berkeley.edu/stats/team/team_57956.html
boy how things change
-
From what I read all GPUs would use one core. But that was few pages ago...
We wait for Raistmer ;D
Theoretically speaking, I think it would be better to distribute them among all cores, or not? :-\
I'll try to do something similar in linux through script.
For nostaliga it is team member #5 before it went to Nix
http://seticlassic.ssl.berkeley.edu/stats/team/team_57956.html
boy how things change
Oh, memories of seti classic! :)
-
From what I read all GPUs would use one core. But that was few pages ago...
We wait for Raistmer ;D
Theoretically speaking, I think it would be better to distribute them among all cores, or not? :-\
I'll try to do something similar in linux through script.
For nostaliga it is team member #5 before it went to Nix
http://seticlassic.ssl.berkeley.edu/stats/team/team_57956.html
boy how things change
Oh, memories of seti classic! :)
Actually I think that is part was what is trying to be processed . Silly me. :o
-
on quad or higher V12b locks 2 cores to each app, v12bx4 locks 1 core (different one, of course) to each app. the idea is to remove competition for cpu between high-priority gpu apps themselves. It will be needed only if we have few gpu apps running, i.e. multi-GPU host.
-
Thanks Raistmer, so you distribute them evenly among available cores, that is also the best in my thinking. I'll write a script in the linux forum to simulate it in linux since we don't have the luxury of a new build.
I'll do something similar for CPU multibeam tasks also, but it is a little trickier. I try to think a "fixed" feature of CPU tasks to "measure" for the affinity, PIDs, running time of the task (older-newer) or anything else seem not good enough characteristics.
To me the best seem slot numbers. They are pretty fixed, they might change only when boinc goes to pre-empt mode. What do you think?
-
yes, slots numbers should go. But it's worth to check if affinity really increase throughput.
I interested affinity locking few times already and always seen performance degradation for app. Only in some special circumstanses as app feeding GPU it could bring some benefits. In most cases windows do good thread allocation between cores (don't know about linux).
It (windows) just can't do thread re-shedule on different core based only on thread priority (maybe it can but doesn't do it when needed).
That is, 2 gpu apps on one core while 2 cpu-only apps on another core is quite possible. gpu app has bigger priority but can't preempt cpu app on second core (experimental fact with 2 early hybrid AP and 4 MB apps on my quad). It looks as pretty big OS core limitation, maybe in some win version it removed already, maybe some msdn reading on this topic could give more info...
-
yes, slots numbers should go. But it's worth to check if affinity really increase throughput.
I interested affinity locking few times already and always seen performance degradation for app. Only in some special circumstanses as app feeding GPU it could bring some benefits.
The logical thing would be that four AK_v8 tasks fixed in their respective cores during their lifetime would be better than have them jumping around cores every few seconds. You say that experiments have shown the opposite?
-
yes, slots numbers should go. But it's worth to check if affinity really increase throughput.
I interested affinity locking few times already and always seen performance degradation for app. Only in some special circumstanses as app feeding GPU it could bring some benefits.
The logical thing would be that four AK_v8 tasks fixed in their respective cores during their lifetime would be better than have them jumping around cores every few seconds. You say that experiments have shown the opposite?
yes, opposite. runtime for affinity locked app is bigger.
for example when one core wait for disk io or smth else it can be better to bring thread from another core. in general, thare never num of active threads <=number of cores. For windows especially vista and so on, number of threads can reach thousands.
for example, my netbook under vista currently runs 59 processes with 765 threads. of course most of these threads are suspended, but there are only 2 CPUs to handle all others....
-
In past with Crunch3r's BOINC V6.1.0 (mod of stock V5.10.1x) and CPU-affinity I saw a crunching speed up of ~ 1 - 2 % with MB WUs.
Intel Core2 Extreme QX6700.
This is a 2x DUO-CPU. 2x shared L2-Cache. So cache miss can happen.
Now with the current available Quad-CPUs which have a L2-Cache/CPU-Core this Cache miss can happen much more.
It would be nice to have again a Truxoft- or Crunch3r- BOINC mod with this.
-
In past with Crunch3r's BOINC V6.1.0 (mod of stock V5.10.1x) and CPU-affinity I saw a crunching speed up of ~ 1 - 2 % with MB WUs.
Intel Core2 Extreme QX6700.
This is a 2x DUO-CPU. 2x shared L2-Cache. So cache miss can happen.
Now with the current available Quad-CPUs which have a L2-Cache/CPU-Core this Cache miss can happen much more.
It would be nice to have again a Truxoft- or Crunch3r- BOINC mod with this.
You could test Raistmers App as he asked you to do. What you could, report would mean he moves it ahead or it truly dead. It is your choice.
It is just that simple... BS Aside.
-
You could test Raistmers App as he asked you to do. What you could, report would mean he moves it ahead or it truly dead. It is your choice.
It is just that simple... BS Aside.
Don't you think it's time to stop with this?
I think enough is ENOUGH. >:(
Let it be!
-
Please, look at this task: http://setiathome.berkeley.edu/result.php?resultid=1483022945
With 3 GPU 275
<core_client_version>6.10.17</core_client_version>
<![CDATA[
<message>
Incorrect function. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
setiathome_CUDA: Found 3 CUDA device(s):
Device 1 : GeForce GTX 275
totalGlobalMem = 939524096
sharedMemPerBlock = 16384
regsPerBlock = 16384
warpSize = 32
memPitch = 262144
maxThreadsPerBlock = 512
clockRate = 1512000
totalConstMem = 65536
major = 1
minor = 3
textureAlignment = 256
deviceOverlap = 1
multiProcessorCount = 30
Device 2 : GeForce GTX 275
totalGlobalMem = 939524096
sharedMemPerBlock = 16384
regsPerBlock = 16384
warpSize = 32
memPitch = 262144
maxThreadsPerBlock = 512
clockRate = 1512000
totalConstMem = 65536
major = 1
minor = 3
textureAlignment = 256
deviceOverlap = 1
multiProcessorCount = 30
Device 3 : GeForce GTX 275
totalGlobalMem = 939524096
sharedMemPerBlock = 16384
regsPerBlock = 16384
warpSize = 32
memPitch = 262144
maxThreadsPerBlock = 512
clockRate = 1512000
totalConstMem = 65536
major = 1
minor = 3
textureAlignment = 256
deviceOverlap = 1
multiProcessorCount = 30
setiathome_CUDA: CUDA Device 3 specified, checking...
Device 3: GeForce GTX 275 is okay
SETI@home using CUDA accelerated device GeForce GTX 275
V12 m˙dification by Raistmer
Priority of worker thread rised successfully
Priority of process adjusted successfully
Total GPU memory 939524096 free GPU memory 256602112
setiathome_enhanced 6.02 Visual Studio/Microsoft C++
Build features: Non-graphics CUDA VLAR autokill enabled FFTW USE_SSE x86
CPUID: Intel(R) Core(TM)2 Quad CPU Q9650 @ 3.00GHz
Cache: L1=64K L2=6144K
CPU features: FPU TSC PAE CMPXCHG8B APIC SYSENTER MTRR CMOV/CCMP MMX FXSAVE/FXRSTOR SSE SSE2 HT SSE3
libboinc: 6.3.22
Work Unit Info:
...............
WU true angle range is : 2.472358
After app init: total GPU memory 939524096 free GPU memory 223047680
Cuda error 'cudaAcc_CalcChirpData_kernel2' in file 'd:/BoincSeti_Prog/sinbad_repositories/LunaticsUnited/SETI_CUDA_MB_exp/client/cuda/cudaAcc_CalcChirpData.cu' in line 106 : unknown error.
Cuda error 'cufftExecC2C' in file 'd:/BoincSeti_Prog/sinbad_repositories/LunaticsUnited/SETI_CUDA_MB_exp/client/cuda/cudaAcc_fft.cu' in line 143 : unknown error.
Cuda error 'cufftExecC2C' in file 'd:/BoincSeti_Prog/sinbad_repositories/LunaticsUnited/SETI_CUDA_MB_exp/client/cuda/cudaAcc_fft.cu' in line 143 : unknown error.
Cuda error 'cudaAcc_GetPowerSpectrum_kernel' in file 'd:/BoincSeti_Prog/sinbad_repositories/LunaticsUnited/SETI_CUDA_MB_exp/client/cuda/cudaAcc_PowerSpectrum.cu' in line 56 : unknown error.
Cuda error );cudaAcc_GetPowerSpectrum_kernel' in file 'd:/BoincSeti_Prog/sinbad_repositories/LunaticsUnited/SETI_CUDA_MB_exp/client/cuda/cudaAcc_PowerSpectrum.cu' in line 56 : unknown error.
Cuda error 'cudaAcc_summax32_kernel' in file 'd:/BoincSeti_Prog/sinbad_repositories/LunaticsUnited/SETI_CUDA_MB_exp/client/cuda/cudaAcc_summax.cu' in line 147 : unknown error.
Cuda error 'cudaAcc_summax32_kernel' in file 'd:/BoincSeti_Prog/sinbad_repositories/LunaticsUnited/SETI_CUDA_MB_exp/client/cuda/cudaAcc_summax.cu' in line 147 : unknown error.
Cuda error 'cudaMemcpy(PowerSpectrumSumMax, dev_PowerSpectrumSumMax, cudaAcc_NumDataPoints / fftlen * sizeof(*dev_PowerSpectrumSumMax), cudaMemcpyDeviceToHost)' in file 'd:/BoincSeti_Prog/sinbad_repositories/LunaticsUnited/SETI_CUDA_MB_exp/client/cuda/cudaAcc_summax.cu' in line 160 : unknown error.
</stderr_txt>
]]>
Thanks for any help
Franz
-
You could test Raistmers App as he asked you to do. What you could, report would mean he moves it ahead or it truly dead. It is your choice.
It is just that simple... BS Aside.
Don't you think it's time to stop with this?
I think enough is ENOUGH. >:(
Let it be!
I think we stop this topic. All PMs sent and recived IMO.
-
Please, look at this task: http://setiathome.berkeley.edu/result.php?resultid=1483022945
With 3 GPU 275
As one can see nVidia's error reporting system provide not much help in debugging.
"unknown error" all that said.
How often you got such results?
If it's only 1 per few dozens - well, you should just live with that for now. If much often, then it's topic to discuss.
ADDON: And I would recommend to change driver to 191.* or 190.* and go with CUDA 2.3
195.* just unndeeded for non-fermi cards and may have some new issues indeed.
[In the world of software it's often happens that newer != better, just look on Vista release over XP ;D ]
-
It's a Win7 OS.
From what I read of Fred and his 2x GTX295 PC, he couldn't let run Win7 without probs (CUDA errors, driver crashs).
So he's down to WinXP and everything is fine. ;)
-
It's a Win7 OS.
From what I read of Fred and his 2x GTX295 PC, he couldn't let run Win7 without probs (CUDA errors, driver crashs).
So he's down to WinXP and everything is fine. ;)
Microsoft Windows 7 ... Ultimate x64 Edition ...Nvida driver: 19562
Pelase look at here http://setiathome.berkeley.edu/result.php?resultid=1483726484
and tell me if there is some problem with 3 GPU in SLI mode or in NO SLI mode.
Franz
-
Your app instance crashed badly on that task.
Try to replace drivers with 191.* ones.
-
"Total GPU memory 939524096 free GPU memory 256024576"
And where other ~700MB of GPU memory ???
-
Franz, this is a known BUG of the opt._CUDA_app.
Raistmer inserted a BUG detection, which should detect and display the BUG of the CUDA BUG.
Sometimes it fail and show this 'funny/strange' output.
Normally it should display the -12 error with BUG report.
But it show sometimes (very rarely) an 'Out Of Memory' error.
Raistmer, you can't remember, we both together found/discovered this.. ;)
You should not have enabled SLI for CUDA. The new driver should disable SLI automatically if CUDA.
Raistmer, in the SETI@home forum I saw a member also with Win7.
He have 2x GTX295 and ALL GPUs had only ~ 400 or ~ 200 MB free GPU RAM before CUDA load.
Sometimes too less and then the CUDA WU was crunched on CPU (falling back to CPU processing...).
I guess it's a Win7 or nVIDIA_driver BUG.
-
Franz, BTW.
This Win7 PC have very much 1 errors.
It's like the other member I helped with his Win7 and 2x GTX295 PC.
Also Fred said, he couldn't let run well CUDA on his Win7 and 2x GTX295.
Not 190.x , 191.x and not 195.x . He continue with WinXP.
So I recommended the member to install WinXP and since then no error.
It could be a BUG in Win7 or the current available nVIDIA_drivers.
So the time will tell if with a new nVIDIA_driver it will be better.
The easiest would be to install WinXP. ;)
-
Raistmer, you can't remember, we both together found/discovered this.. ;)
Ah, good that you remind me. Yes, sometimes memories fall :)
-
i am using W7 on my machine with 9600GT and 8500GT for 4 monts with always latest drivers (beta or not) and NEVER had problems low mem on any CUDA project
-
i am using W7 on my machine with 9600GT and 8500GT for 4 monts with always latest drivers (beta or not) and NEVER had problems low mem on any CUDA project
x86 or x64 OS ?
-
x64
-
Yuo are lucky one then :)
Perhaps newly GPUs work differently with same driver...
-
Franz, BTW.
This Win7 PC have very much 1 errors.
It's like the other member I helped with his Win7 and 2x GTX295 PC.
Also Fred said, he couldn't let run well CUDA on his Win7 and 2x GTX295.
Not 190.x , 191.x and not 195.x . He continue with WinXP.
So I recommended the member to install WinXP and since then no error.
It could be a BUG in Win7 or the current available nVIDIA_drivers.
So the time will tell if with a new nVIDIA_driver it will be better.
The easiest would be to install WinXP. ;)
Hi, about the error I show you (on win7 (64), the solution was:
reinstall everything but CUDA DLL's 2.3 (leave the original BOINC dll's) It's working fine.
Now I heve a lot off errors on an other WIN7 (32) PC (with GTS 250)
http://setiathome.berkeley.edu/results.php?hostid=5120050&offset=0&show_names=0&state=5
For example a lot of:
Cuda error 'find_pulse_kernel2<3, false>' in file 'd:/BoincSeti_Prog/sinbad_repositories/LunaticsUnited/SETI_CUDA_MB_exp/client/cuda/cudaAcc_pulsefind.cu' in line 1603 : unknown error.
and finally a line like this:
Cuda error 'cudaMemcpy(&flags, dev_find_pulse_flag, sizeof(*dev_find_pulse_flag), cudaMemcpyDeviceToHost)' in file 'd:/BoincSeti_Prog/sinbad_repositories/LunaticsUnited/SETI_CUDA_MB_exp/client/cuda/cudaAcc_pulsefind.cu' in line 1733 : unknown error.
Bye,
Franz
-
Check in windows system log if you had driver restarts.
Also there is 196.xx driver update available now AFAIK. Maybe it could solve your troubles.
-
Check in windows system log if you had driver restarts.
Also there is 196.xx driver update available now AFAIK. Maybe it could solve your troubles.
That host is using driver 19621
Tomorrow I'll loot at system log.
Bye
Framnz