Seti@Home optimized science apps and information
Optimized Seti@Home apps => Windows => GPU crunching => Topic started by: Miep on 19 Jul 2010, 11:38:28 am
-
Hello world ;)
Ok, so I'm running a high spec Notebook with a NVIDIA Quadro FX 570M , 256MB. this host (http://setiathome.berkeley.edu/show_host_detail.php?hostid=5200185) for a few more details.
And from the start it has been having stability issues with the graphic driver (IRQ_zero at system service bluescreens among others). Well I assume it's the graphic driver,
since updating it has increased stability.
Up to 195.62 which is the most stable and the one I'm currently running.
I did upgrade to 257.21, but with the decrease in reported RAM it didn't want to run stock 6.08 any more and it was more unstable on top. (Looks like they just released 258.96, haven't tried that one...)
I downgraded again last week so I could at least run stock 6.08.
So. I tried optimized CUDA MBnokill 2.2 dll and it errored out on me. Didn't go through to report and I didn't remember to preserve the stderr, but from what I've read I assume it's memory issues - stock 6.08 runs just fine (AND I've got aero enabled ;P). I can probably free up some RAM if somebody explains how to do it (or just points to the right post).
As an interim measure I've stuck 6.08 into app_info, so at least I get some WUs to play around with. So far it hasn't errored (first one at 15% done right now) and since the errors with optimized occured in the first two minutes or so, I hope it's playing along for the moment.
Any further suggestions?
-
Miep:
Maybe others here can help you (especially if you can link to errored WU and/or provide STDERR output) for the WU(s) that errored out when you tried optimized NO-VLAR-kill app.
Asisde: Just an FYI for MB WUs run on GPU using the optimized VLAR-Kill version: If the WU terminates after only a few elapsed seconds, it was terminated because the VLAR was too low (ie the VLAR-Kill caussed termination). In such cases you'll see something similar to this
Stderr output
<core_client_version>6.10.56</core_client_version>
<![CDATA[
<message>
- exit code -6 (0xfffffffa)
</message>
<stderr_txt>
VLAR WU (AR: 0.009264 )detected... autokill initialised
SETI@home error -6 Bad workunit header
The "error -6 " means VLAR-Kill did it's job.
-
Hi here :)
Try to disable Aero first. Also, if 191.xx drivers work with CUDA 2.3 DLLs stable - no need to upgrade to higher versions (still)
-
Hi here :)
Try to disable Aero first. Also, if 191.xx drivers work with CUDA 2.3 DLLs stable - no need to upgrade to higher versions (still)
When Miep asked about this on the main board last week (Nvidia driver revert? (http://setiathome.berkeley.edu/forum_thread.php?id=60744)), I had a scout round and found that 190/191 drivers were never released (even in Beta) for her Quadro FX 570M / Vista 32. The first available driver with CUDA 2.3 support was the 195.62 she's using now.
-
Ah well, next iteration. Disabled aero, rerun.
I think it might just be possible to get to many debugging messages... Are there actually people who can make sense of core dumps?
So, memory, as assumed:
<stderr_out>
<![CDATA[
<message>
- exit code -1073741819 (0xc0000005)
</message>
anf then further down
After app init: total GPU memory 268435456 free GPU memory 38637568
Cuda error 'cudaMalloc((void**) &dev_WorkData' in file 'd:/BoincSeti_Prog/sinbad_repositories/LunaticsUnited/SETI_CUDA_MB_exp/client/cuda/cudaAcceleration.cu' in line 293 : out of memory.
setiathome_CUDA: CUDA runtime ERROR in device memory allocation (Step 1 of 3). Falling back to HOST CPU processing...
Unhandled Exception Detected...
- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x726F662F read attempt to address 0x726F662F
Engaging BOINC Windows Runtime Debugger...
followed by no less that 7 pages of runtime debugger messages.
Kept client_state in case anybody feels like having a closer look.
Fine,back to 6.08 for now.
-
no need to red those dumps further. It's clear that memory allocation failed. AFAIK last GPU-Z shows memory usage for nVidia cards (no such service for ATI GPUs still). Maybe you can see how much GPU memory in use before running CUDA MB with this tool?
-
OK.
So GPU-Z 0.4.4 reports 104MB used. Erm. Not good. A bit of tweaking (appareance to Basic and colours to 16bit) brought that down to 19 (oh and the windows fix to temporarily disable .lnk and .pif not sure if that has an impact).
Still not enough. :(
While I was looking at client_state I noticed
Cuda error 'cudaMalloc((void**) &dev_GaussFitResults' in file 'c:/sw/gpgpu/seti/seti_boinc/client/cuda/cudaAcceleration.cu' in line 314 : out of memory.
setiathome_CUDA: CUDA runtime ERROR in device memory allocation (Step 1 of 3). Falling back to HOST CPU processing...
even on the WUs that had (from a users perspective) run error free.
?!
So, what's it doing then? It does put load on the GPU after all?
I'll run with more free memory and see if that clears up...
-
No, it performs fallback to CPU and not using GPU at all if you see listed message. It's slowest way to do seti task actually :) Better disable GPU processing completely then (if you can't free more memory for app).
Another way is to fallback to older CUDA DLLs (like 2.2 or even older) they have slower cuFFT but require less memory. That way will be still much better than CPU fallback mode.
-
It's slowest way to do seti task actually :)
No it isn't. Asking my dog to process tasks is much slower. He hasn't returned a valid result yet :o
-
If it was doing just that, fine (not by cruching standards, but by 'I know what it's doing').
But how do you then explain
a) load on the GPU - from the heat generated that is. I'll run with GPU-Z in the background over night and see if iit shows load.
b) crunching times - well hard to say actually CPU is ak_v8b_win_ssse3x.exe and ap_5.05r409_sse.exe respectively GPU is 6.08 stock.
and I'm getting 1h50' avarage for shorties on GPU and 1h30' on CPU
When I was running both on stock GPU used to take about 1/3 of CPU - and besides the driver I've not conciously changed the setup.
[times are not really comparable though, I'm still finetuning how hot I allow CPU and GPU to get, while not having them throttled into oblivion]
c) both CPU cores are on approximately the same times as without GPU tasks. (same throughput)
d) shouldn't then a cpu core get taken over? I still get 3 tasks running.
So, from all I can tell at least part of it uses the GPU.
I don't think I've returned any of those tasks yet, so I don't know if they validate. Old setup used to, but as there was never any reason to check stderr I've no idea if it was showing
the same behaviour.
-
The CPU fallback in the CUDA apps happens without BOINC knowing about it, so it assumes the CUDA task it started is only using a small fraction of a CPU and will start a CPU task also. The CUDA task is running at a higher priority than the CPU task, so runs mostly uninterrupted using the minimal default set of CPU code. The task started for the CPU will probably run quite slowly since it will get much less CPU time. So not only would the task started for CUDA be running slowly on CPU, a CPU task would also be running slowly; as Raistmer said, the worst possible way to do work.
Raistmer wrote:
Another way is to fallback to older CUDA DLLs (like 2.2 or even older) they have slower cuFFT but require less memory. That way will be still much better than CPU fallback mode.
I think that is the only way you'll be able to crunch with 256 MB VRAM.
Joe
-
It'll be a bit difficult to work out what's happening until the outage is over, and we can see both CPU and Elapsed times reported on the website: I think BOINC does record actual CPU usage, even on a nominal GPU-allocated task.
At the moment, I'm getting confused signals. It looks as if the optimized CUDA MBnokill you started with has broken CPU fallback code, and just errors on low memory. From what you've said, the v6.08 CPU fallback mode is kicking in as designed, and at least generating a result - but at the lowest possible speed. The CPU code in that build has few, if any, of the optimisations present in even the stock v6.03 CPU application, let alone the additional ~2x speedup available with AK_v8b. Why NVidia incorporated such a crappy CPU codebase, I'll never know.
But your later observations don't bear that out. More heat from the GPU? That implies work - so what's it doing? (Unless, just possibly, merely starting the app kicks up a fequency shift, from idle to active - that sort of thing might be expected in a notebook). And if the CPU is doing the bulk of the heavy lifting for the nominal 'GPU' app, you'd expect that the elapsed timings for CPU tasks (I'm presuming those are what you're reading, from BOINC Manager) would increase significantly, even if CPU time should remain constant. Unless, again, the higher-priority GPU app is triggering a frequency increase in the CPU, from idle to performance. That's been a nasty gotcha for Linux users, which took a long time to track down the first time we came across it - presenting problem was that the Linux app was about half as efficient as the equivalent Windows app, which didn't make sense.
It would certainly be worth running GPU-Z in the background, and watching in particular what happens to GPU speed and utilisation as a new task starts up (from scratch, obviously, rather than replacing an existing running task). It might also be worth having a look at what the power management settings are doing for the system as a whole. If the notebook is high enough spec to have a Quadro as standard, then it's probably got some good power control stuff as well. Check both the BIOS, and for any Vista power extensions. Just for experimentation and understanding what's going on, it would be good to eliminate any variability from frequency shifting you can, while you test everything else. And you might even be able to allocate more RAM to the GPU in BIOS.
-
The CPU fallback in the CUDA apps happens without BOINC knowing about it, so it assumes the CUDA task it started is only using a small fraction of a CPU and will start a CPU task also. The CUDA task is running at a higher priority than the CPU task, so runs mostly uninterrupted using the minimal default set of CPU code. The task started for the CPU will probably run quite slowly since it will get much less CPU time. So not only would the task started for CUDA be running slowly on CPU, a CPU task would also be running slowly; as Raistmer said, the worst possible way to do work.
I hate it when I manage to kill my own post while writing... OK let's see...
You were right , of course ;)
GPU memory shifts from 19 to 136 used and after a few secs down to 49 and stays there. Core clock, memory clock and shader clock all go up and stay up. (whatever that exactly is...)
I suppose Richard is right - enough to shift (and heat up) not enough to run.
We are talking 6.08 stock right now, as I understood it, optimized has higher memory requirements? So, if I can't get stock to run, no point in vieing for opt.
Raistmer wrote:
Another way is to fallback to older CUDA DLLs (like 2.2 or even older) they have slower cuFFT but require less memory. That way will be still much better than CPU fallback mode.
I think that is the only way you'll be able to crunch with 256 MB VRAM.
Joe
Ah, I think you laid the finger in the wound.
When I downgraded the driver last week I was far to stressed to pay attention to details. I wrongly assumed the libs would follow suit. However:
22/07/2010 09:47:08 NVIDIA GPU 0: Quadro FX 570M (driver version 19562, CUDA version 3000, compute capability 1.1, 256MB, 61 GFLOPS peak)
So Cuda 3.0 dll lib?
Well guess I have to remove a couple of files then to get back to 2.2. So, what needs removing/replacing? I doubt I can find all relevant files on my own...
Thanks a lot.
-
Don't worry about the 'version 3000' from BOINC - that's just showing the maximum version you could use with that driver.
You can use lower versions, just by changing the two DLLs in the boinc\projects\sah directory.
Attaching, for simplicity, the 2.2 and 2.1 versions - straight file replace, no rename. Stop BOINC first, obviously, and try: 2.2 first, 2.1 if that fails.
Let me know when you've downloaded the files, and I'll remove the attachment - save server space.
Edit - attachments removed, served their purpose.
-
Ta.
Ack. That was 2.2 in there... Ok trying 2.1
As the GPU is on a 4 min wait for inactivity, no results expected before tomorrow.
-
So, that did the trick.
memory usage up to 218, no errors.
Fine, stock 6.08 working on 2.1 dlls - does the optimized V12nokill whetever require 2.2 dlls or will it run on the 2.1?
-
should run with 2.1 too as I can remember. Good that you able to use your gpu now for seti
-
So, that did the trick.
memory usage up to 218, no errors.
Fine, stock 6.08 working on 2.1 dlls - does the optimized V12nokill whetever require 2.2 dlls or will it run on the 2.1?
ThE`e's not much in the way of actual optimisation in V12, kill or no kill. A bit of startup delay eliminated, and a higher base driver requirement (but you're well past that limit already). The other change was the priority mods, but they got incorporated into the later BOINCs.
The best step you could take, speed-wise, would be to step up the DLLs. Now you've got it working, would it be worth trying once more with 2.2? Suspend all CUDA tasks after the one that's currently running, drop the new DLLs into the project folder, then release one as a trial. Don't even need to stop BOINC for that test. If ht works, bigger gain than any of those early experimental builds.
-
Well I upgraded BOINC to 6.10.58 and swapped the V12 back into app_info. seemed to run, but as usual will have to wait over night.
I dn't think 2.2s will work Richard, the CPU fallback stopped as soon as I had dropped the 2.1s in...
I can try tomorrow, but I had initially V12 running on 2.2 and that just errors out - on 6.08 it drops to CPU. With 2.1 I get load onto the GPU...
-
I'm really surprised at that - the 2.2 DLLs caused no problems at all when they came out, and the FFT DLL itself is only 1MB bigger than its predecessor (have to ask Jason what the actual difference in FFT processing is, and whether it needs significantly more space for data handling). Trouble is, nobody else has really stressed out 256 MB under Vista before (though IIRC, Claggy eventually got 128 MB running, also on a mobile GPU - or was that just testing the CPU fallback code?)
-
Just the CPU fallback code, and it was so so slow,
Claggy
-
I told you i was running, what I assume to be the bare minimum...
When CUDA came out last year (?) I looked my GPU up and it stated it could do it. Required me to figure out that I needed to upgrade the driver though. It was crunching happily then - you can just about see the increased throughput from 8/09 to 2/10 on the boincstats graphs... but unless I find I have a non kompressed backup of my data partition from that period I can't check the respective driver/dll/client_stat. And that was with aero and 32bit colours - I've disabled that atm.
I was running stock then, so whatever dll you get/got delieverd with 6.08. (well last august...)
It was just the reduced reported RAM from that @!£$ (sorry) 197 driver update that took the GPU out of action for SETI.
I'll see if I can find a backup on Monday, would answer a few questions...
-
hm.... could it be not only driver update but boinc itself update too?
Early BOINC versions didn't handle GPU but didn't load it also.
Currently BOINC is gpu-aware, but this could require some gpu memory too. At least what needed is context creation. How much memory driver assosiated with context by default?.... IMO it's worth to check GPU memory usage with BOINC doing only CPU tasks and w/o BOINC at all too.
Also, maybe try to go to older instead of newer BOINC versions.
-
I don't see how the BOINC version can make any difference at all. You know the code better than me, but surely there are only two places where BOINC interacts directly with any GPU:
1) At startup, when it queries the device capabilities - calling the APIs in the driver's DLLs, and populating the COPROCS structure. But that would be held in CPU / system RAM - I don't see why changes to this detection phase should tie up any extra VRAM.
2) At application launch - every project science app has BOINC code in it, from the API libraries, to allow communication and control with the core client. But that's built into the application itself - AFAIK, no application picks up any new library code from a later BOINC installation.
The only possibility I can see is that Carola might have been using an even older NVidia driver, possibly Notebook Release 186 WHQL 186.81 or even Quadro FX Release 186 for Notebooks 186.03, which would rule out CUDA 2.3 but might allow 2.2 to run.
-
@Raistmer context creation? sorry you lost me - you want me to check GPU usage under different scenarios, to see what influences the components have? ok, I'll try.
btw ther's a 'H' missing in your 'IMO' ;)
You made me remember that I have a Boinc directory backup from May. not old enough to check what the GPU was running with or any errors, but stdout(.old) should go back far enough to see what boinc version and dribver Iwas running:
Boinc 6.10.18 and Nvidia 195.62. (while still showing 6.08 tasks) in february
upgraded to 6.10.36, still running 6.08 - but of course with no idea whther it had not fallen back tp the CPU already.
and then with the upgrade to 197.16 and the reported memory of 242 it didn't get more 6.08 tasks - though it took me a while to realize that that was the cause...
dll are from 11/09 and look like 2.1 - no 2.2 in the details tab.
I tend to run latest recommended and newest driver...
I'll have a look for backups on Monday, but I'don't think I ever upgraded to 2.2 dll - if that had to be done manually almost certainly not.
The 2.2 I found in place are most likely from the .36 installer...
-
One thing I've noticed from the results you've reported recently (pending (http://setiathome.berkeley.edu/results.php?hostid=5200185&offset=0&show_names=0&state=2), valid (http://setiathome.berkeley.edu/results.php?hostid=5200185&offset=0&show_names=0&state=3)) is that your elapsed time seems to be ~2.5 times your CPU time, for both CPU and (nominal) GPU tasks. That ratio feels high, even for three tasks (2 x CPU, 1 x fallback) running on two cores. Back in May (main board message 1009028 (http://setiathome.berkeley.edu/forum_thread.php?id=60301&nowrap=true#1009028)), elapsed and CPU timings were within 400 seconds. Feels like something is eating CPU cycles, more than just browsing would do. Anything in Task Manager?
-
@Raistmer context creation? sorry you lost me - you want me to check GPU usage under different scenarios, to see what influences the components have? ok, I'll try.
btw ther's a 'H' missing in your 'IMO' ;)
LoL, that opinion wasn't "Humble" one it seems ;D
In addition some BOINC versions made check of free GPU memory. And not only at startup. So, if driver version fallbackwill not help, BOINC version fallback still worth to try.
Your priority should be to get stable GPU processing w/o fallback mode and then, 2.2 or 2.3 CUDA DLLs versions if possible. It will give nice GPU speed boost, much bigger then any new drivers or BOINCs.
-
I thought I mentioned I was using TThrottle to keep the temepratures down. (and I still think its running way too hot)
Now with CPU only it is at about 50-60%, when the GPU generates extra heat they both get around 40% - would have to check the graphs after inactivity, doesn't seem to get logged.
this task (http://setiathome.berkeley.edu/result.php?resultid=1661726168) ran partly on 6.08 and on V12 - no fallback. :)
I can give the 2.2 dll another try.
-
In addition some BOINC versions made check of free GPU memory. And not only at startup. So, if driver version fallbackwill not help, BOINC version fallback still worth to try.
Your priority should be to get stable GPU processing w/o fallback mode and then, 2.2 or 2.3 CUDA DLLs versions if possible. It will give nice GPU speed boost, much bigger then any new drivers or BOINCs.
IIRC, when that code is active, BOINC just suspends the GPU app entirely, and periodically retests to see if the free GPU memory available has gone back up to a high enough level to allow the app to run again. Shouldn't crash anything, certainly can't influence the internal workings of the app to induce CPU fallback. And I don't think it would work under Anonymous Platform anyway. BOINC would need to know how much free VRAM the app requires: it can't get that from the server under A_P, and we've never tested or advised putting it into app_info.xml (and it's not in the specifications (http://boinc.berkeley.edu/wiki/Anonymous_platform) for A_P, though I suspect the documentation is incomplete: Gary Roberts at Einstein has certainly got more to work than the documentation says is supported).
-
Richard, could you confirm that BOINC itself doesn't take GPU memory?
If not all details when and how it does anything not matter much. If GPU memory reduced it can influence on science app.
-
Logging shows no GPU memory takeup when Boinc starts or comes out of initial delay. Doesn't have decimals, though. When the GPU task starts it goes from 19 to 89, 106,210 used and some 15 seconds later to 218.
V12nokill and 2.1 dll (are you sure they are 2.1 and not 2.0 Richard? They don't have a version number on the details tab.) boinc 6.10.58 nvidia driver 195.62
-
@ Raistmer - I can't confirm that BOINC takes no CUDA memory - you and Jason have a better grip on the code. Heck, you're been compiling and releasing it practically since we dicovered VLAR together, with Jord's NB, back at the transition from v6.07 to v6.08 in January 2009. All I can say is that BOINC has no business in, and nothing to be gained by using, any of the GPU memory area. No science in BOINC! BOINC is management and control only. It needs to know about the GPU - memory, speed, capabilities etc. - but all its local storage surely goes in system RAM with the eqivalent knowledge about CPU capabilities.
@ Carola - no, I'm not sure what the exact designation of the antediluvian DLLs is. There's a set downloadable from the project fanout since about December 2008, which NVidia didn't see fit to label. The next set anybody noticed was definitely 2.2: Jason tested them, and they were better. Then came 2.3: I tested them, and they were better still. Certainly, nobody registered a switch from 2.0 to 2.1: my download archives don't have separate sets. When I get back home to a proper rig (tomorrow or Monday), I can do a proper search of the NVidia toolkit archives, extract the DLLs, and do some proper research: if there's anything between stock 'cuda' and 2.2 and stock 'cuda23', I'll send it over for testing.
-
So I tried the 2.2 dll again - as previously with V12: it errors out. I let it through: this task (http://setiathome.berkeley.edu/result.php?resultid=1661917457) for closer inspection, but I'd say it just tries to fall back to CPU and can't.
I can try higher dlls (if somebody tells me where to get them) but don't they need even more memory?
Any thoughts on whether to try to upgrade to newest driver?
-
So I tried the 2.2 dll again - as previously with V12: it errors out. I let it through: this task (http://setiathome.berkeley.edu/result.php?resultid=1661917457) for closer inspection, but I'd say it just tries to fall back to CPU and can't.
But that stderr_txt shows you're using Raistmer's code - and I thought we'd established that the CPU fallback was broken on that build? The question would be whether you can get the STOCK v6.08 app into app_info, and run it with the 2.2 DLLs. That would give better results than Raistmer's app with 2.1 DLLs, and still have a working fallback if things go wrong.
I can try higher dlls (if somebody tells me where to get them) but don't they need even more memory?
Any thoughts on whether to try to upgrade to newest driver?
I'm afraid both newer DLLs, and newer drivers, are likely to outface your limited memory - unless you can squeeze more RAM out of the BIOS.
-
So I tried the 2.2 dll again - as previously with V12: it errors out. I let it through: this task (http://setiathome.berkeley.edu/result.php?resultid=1661917457) for closer inspection, but I'd say it just tries to fall back to CPU and can't.
But that stderr_txt shows you're using Raistmer's code - and I thought we'd established that the CPU fallback was broken on that build? The question would be whether you can get the STOCK v6.08 app into app_info, and run it with the 2.2 DLLs. That would give better results than Raistmer's app with 2.1 DLLs, and still have a working fallback if things go wrong.
And I thought we had established that 6.08 with 2.2 does CPU fallback...
But ok, I'll swap the 6.08 and the 2.2 back in. next one up is a shorty so not too painfull for fallback...
I can try higher dlls (if somebody tells me where to get them) but don't they need even more memory?
Any thoughts on whether to try to upgrade to newest driver?
I'm afraid both newer DLLs, and newer drivers, are likely to outface your limited memory - unless you can squeeze more RAM out of the BIOS.
I'm not prepared to touch the BIOS while I need this machine.
197.16 was fine running what was there, just didn't get new tasks as per reduced reported memory. I think that might partly answer the question regarding Boinc checking GPU ram requirement.
I could maybe try removing app_info and look what boinc, the application and the server think about 6.08 with 2.1 and 2.2 respectively.
I'll probably have a go at the newest driver in an idle moment. I can always downgrade again.
Edit: forum doesn't seem to cope very well with multiple quotes, sorry.
-
So, swapped in 6.08 and 2.2dll over night. [url-=http://setiathome.berkeley.edu/result.php?resultid=1661917453]task[/url] CPU fallback as before. Can't comment on the runtime, nothing available for comparison. Times for the 2 CPU tasks went through the roof. well actually I think they are about 2x the previous average, consistent with CPU fallback grabbing one core and the two CPU tasks having to share the other.
Right, now I get it. The incresed speed of V12 is not primarily due to the application, but to the fact, that the installer supplies 2.2 or 2.3 dlls... Nice work ;)
-
AFAIK stock plan class cuda23 supplies 2.3 DLLs as well.
-
AFAIK stock plan class cuda23 supplies 2.3 DLLs as well.
Certainly does, but also checks for driver version (>= 190.38, IIRC) and VRAM (384MB) before it'll issue @ cuda23.
-
I recently got some WUs with Out Of Memory errors, like this one here:
http://setiathome.berkeley.edu/result.php?resultid=1666139183
I never had this before. Are the 512MB of my GT240 not enough any more?
-
It's just "-12" error. It was discussed many times look SETI main forums for details.
"out of memory" is "out of host not GPU memory" and don't ask why C++ runtime library says that on exception handling ;D
-
Richard? Care to throw the 2.3 dll's my way? Not that I think they will work, but then I will haved tested that...
Yes I know, the v36 installer has them sonewhere, but I don't want to write over the current setup
Edit: nevermind, I realized I can just backup, install, extract the files and put the backup back in...
-
Richard? Care to throw the 2.3 dll's my way? Not that I think they will work, but then I will haved tested that...
Can do when I get home, but don't have a set here. Makes sense for you to have then in a package for local swapping in/out as you test, anyway.
-
Richard? Care to throw the 2.3 dll's my way? Not that I think they will work, but then I will haved tested that...
Yes I know, the v36 installer has them sonewhere, but I don't want to write over the current setup
Edit: nevermind, I realized I can just backup, install, extract the files and put the backup back in...
The 2.3 dll's are in downloads: Windows Setiathome apps (http://lunatics.kwsn.net/index.php?module=Downloads;catd=9)
Claggy
-
The 2.3 dll's are in downloads: Windows Setiathome apps (http://lunatics.kwsn.net/index.php?module=Downloads;catd=9)
Claggy
Much easier, thanks. (of course 'warum einfach, wenns auch kompliziert geht' - why easy, when you can do it complicated)
I've upgraded to driver 258.96 and after sorting out the hiccups (had reverted to 32bit colours) it looks like base memory usage is unchanged and preliminary results suggest app is running fine.
Intrestingly max mem usage seems to be down to 201 from previous 218.
-
Richard? Care to throw the 2.3 dll's my way? Not that I think they will work, but then I will haved tested that...
Yes I know, the v36 installer has them sonewhere, but I don't want to write over the current setup
I've put them all up on my public SkyDrive SETI@home stuff (http://cid-8d83bf774a4a86f5.office.live.com/browse.aspx/SETI@home%20stuff) folder. I'll find 3.1 and add it when I've sorted myself out a bit. Later - v31.9 DLLs added to the same location - took them from Jason's public beta download, so blame him of they don't work. :P
-
Heads up guys - tidy the place up a bit, we've got a professional in our midst ;D
V12nokill and 2.1 dll (are you sure they are 2.1 and not 2.0 Richard? They don't have a version number on the details tab.) boinc 6.10.58 nvidia driver 195.62
You're quite right to call me out on that one - the original (and probably still current) v6.08 Berkeley download supplies 2.0 DLLs
I've updated SkyDrive, getting the number right this time, and adding genuine 2.1 DLLs. I don't remember anybody testing anything between the original release, and 2.2 (which were certainly tested, and a big improvement) - we were too busy getting the darn thing to run at all, any which way. 2.1 look to be close in size to 2.2, so might have some improvements over 2.0 - but no idea how much memory they need to run, or what the improvement might be. Miep, care to give them a whirl?
In researching the above, I came across http://developer.nvidia.com/object/cuda_archive.html - version numbers with release dates, and link through to the matching toolkit and driver downloads. That's gone onto my favourites list for future reference.
-
Heads up guys - tidy the place up a bit, we've got a professional in our midst ;D
You're quite right to call me out on that one - the original (and probably still current) v6.08 Berkeley download supplies 2.0 DLLs
I've updated SkyDrive, getting the number right this time, and adding genuine 2.1 DLLs. I don't remember anybody testing anything between the original release, and 2.2 (which were certainly tested, and a big improvement) - we were too busy getting the darn thing to run at all, any which way. 2.1 look to be close in size to 2.2, so might have some improvements over 2.0 - but no idea how much memory they need to run, or what the improvement might be. Miep, care to give them a whirl?
In researching the above, I came across http://developer.nvidia.com/object/cuda_archive.html - version numbers with release dates, and link through to the matching toolkit and driver downloads. That's gone onto my favourites list for future reference.
Bah, again nothing on the details page. how are you supposed to tell them apart, aprt from the datestamp?! *mutter*
Professional? Nice joke. I only pay attention, and the stock dlls were refered to as 2.0 on the SETI boards. As I said nothing on the details tab to indicate version number.
So, dropped the new 2.1 in. I see less maximum RAM usage on the new 258.96 driver, so I'll revisit the higher numbers.
EDIT: If we get verification from Chelski on SETI NC about his being a mobile GPU there might be justification for adding 2.0 dlls to the installer...
-
Bah, again nothing on the details page. how are you supposed to tell them apart, aprt from the datestamp?! *mutter*
This shambles was largely due to the original Cuda DLLs being only 'weakly versioned'. Probably due to customer support situations growing, and in part from people swapping around DLLs arbitrarily prior to Cuda 3: nVidia have moved to a 'strongly versioned' model from 3.0 onwards, include the version in the name, and applications built with one Cuda SDK are meant to use particular library versions. That and some other growing pain issues seem to indicate that early in Cuda's development, the scale of the idea itself may have been underestimated.
As far as getting the memory requirements down in the applications goes, with Cuda 3.1 there's a few options I'm looking at to remove use of the hefty CUFFT library, and reduce the minimum footprint. Will be some time before that can be realised in working form though, so this investigation is valuable toward many users.
Jason
-
Oh wow.
So, the 2.1 dll works (202M max, one more than 2.0) - and probably thanks to the new driver, so does the 2.2, with a memory max of 250MB. That's a memory usage of 230MB above baseline.
(max memiory usage by 2.0 dropped by 17 MB with 258.96 vs 195.62 driver)
I'll try higher ones, but with a margin of 6MB left, it's a very small chance they work - the 2.3 at least. Have the memory requirements of 3.0 and 3.1 been tested?
-
Oh wow.
So, the 2.1 dll works (202M max, one more than 2.0) - and probably thanks to the new driver, so does the 2.2, with a memory max of 250MB. That's a memory usage of 230MB above baseline.
(max memiory usage by 2.0 dropped by 17 MB with 258.96 vs 195.62 driver)
I'll try higher ones, but with a margin of 6MB left, it's a very small chance they work - the 2.3 at least. Have the memory requirements of 3.0 and 3.1 been tested?
Not the memory requirements, no - I think most of the testers who have chimed in are running 512MB or higher.
There are special problems with running the 3.0 or 3.1 DLLs with the older v6.08, v6.09 or V12 applications, because of the 'strong versioning' introduced with 3.0, as Jason mentioned. The apps expect DLLs called 'cudart.dll' and 'cufft.dll', but the fft.dll has to have access to 'cudart32_30_14.dll' or 'cudart32_31_9.dll'. You end up having to have three CUDA DLLs, not two, with the 'rt' file copied and renamed (and listed under both names in both sections of app_info.xml.
-
Not the memory requirements, no - I think most of the testers who have chimed in are running 512MB or higher.
There are special problems with running the 3.0 or 3.1 DLLs with the older v6.08, v6.09 or V12 applications, because of the 'strong versioning' introduced with 3.0, as Jason mentioned. The apps expect DLLs called 'cudart.dll' and 'cufft.dll', but the fft.dll has to have access to 'cudart32_30_14.dll' or 'cudart32_31_9.dll'. You end up having to have three CUDA DLLs, not two, with the 'rt' file copied and renamed (and listed under both names in both sections of app_info.xml.
Sorry, you lost me on that one. I was going to take the 'easy' approach and rename the files to cut off the version number instead of editing app_info.
For the record 2.3 dll runs and goes to 250MB as well - isn't that one supossed to use more RAM? Going to give it the full over night run and deal with the next generation tomorrow.
-
That's the point. The "easy" approach fails after 2.3 - we found that out the hard way when 3.0 came out. I can talk you through it, but too late for both of us tonight.
I don't think anyone really "knows" that 2.3 requires more VRAM than 2.2, 2.1 or 2.0: it's been asserted, but without evidence. If they run overnight, that'll be good news.
-
That's the point. The "easy" approach fails after 2.3 - we found that out the hard way when 3.0 came out. I can talk you through it, but too late for both of us tonight.
I don't think anyone really "knows" that 2.3 requires more VRAM than 2.2, 2.1 or 2.0: it's been asserted, but without evidence. If they run overnight, that'll be good news.
Well, can only report what I see on my system - and since the 2.2/2.3 only work since the newest driver (258.96) wouldn't know if memory uptake on older drivers is different. They certainly all require less memory to run than before. (well the 2.0/2.2 do)
-
That ran like a treat. And the increase in speed with the higher dlls is remarkable.
Of the 3 tasks that ran O/N 1 has validated and 2 are waiting for wingman. (for reference: valid (http://setiathome.berkeley.edu/result.php?resultid=1666587544) pending (http://setiathome.berkeley.edu/result.php?resultid=1661919151) pending (http://setiathome.berkeley.edu/result.php?resultid=1661919063))
Is there anything in stderr to check which dlls were used?
-
Is there anything in stderr to check which dlls were used?
I'm afraid not.
Since 2.3 there was not such big speed improvement so probably with 2.3 DLLs you reached peak GPU performance until V13/CUDA 3.1 app will be introduced to public.
-
Is there anything in stderr to check which dlls were used?
I'm afraid not.
Since 2.3 there was not such big speed improvement so probably with 2.3 DLLs you reached peak GPU performance until V13/CUDA 3.1 app will be introduced to public.
Thanks, much obliged.
I thought there wasn't, since I couldn't see anything but you can always have 'tomatoes on your eyes' - I'm not awake enough to translate german proverbs.
I'll throw the 3.0 dll in there anyway, if only to check the memory uptake.
-
That ran like a treat. And the increase in speed with the higher dlls is remarkable.
Of the 3 tasks that ran O/N 1 has validated and 2 are waiting for wingman. (for reference: valid (http://setiathome.berkeley.edu/result.php?resultid=1666587544) pending (http://setiathome.berkeley.edu/result.php?resultid=1661919151) pending (http://setiathome.berkeley.edu/result.php?resultid=1661919063))
Is there anything in stderr to check which dlls were used?
I just realised you're running Vista (That'll teach me to look properly, and make proper connections ::)). The newer series of drivers are Microsoft's WDDM driver model based, which uses a paged scheme for memory management where each application sees its own full address range. That is opposed to XP drivers physical video memory model which was prone to fragmentation & shared the video memory as one block. IMO with the new drivers under Vista you *should* have no low memory issues running any of the newest builds (Though experimental Cuda 3.0 & 3.1 builds still to be determined ...*hint* *hint*)
Jason
-
I just realised you're running Vista (That'll teach me to look properly, and make proper connections ::)). The newer series of drivers are Microsoft's WDDM driver model based, which uses a paged scheme for memory management where each application sees its own full address range. That is opposed to XP drivers physical video memory model which was prone to fragmentation & shared the video memory as one block. IMO with the new drivers under Vista you *should* have no low memory issues running any of the newest builds (Though experimental Cuda 3.0 & 3.1 builds still to be determined ...*hint* *hint*)
Jason
Hmm good point. I was going to say I tried V12 on 257.21, but now I'm not sure. Will have to check the logs. Edit: I didn't. Only 195.62 and 258.98 got tested with the variuos dll's (3.x still pending).
One thing after the other Jason... no need to pull out fenceposts for waving ;)
-
If GPU memory paging occurs you should not only check is it works or not but elapsed/cpu times too.
GPU memory swap could be performance killer (or not ).
-
Todays O/N run just got reserved for Jason's *hint* *hint* fencepost.
daytime and O/N run Wu runtimes are not very comparable on my system. daytime will see a number of restarts, depending on how much idle time arises, O/N can run uniterreupted.
Also cooling and resulting throttling are different at work and at home. (the office has A/C but the gap between notebook and table is smaller)
I can note down elapsed, cpu and AR on the diffrent builds, dlls and environments, if you want me to. (and any other easily accessible parameter)
-
So, for the record: V12 works with 3.0 (insert memory load here) but errors out with 3.1 (redo and make sure it's a memory issue)