CUDA and SLI

[pp]

Hmm, I added

Code: [Select]

Option "MultiGPU" "1"to my xorg.conf to enable SLI on my GTX295 and BOINC immediately aborted 1842 CUDA-jobs because it didn't find any CUDA devices. Wasn't this problem fixed long ago? I'm using 190.36.
/PP

[sunu]

Wasn't this problem fixed long ago? I'm using 190.36.
/PP

What problem, boinc's or nvidia's? For boinc I would say it's standard procedure, if it can't find a graphics card, it aborts all GPU-assigned tasks.

As for nvidia, when I first got my GTX295, I experimented with various sli options in xorg.conf. When sli was enabled I strangely had much worse desktop performance than with sli disabled but I still could do cuda tasks in one core of course (back then, with sli enabled you could use only one core for cuda, recent drivers supposedly enable you to run two cuda tasks in GTX295 even with sli enabled). Because of the worse performance I've never bothered with sli since.

[pp]

That was my understanding too but in my case Boinc 6.10.13 didn't find ANY cuda-devices when I had SLI enabled... I'll just leave it off for now.

This is what I got in Xorg.0.log... couldn't find any errors there.

Code: [Select]

(**) NVIDIA(0): Option "Coolbits" "1"
(**) NVIDIA(0): Option "MultiGPU" "1"
(**) Oct 09 23:25:02 NVIDIA(0): Enabling RENDER acceleration
(**) Oct 09 23:25:02 NVIDIA(0): NVIDIA Multi-GPU auto-select rendering option.
(II) Oct 09 23:25:02 NVIDIA(0): Support for GLX with the Damage and Composite X extensions is
(II) Oct 09 23:25:02 NVIDIA(0):     enabled.
(II) Oct 09 23:25:05 NVIDIA(0): NVIDIA Multi-GPU enabled.
(II) Oct 09 23:25:05 NVIDIA(0): NVIDIA GPU GeForce GTX 295 (GT200) at PCI:5:0:0 (GPU-0)
(--) Oct 09 23:25:05 NVIDIA(0): Memory: 917504 kBytes
(--) Oct 09 23:25:05 NVIDIA(0): VideoBIOS: 62.00.6c.00.01
(II) Oct 09 23:25:05 NVIDIA(0): Detected PCI Express Link width: 16X
(--) Oct 09 23:25:05 NVIDIA(0): Interlaced video modes are supported on this GPU
(--) Oct 09 23:25:05 NVIDIA(0): Connected display device(s) on GeForce GTX 295 at PCI:5:0:0:
(--) Oct 09 23:25:05 NVIDIA(0):     HP LP2465 (DFP-1)
(--) Oct 09 23:25:05 NVIDIA(0): HP LP2465 (DFP-1): 330.0 MHz maximum pixel clock
(--) Oct 09 23:25:05 NVIDIA(0): HP LP2465 (DFP-1): Internal Dual Link TMDS
(II) Oct 09 23:25:05 NVIDIA(0): NVIDIA GPU GeForce GTX 295 (GT200) at PCI:4:0:0 (GPU-1)
(--) Oct 09 23:25:05 NVIDIA(0): Memory: 917504 kBytes
(--) Oct 09 23:25:05 NVIDIA(0): VideoBIOS: 62.00.6c.00.02
(II) Oct 09 23:25:05 NVIDIA(0): Detected PCI Express Link width: 16X
(--) Oct 09 23:25:05 NVIDIA(0): Interlaced video modes are supported on this GPU
(--) Oct 09 23:25:05 NVIDIA(0): Connected display device(s) on GeForce GTX 295 at PCI:5:0:0:
(--) Oct 09 23:25:05 NVIDIA(0):     HP LP2465 (DFP-1)

[sunu]

That was my understanding too but in my case Boinc 6.10.13 didn't find ANY cuda-devices when I had SLI enabled... I'll just leave it off for now.

That could mean either a boinc problem (couldn't see the graphics card, even though it was there) or an nvidia driver problem that couldn't load when you had sli enabled.

[pp]

Added an Xorg.0.log snippet in my previous post. It said Multi-GPU enabled and X started without problems... no big deal. I took the opportunity to try it while I was upgrading Gnome to 2.26.

[sunu]

So probably the GPU was working ok. It would be good if we had the startup messages from boinc for that session.

[pp]

Hmm, didn't save it. Just saw the regular message about no CUDA enabled devices found and despite the fact that I had suspended the project before I started boinc it still aborted my whole cache.... don't really feel upto trying it again right now. Still filling up the cache anew 

[pp]

Ah, it was still in the old log.

Code: [Select]

09-Oct-2009 23:27:39 [---] Starting BOINC client version 6.10.13 for x86_64-pc-linux-gnu
09-Oct-2009 23:27:39 [---] Config: use all coprocessors
09-Oct-2009 23:27:39 [---] log flags: file_xfer, sched_ops, task
09-Oct-2009 23:27:39 [---] Libraries: libcurl/7.19.6 OpenSSL/0.9.8k zlib/1.2.3 libidn/1.15
09-Oct-2009 23:27:39 [---] Running as a daemon
09-Oct-2009 23:27:39 [---] Data directory: /home/pp/BOINC
09-Oct-2009 23:27:39 [---] Processor: 8 GenuineIntel Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz [Family 6 Model 26 Stepping 5]
09-Oct-2009 23:27:39 [---] Processor: 8.00 MB cache
09-Oct-2009 23:27:39 [---] Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology tsc_reliable nonstop_tsc pni dtes64 monitor ds_cpl
09-Oct-2009 23:27:39 [---] OS: Linux: 2.6.31-gentoo
09-Oct-2009 23:27:39 [---] Memory: 5.84 GB physical, 6.01 GB virtual
09-Oct-2009 23:27:39 [---] Disk: 843.72 GB total, 777.00 GB free
09-Oct-2009 23:27:39 [---] Local time is UTC +2 hours
09-Oct-2009 23:27:39 [---] No usable GPUs found
09-Oct-2009 23:27:39 [SETI@home] Found app_info.xml; using anonymous platform
09-Oct-2009 23:27:40 [SETI@home] Missing coprocessor for task 28au09ab.30475.4162.13.10.149_2; aborting
09-Oct-2009 23:27:40 [SETI@home] Missing coprocessor for task 28au09aa.19943.2930.15.10.122_2; aborting
09-Oct-2009 23:27:40 [SETI@home] Missing coprocessor for task 11au09aa.22533.11928.8.10.94_1; aborting
09-Oct-2009 23:27:40 [SETI@home] Missing coprocessor for task 11au09aa.22533.11928.8.10.92_0; aborting
09-Oct-2009 23:27:40 [SETI@home] Missing coprocessor for task 11au09aa.22533.11928.8.10.86_0; aborting
09-Oct-2009 23:27:40 [SETI@home] Missing coprocessor for task 11au09aa.22533.11928.8.10.82_1; aborting
09-Oct-2009 23:27:40 [SETI@home] Missing coprocessor for task 11au09aa.22533.11928.8.10.80_1; aborting
<BIG SNIP>


[sunu]

Well boinc couldn't see your GPU but still could be either boinc's or nvidia's fault. You could try:
a) I see you run boinc in daemon mode. I always run it through run_client. Try it to see if it makes a difference.
b) With sli enabled try to run seti standalone with --device 0 and --device 1 to see if it works.

[pp]

This post and the fix listed under CUDA makes me wonder if there perhaps was a regression in 190.x...

[Claggy]

This post and the fix listed under CUDA makes me wonder if there perhaps was a regression in 190.x...

Fixed the link, it has a " at the end.

Claggy

[sunu]

That is a windows driver. If you want to experiment, try the two things I posted above.

[pp]

Yes, I know it was the Windows driver. Not having any insight into nvidia's developing process, I just assumed all the different platform drivers comes from the same code base. I'm not in the experimental mode atm unless someone smells a bug here and would benefit from me testing this more thoroughly.
Thanks Claggy for fixing the link. I've corrected it in the original post. Not used to forum software that don't use quotation marks in their url bbcode...

[sunu]

Yes, the code base is more or less the same.

Apart from the problem with cuda, did you use your pc with sli enabled for a while? How was the performance? When I got my GTX295, I tried sli and the performance was much much worse.

[pp]

No, this was the very first time I enable SLI on the card and I never got around to try any games since I wanted Boinc up and running again. I would assume AFR mode would give the best performance but this was something I wanted to try since I didn't even know what mode was activated when set to "auto". Judging from one of the Q&A:s in the README, the mode selection might be dynamic depending on what game is running and this detection seems to fail sometimes. Don't know if that's related to your experience.


Q. Why is Doom 3 slower when SLI or Multi-GPU is enabled?

A. The NVIDIA Accelerated Linux Graphics Driver does not automatically detect
   the optimal SLI or Multi-GPU settings for games such as Doom 3 and Quake 4.
   To work around this issue, the environment variable __GL_DOOM3 can be set
   to tell OpenGL that Doom 3's optimal settings should be used. In Bash, this
   can be done in the same command that launches Doom 3 so the environment
   variable does not remain set for other OpenGL applications started in the
   same session:
   
       % __GL_DOOM3=1 doom3
   
   Doom 3's startup script can also be modified to set this environment
   variable:
   
       #!/bin/sh
       # Needed to make symlinks/shortcuts work.
       # the binaries must run with correct working directory
       cd "/usr/local/games/doom3/"
       export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:.
       export __GL_DOOM3=1
       exec ./doom.x86 "$@"
   
   This environment variable is temporary and will be removed in the future.