+- +-
Say hello if visiting :) by Gecko
11 Jan 2023, 07:43:05 pm

Seti is down again by Mike
09 Aug 2017, 10:02:44 am

Some considerations regarding OpenCL MultiBeam app tuning from algorithm view by Raistmer
11 Dec 2016, 06:30:56 am

Loading APU to the limit: performance considerations by Mike
05 Nov 2016, 06:49:26 am

Better sleep on Windows - new round by Raistmer
26 Aug 2016, 02:02:31 pm

Author Topic: Does anyone use multi gpu on one machine on linux?  (Read 10263 times)

guildwar

  • Guest
Does anyone use multi gpu on one machine on linux?
« on: 22 Mar 2009, 08:55:10 am »
I have a machine with one cuda device, and run setiathome CUDA for linux perfectly

Recently, I plug another cuda device which the same with the previous one
when run boinc, it say I have two cuda device, so far so good
but I notice,  the job's cpu time is not the same, from about 300 seconds to  1600 seconds
284.96     13.79     13.79
to
1,640.06     13.79     13.79

and after check, I found setiathome use cpu instead of gpu to crunch  the workunit

ps ax|grep seti
4311 pts/1    RNLl   9:42 setiathome-CUDA-6.08.x86_64-pc-linux-gnu --device 0
4312 pts/1    RNLl   9:42 setiathome-CUDA-6.08.x86_64-pc-linux-gnu --device 0

two setiathome cuda use the same gpu
so, I change app_info.xml
<coproc>
<type>CUDA</type>
<count>2</count>
</coproc>
and restart boinc

ps ax | grep seti
4423 pts/1    RNLl   9:45 setiathome-CUDA-6.08.x86_64-pc-linux-gnu --device 0 --device 1

only show one instence of setiathome cuda, but whith two devices

but after I check result, the cpu time and walltime is the same with only one gpu
can somebody tell me why?

Offline Raistmer

  • Working Code Wizard
  • Volunteer Developer
  • Knight who says 'Ni!'
  • *****
  • Posts: 14349
Re: Does anyone use multi gpu on one machine on linux?
« Reply #1 on: 22 Mar 2009, 01:53:47 pm »
Singlu CUDA MB instance can use only one GPU device.
That is, you need run one copy with device 0 and second with device 1.
Not both with device 0 and not single with two devices.

guildwar

  • Guest
Re: Does anyone use multi gpu on one machine on linux?
« Reply #2 on: 22 Mar 2009, 02:01:59 pm »
Singlu CUDA MB instance can use only one GPU device.
That is, you need run one copy with device 0 and second with device 1.
Not both with device 0 and not single with two devices.


Thanks for your reply
But I don't know how to use two instence of MB cuda
How to write app_info.xml?

Offline Metod, S56RKO

  • Alpha Tester
  • Knight o' The Realm
  • ***
  • Posts: 51
Re: Does anyone use multi gpu on one machine on linux?
« Reply #3 on: 08 Jul 2009, 05:11:21 am »
Singlu CUDA MB instance can use only one GPU device.
That is, you need run one copy with device 0 and second with device 1.
Not both with device 0 and not single with two devices.


Thanks for your reply
But I don't know how to use two instence of MB cuda
How to write app_info.xml?

Any idea? I've got a dual-GPU GF9800 GX2 and I'm wondering if I can make it play nice ...

Offline sunu

  • Alpha Tester
  • Knight who says 'Ni!'
  • ***
  • Posts: 771
Re: Does anyone use multi gpu on one machine on linux?
« Reply #4 on: 08 Jul 2009, 05:28:38 am »
Any idea? I've got a dual-GPU GF9800 GX2 and I'm wondering if I can make it play nice ...

Have you tried it? What problems  do you have?

Offline Metod, S56RKO

  • Alpha Tester
  • Knight o' The Realm
  • ***
  • Posts: 51
Re: Does anyone use multi gpu on one machine on linux?
« Reply #5 on: 08 Jul 2009, 07:14:25 am »
Any idea? I've got a dual-GPU GF9800 GX2 and I'm wondering if I can make it play nice ...
Have you tried it? What problems do you have?

I'm trying .. hard ;-) It actually seems to me that I'm plagued with the same problem as described in this post: http://lunatics.kwsn.net/linux/seti-mb-cuda-for-linux.150.html#msg19175 - also Ubuntu 9.04 etc. I seem to recall similar error messages, but I can't check right now ... SAH server seems to have some functionality temporarily disabled. Anyway, here's the link to host's results: http://setiathome.berkeley.edu/results.php?hostid=4994295.

I was wondering if there's some special magic when machine's got more than one GPU...

Offline sunu

  • Alpha Tester
  • Knight who says 'Ni!'
  • ***
  • Posts: 771
Re: Does anyone use multi gpu on one machine on linux?
« Reply #6 on: 08 Jul 2009, 08:12:36 am »
I assume that you've setup everything correctly? Please check my posts carefully here and here. Do a ldd in your cuda app and post your xorg.conf and xorg.0.log.

Offline Metod, S56RKO

  • Alpha Tester
  • Knight o' The Realm
  • ***
  • Posts: 51
Re: Does anyone use multi gpu on one machine on linux?
« Reply #7 on: 10 Jul 2009, 09:21:08 am »
I assume that you've setup everything correctly? Please check my posts carefully here and here. Do a ldd in your cuda app and post your xorg.conf and xorg.0.log.

Here's the state of the matter:
  • I've compiled kernel 2.6.30.1, albeit using configuration from Ubuntu 9.04 stock kernel
  • I've installed (and made sure that driver gets used) nvidia 185.18.14 driver and cuda toolkit
  • I've installed latest BOINC client 6.6.36

I can't get seti CUDA to work. Example of errored WU is this: http://setiathome.berkeley.edu/result.php?resultid=1297997619

Quite the same stuff with a WU from GPUgrid.net ( http://www.gpugrid.net/result.php?resultid=947630 ).

Libraries:

Code: [Select]
$ ldd projects/setiathome.berkeley.edu/setiathome-CUDA-6.08.x86_64-pc-linux-gnu
linux-vdso.so.1 => (0x00007fffe0193000)
libcufft.so.2 => /usr/local/cuda/lib/libcufft.so.2 (0x00007f4fd1bd7000)
libcudart.so.2 => /usr/local/cuda/lib/libcudart.so.2 (0x00007f4fd1997000)
libcuda.so.1 => /usr/lib/libcuda.so.1 (0x00007f4fd14ca000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f4fd11bd000)
libm.so.6 => /lib/libm.so.6 (0x00007f4fd0f38000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007f4fd0d1c000)
libc.so.6 => /lib/libc.so.6 (0x00007f4fd09aa000)
libdl.so.2 => /lib/libdl.so.2 (0x00007f4fd07a6000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f4fd058e000)
librt.so.1 => /lib/librt.so.1 (0x00007f4fd0386000)
libz.so.1 => /lib/libz.so.1 (0x00007f4fd016e000)
/lib64/ld-linux-x86-64.so.2 (0x00007f4fd1ef2000)

$ ls -l /usr/local/cuda/lib/
total 24536
lrwxrwxrwx 1 root root 17 2009-07-06 14:41 libcublasemu.so -> libcublasemu.so.2
lrwxrwxrwx 1 root root 19 2009-07-06 14:41 libcublasemu.so.2 -> libcublasemu.so.2.2
-rwxr-xr-x 1 root root 4745456 2009-07-06 14:41 libcublasemu.so.2.2
lrwxrwxrwx 1 root root 14 2009-07-06 14:41 libcublas.so -> libcublas.so.2
lrwxrwxrwx 1 root root 16 2009-07-06 14:41 libcublas.so.2 -> libcublas.so.2.2
-rwxr-xr-x 1 root root 18684568 2009-07-06 14:41 libcublas.so.2.2
lrwxrwxrwx 1 root root 14 2009-07-06 14:41 libcudart.so -> libcudart.so.2
lrwxrwxrwx 1 root root 16 2009-07-06 14:41 libcudart.so.2 -> libcudart.so.2.2
-rwxr-xr-x 1 root root 261304 2009-07-06 14:41 libcudart.so.2.2
lrwxrwxrwx 1 root root 16 2009-07-06 14:41 libcufftemu.so -> libcufftemu.so.2
lrwxrwxrwx 1 root root 18 2009-07-06 14:41 libcufftemu.so.2 -> libcufftemu.so.2.2
-rwxr-xr-x 1 root root 272896 2009-07-06 14:41 libcufftemu.so.2.2
lrwxrwxrwx 1 root root 13 2009-07-06 14:41 libcufft.so -> libcufft.so.2
lrwxrwxrwx 1 root root 15 2009-07-06 14:41 libcufft.so.2 -> libcufft.so.2.2
-rwxr-xr-x 1 root root 1153304 2009-07-06 14:41 libcufft.so.2.2

Now, as GPUgrid app exhibits same error seems that it's some configuration of Ubuntu kernel that kicks in ... I'll re-compile kernel when I find some time (which will not happen in next couple of weeks).

'till then.

Offline sunu

  • Alpha Tester
  • Knight who says 'Ni!'
  • ***
  • Posts: 771
Re: Does anyone use multi gpu on one machine on linux?
« Reply #8 on: 10 Jul 2009, 10:46:53 am »
Please follow my post here to the letter

Kunin

  • Guest
Re: Does anyone use multi gpu on one machine on linux?
« Reply #9 on: 15 Jul 2009, 09:07:17 am »
I followed your post, but it's still only using one of my cards, I have two GTX 260 NOT in SLI.

I placed a copy of the cuda app in /bin, all the drivers are installed and using BOINC 6.6.20 (6.6.36 wouldn't work at all).

The one that is running says: "Running (0.04 CPUs, 2 CUDA)" for the Status.
« Last Edit: 15 Jul 2009, 09:09:59 am by Kunin »

Offline Richard Haselgrove

  • Messenger Pigeon
  • Knight who says 'Ni!'
  • *****
  • Posts: 2819
Re: Does anyone use multi gpu on one machine on linux?
« Reply #10 on: 15 Jul 2009, 09:59:23 am »

The one that is running says: "Running (0.04 CPUs, 2 CUDA)" for the Status.


Are you Joseph Monk at SETI?

I just answered an identical question on the main board:

Quote
Your app_info.xml file is faulty.

The fragment which reads

Code: [Select]
<coproc>
<type>CUDA</type>
<count>1</count>
</coproc>

(probably near the bottom) needs to be like that - unchanged - even when you have two cards. It's the number of CUDA cards needed per task - and none of the applications need more than one per task yet.

Kunin

  • Guest
Re: Does anyone use multi gpu on one machine on linux?
« Reply #11 on: 15 Jul 2009, 08:32:51 pm »

The one that is running says: "Running (0.04 CPUs, 2 CUDA)" for the Status.


Are you Joseph Monk at SETI?

I just answered an identical question on the main board:

Quote
Your app_info.xml file is faulty.

The fragment which reads

Code: [Select]
<coproc>
<type>CUDA</type>
<count>1</count>
</coproc>

(probably near the bottom) needs to be like that - unchanged - even when you have two cards. It's the number of CUDA cards needed per task - and none of the applications need more than one per task yet.

Double thanks, after I asked there I realized I got the app here so this would be a better place to ask.  It's working now... but something is odd as tasks now seem to take A LOT longer and only one of my GPUs are heating up.  Last night it ran with the 2 count and completed about 30 tasks in 3-8 minutes each, now it's running 2 at a time but taking hours.

Offline sunu

  • Alpha Tester
  • Knight who says 'Ni!'
  • ***
  • Posts: 771
Re: Does anyone use multi gpu on one machine on linux?
« Reply #12 on: 23 Jul 2009, 04:46:20 am »
Kunin, if you haven't seen it already, check http://lunatics.kwsn.net/linux/seti-mb-cuda-for-linux.msg19767.html#msg19767 and subsequent posts in that thread.

Kunin

  • Guest
Re: Does anyone use multi gpu on one machine on linux?
« Reply #13 on: 29 Jul 2009, 02:35:51 pm »
Thanks, been using 6.6.11 for a while.  Another user here and I have a long thread going on the SETI forums with our various attempts.

I believe I have located the problem in 6.6.36, but all attempts to compile my own client has failed (complains about curl, but curl is installed properly).

If you have any experience there let me know, I'd be more than happy to run a couple tests and confirm my suspicions then get it fixed and submit a patch back.

 

Welcome, Guest.
Please login or register.
 
 
 
Forgot your password?
Members
Total Members: 97
Latest: ToeBee
New This Month: 0
New This Week: 0
New Today: 0
Stats
Total Posts: 59559
Total Topics: 1672
Most Online Today: 21
Most Online Ever: 983
(20 Jan 2020, 03:17:55 pm)
Users Online
Members: 0
Guests: 15
Total: 15
Powered by EzPortal