SETI MB CUDA for Linux

[Kunin]

I can confirm the above, I tried it with 6.6.36, 6.6.20 and 6.4.5 and ONLY 6.4.5 worked correctly (one WU to one GPU).  Both 6.6.20 and 6.6.36 put both on the same device, and 6.6.36 I had to do funky things to get it working (had to install 6.6.20, run it, shut down, install 6.6.36 without removing anything, start 6.6.36... and repeat that any time I shut it down, VERY odd).

It could just be something with our OS, Ubuntu 8.10 x64, but for now 6.4.5 works... now I just need to find some way to get the CPU to do some work.

[Josef W. Segur]

If this has already been posted, please ignore, but the BOINC developers for some reason decided to use only the most capable GPU by default. Before that they had something which would use more than 1 only if they were identical, etc.

Anyhow, if you have more than one GPU and want BOINC 6.6.25 or later to use all, you need a cc_config.xml with the option <use_all_gpus>1</use_all_gpus>.
                                                                             Joe

[riofl]

If this has already been posted, please ignore, but the BOINC developers for some reason decided to use only the most capable GPU by default. Before that they had something which would use more than 1 only if they were identical, etc.

Anyhow, if you have more than one GPU and want BOINC 6.6.25 or later to use all, you need a cc_config.xml with the option <use_all_gpus>1</use_all_gpus>.
                                                                             Joe

I had that in cc_config since I installed the 2nd card, but it made no difference in how the 6.6.x boinic feeds the devices workunits. depending on the config you can feed 2 wu to the 1st device, or feed 1 wu to the 2nd device with --device 0 --device 1 on the cmdline. it is incapable of feeding one wu to each device the way the code behaves. In 6.6.20 it properly identified both devices and displayed such in the messages window but still reported to seti 2 of the 2nd device and did not properly feed the devices workunits. in 6.6.36 i noticed they changed it to display devices one per line and it broke causing it to report 2 of the 2nd device. .  but still had the same behavior as all the other 6.6.x in feeding devices.
it doesn't.
6.4.5 also reports 2 of the 2nd device but it feeds both devices properly.

[Kunin]

I'm sticking with 6.4.5 for now, combined with the CPU_GPU_rebrand_V5.pl script (on the forums here) I get all the functionality (and a little more) that I wanted out of 6.6.36, so that's good enough for now.  Later, when I have some free time, I want to take a look at the 6.6.36 code and see if I can't figure out why it's screwing up.

[skip da shu]

Was hoping somebody here might help a noob at this.

I downloaded the setiathome-CUDA-6.08.x86_64.tar.bz2 file and extracted it to my desktop. 
I copied all the files in the cudalibs folder to /usr/lib64.
I copied the setiathome-CUDA-6.08.x86_64-pc-linux-gnu and app_info.xml to /var/lib/boinc-client

I ran the ldd:

Code: [Select]

\\skip@c17-desktop:/var/lib/boinc-client/projects/setiathome.berkeley.edu$ ldd setiathome-CUDA-6.08.x86_64-pc-linux-gnu
linux-vdso.so.1 =>  (0x00007fff69fff000)
libcufft.so.2 => /usr/lib/libcufft.so.2 (0x00007f16618de000)
libcudart.so.2 => /usr/lib/libcudart.so.2 (0x00007f16616a0000)
libcuda.so.1 => /usr/lib/libcuda.so.1 (0x00007f16611ff000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f1660ef2000)
libm.so.6 => /lib/libm.so.6 (0x00007f1660c6d000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007f1660a51000)
libc.so.6 => /lib/libc.so.6 (0x00007f16606df000)
libdl.so.2 => /lib/libdl.so.2 (0x00007f16604db000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f16602c3000)
librt.so.1 => /lib/librt.so.1 (0x00007f16600bb000)
libz.so.1 => /lib/libz.so.1 (0x00007f165fea3000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1661bf8000)

I restarted BOINC.

GPUgrid is running/working on this machine (Ubuntu v9.04, 64b) with BOINC v6.4.5 from getdeb.

I get these messages:

Tue 21 Jul 2009 09:44:52 PM CDT||file projects/setiathome.berkeley.edu/libcudart.so.2 not found
Tue 21 Jul 2009 09:44:52 PM CDT||file projects/setiathome.berkeley.edu/libcufft.so.2 not found
Tue 21 Jul 2009 09:44:52 PM CDT||[error] No URL for file transfer of libcudart.so.2
Tue 21 Jul 2009 09:44:52 PM CDT||[error] No URL for file transfer of libcufft.so.2

and BOINC xfer tab shows libcudart.so.2 and libcufft.2

Can anyone explain to me what I've missed?


For now I've removed the app_info.xml and aborted the xfers of the libcu* files

[Raistmer]

Apparently BOINC can't find 2 required libraries.
try to put these 3 libs with executable. Not sure Linux  checks that place in attempt to load shared libs though...

[riofl]

easiest way is to simply copy everything from the sdk lib dir into /var/lib/boinc and also into the project directory /var/lib/boinc/projects/<projectname>/

also depending on what your distro requires, add the projectname directory path to /etc/ld.so.conf or to whatever master file your distro says to and re-run ldconfig or env-update or whatever you have to scan the lib locations.

it will magically start working. my seti project dir contains these lib files:

lrwxrwxrwx 1 root wheel       17 2009-07-17 13:05 libcublasemu.so -> libcublasemu.so.2
lrwxrwxrwx 1 root wheel       19 2009-07-17 13:05 libcublasemu.so.2 -> libcublasemu.so.2.2
-rwxr-xr-x 1 root wheel  4704664 2009-07-09 08:00 libcublasemu.so.2.2
lrwxrwxrwx 1 root wheel       14 2009-07-17 13:06 libcublas.so -> libcublas.so.2
lrwxrwxrwx 1 root wheel       16 2009-07-17 13:05 libcublas.so.2 -> libcublas.so.2.2
-rwxr-xr-x 1 root wheel 18643744 2009-07-09 08:00 libcublas.so.2.2
lrwxrwxrwx 1 root wheel       14 2009-07-17 13:06 libcudart.so -> libcudart.so.2
lrwxrwxrwx 1 root wheel       16 2009-07-17 13:05 libcudart.so.2 -> libcudart.so.2.2
-rwxr-xr-x 1 root wheel   261472 2009-07-09 08:00 libcudart.so.2.2
lrwxrwxrwx 1 root wheel       16 2009-07-17 13:06 libcufftemu.so -> libcufftemu.so.2
lrwxrwxrwx 1 root wheel       18 2009-07-17 13:05 libcufftemu.so.2 -> libcufftemu.so.2.2
-rwxr-xr-x 1 root wheel   273064 2009-07-09 08:00 libcufftemu.so.2.2
lrwxrwxrwx 1 root wheel       13 2009-07-17 13:06 libcufft.so -> libcufft.so.2
lrwxrwxrwx 1 root wheel       15 2009-07-17 13:06 libcufft.so.2 -> libcufft.so.2.2
-rwxr-xr-x 1 root wheel  1153440 2009-07-09 08:00 libcufft.so.2.2


if you are not using the 2.2 cuda environment you will not have some of these files but no matter. as long as all files from the sdk lib dir are copied in it will work.

[riofl]

I can confirm the above, I tried it with 6.6.36, 6.6.20 and 6.4.5 and ONLY 6.4.5 worked correctly (one WU to one GPU).  Both 6.6.20 and 6.6.36 put both on the same device, and 6.6.36 I had to do funky things to get it working (had to install 6.6.20, run it, shut down, install 6.6.36 without removing anything, start 6.6.36... and repeat that any time I shut it down, VERY odd).

It could just be something with our OS, Ubuntu 8.10 x64, but for now 6.4.5 works... now I just need to find some way to get the CPU to do some work.

i'm using the AK optimized MB app for my cpus and with this plus the cuda app in my app_info.xml my cpus and gpus are all happily crunching away. i don't know where it would be on this site but if it is not here i can give you a link to crunch3r's download page where i got mine.  my app name is

AK_V8_linux64_ssse3

they are highly optimized for the processor so be sure you get the correct one. a "cat /proc/cpuinfo" will get you the processor capabilities.

[sunu]

After a catastrophic GPU failure (probably something burned inside my GTX280  )  that left me offline for a week, I'm back up and slowly catching up.

Since many linux users, lately, had problems with multi-GPU configs, I decided my next card would be a GTX 295 so I have a proper testbed to research things further. My various tweaks and experiments with xorg.conf or different video drivers didn't have any effect. Boinc still used only one of the two GPUs.

Then I started with the latest boinc (6.6.37) and moved backwards to see if I could find a boinc version that worked as expected.

I've found that the last boinc that can use both GPUs properly is 6.6.11. Boinc 6.6.12 doesn't even assign --device X flags to cuda instances. It just launches two of them and they (probably automatically) go straight to the first GPU. The next boinc linux release (6.6.15) corrects the missing --device X flags but it does this wrongly, it assigns both of them to --device 0.

So something in the transition 6.6.11 --> 6.6.12 --> 6.6.15 broke and stayed broken ever since with all subsequent boinc releases.

I'm using 6.6.11 for a few hours now with no apparent problems or bugs. Direct link http://boincdl.ssl.berkeley.edu/dl/boinc_6.6.11_x86_64-pc-linux-gnu.sh .

If someone remembers any showstopper bugs with 6.6.11 please tell.

[Richard Haselgrove]

If someone remembers any showstopper bugs with 6.6.11 please tell.

That was just before the bombshell about not supporting app_info.xml properly. v6.6.12 doesn't aupport app_info, v6.6.14 started to put it back together (there was no .13). So you may have problems getting v6.6.11 working with both CUDA and an optimised CPU app, or indeed Astropulse.

[riofl]

I just tried 6.6.11. it recognizes both devices just fine and properly i might add. however, i can only assume since i have a very large backlog of uploads, it refuses to do anything. both cuda processes are "waiting to run" and it does not load the gpu apps. with the upload clogs the way they are this is not acceptable so i went back to 6.4.5 to at least finish the existing backlog of waiting wu i have.

after things are caught up (if that ever happens ) i will try it again.

[sunu]

That was just before the bombshell about not supporting app_info.xml properly. v6.6.12 doesn't aupport app_info, v6.6.14 started to put it back together (there was no .13). So you may have problems getting v6.6.11 working with both CUDA and an optimised CPU app, or indeed Astropulse.

Now that you mention it, I think all the newly downloaded wus went to cuda 6.08. No assignments to 6.03. Maybe it didn't ask?  . When did boinc started supporting both 6.03 and 6.08 concurrently?

[Richard Haselgrove]

Now that you mention it, I think all the newly downloaded wus went to cuda 6.08. No assignments to 6.03. Maybe it didn't ask?  . When did boinc started supporting both 6.03 and 6.08 concurrently?

I'd need to check if you need a detailled answer, but I think:

v6.6.1 for stock applications
v6.6.14 with app_info.xml

Edit: v6.4.5 allows you to specify and run both apps, but will only fetch for the higher-numbered one.

[sunu]

I'd need to check if you need a detailled answer, but I think:

v6.6.1 for stock applications
v6.6.14 with app_info.xml

Edit: v6.4.5 allows you to specify and run both apps, but will only fetch for the higher-numbered one.

If you could find when boinc started to fetch for both 6.03 and 6.08, I would be very grateful.

I'll keep monitoring 6.6.11 to see how it behaves. If it can't fetch for 6.03, I'll have to do it manually till a fixed boinc is released. I'll try to report the bug in boinc_alpha and I hope I'll get some attention.

[riofl]

I'd need to check if you need a detailled answer, but I think:

v6.6.1 for stock applications
v6.6.14 with app_info.xml

Edit: v6.4.5 allows you to specify and run both apps, but will only fetch for the higher-numbered one.

If you could find when boinc started to fetch for both 6.03 and 6.08, I would be very grateful.

I'll keep monitoring 6.6.11 to see how it behaves. If it can't fetch for 6.03, I'll have to do it manually till a fixed boinc is released. I'll try to report the bug in boinc_alpha and I hope I'll get some attention.

because of my upload backlog nothing gets anything, but once that is cleared up if 6.4.5 only gets for 6.03 which was its behavior before, then all i wil do is use 6.6.37 to get 4 or 5 days worth of units then switch back to 6.4.5 to properly process them. pain but at least it works. that will be after i try 6.6.11 to see what it does.