Seti@Home optimized science apps and information

Optimized Seti@Home apps => Linux => Topic started by: Simon on 25 Jun 2006, 06:18:50 pm

Title: How to make your own optimized Seti@Home client for Linux
Post by: Simon on 25 Jun 2006, 06:18:50 pm: Hi,

you can find this How-To in the Articles section - here's a direct link:

How to make your own optimized Seti@Home client for Linux (http://lunatics.at/index.php?op=Articles;article=2).

Please post about your success or failure using it!

Good luck,
Simon
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: Jocker on 17 Aug 2006, 12:39:18 pm: Hi,

I've tried to build the Seti@Home client myself, but the Intel C Compiler did not supported GCC 4.1.
Yesterday i've seen that there is a new version of the compiler and i've loaded it.
But there is no more a fftw2mkl. So how to build with Intel's new compiler the Client?
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: Simon on 17 Aug 2006, 05:29:30 pm: Jocker,

the Linux How-Tos need an update - MKL is not really necessary (I found out it doesn't alter speed at all).

Still, the source package and build scripts that are currently online still include references to both, which would be too much to list here.

Instead, I'll be updating the source packages this weekend, and please have patience until then - I've had much better build results with gcc-3.3/3.4 than 4.0, by the way, though it might not be so easy to switch.

Regards,
Simon.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: ngdias on 28 Aug 2006, 07:37:12 am: Somewhere on your guide you mention:

"Note: all these instructions (as well as the scripts) are tuned for 32-bit Linux. You will have to edit them for 64-bit Linux (especially the compile-boinc-* and compile-seti-* scripts)."

I'm an Ubuntu 6.06 64-bit user and I have very little experience in compiling software. Could you please add to your guide the required instructions for 64bit?

Another 3 questions:
1. I have a Pentium D, so I guess the Intel compiler is adequate. But the AMD users should use it too?

2. If the SETI team decides to upgrade the client, what happens if I'm using yours, which is modified?

3. I think Pentium D supports SSE3, can I use this in ./compile-boinc-xxxx? Or should I use EM64T? This question also extends to ' CFLAGS="-march=pentium4 -msse2 -mfpmath=sse ${CFLAGS}" '. What should I have instead of 'pentium4'?
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: Simon on 28 Aug 2006, 12:07:46 pm: Hi ngdias,

yes, the Intel compiler is appropriate to use for AMD owners, as well, as it produces the fastest code for PCs around - even though AMD CPUs may be slightly slower than Intels using these versions, they are still quicker than if you use any other compiler/library package.

When the S@H team decides to release a new app, I will release one as well (based on the official sources). As with all other optimized apps so far, you will have to update yours manually. Sorry, there is and will be no auto-update of the applications, that is way outside the scope of my activities :) Still, I may release a separate app that tries and finds out what version you're using and whether there is a more recent one available. Will take time.

Yes, you can create a new compile script that uses SSE3 - however, you will have to do more than just copy it to a new name - specifically, you will need to edit the compile script and change all occurrences of sse2 to sse3 as well as change "/xW" or "/xN" to "/xP". Please consult the ICC man pages for explanations on what these options really do :)

I will release updated Linux sources and update the How-To as well to reflect these changes, so either do it yourself now or be patient, please :)

As for 64-bit compilation, I really cannot give you hints there as I haven't been able to devote much time to it yet.

Regards,
Simon.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: ngdias on 28 Aug 2006, 04:29:08 pm: Thanks for your reply. I'll wait for your update to see how it's done... I hope you will be able to do some more testing in 64bit too.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: Simon on 03 Sep 2006, 04:49:33 pm: Hi folks,

I finally had time to update the BOINC & SETI@Home Linux sources and build scripts. So, the sources archive in the Downloads section now points to the new file. These sources include build scripts for MMX-SSE3 with all flavours in between, the checkpoint() fix as well as some minor tweaks in the IPP department.

HTH,
Simon.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: sancio on 20 Dec 2006, 03:16:48 pm: Hi, I tried to compile 1.3 source but unsuccessfully.
My system:
Pentium 4 3GHz
Slackware 10.1
ICC 9.1.045
IPP beta 5.2.011

Only missing requisite libcurl>=7.13.2, I have 7.12.2.

boinc compile fine with compile-boinc-sse3-p4 adding the option --disable-client to the configure because of the libcurl
Could be this the error?
The client is needed or are enough the library?

For seti_boinc I used compile-seti-sse3-p4 not modified with the exception of the icc/ipp path (in attach)
I tried too the configure modify that you suggest for SSE2, but with the same results with the exception of this warning:
icpc: Command line warning: ignoring unknown option '-mfpmath=sse'

The errors are in attach, some example:

icpc: Command line warning: ignoring unknown option '-finclude'
/home/gtoso/src/kwsn/boinc/lib/util.h(66): remark #981: operands are evaluated i
n unspecified order
s.size()>=suffix.size() &&
[...]
../db/track_mem.h(31): remark #193: zero used for undefined preprocessing identi
fier
#if DEBUG_ALLOCATIONS
^
[...]
icpc: Command line warning: no action performed for specified file(s)
[...]
sincos.h(44): error: omission of exception specification is incompatible with pr
evious function "sincos" (declared at line 82 of "/usr/include/bits/mathcalls.h"
)
void sincos(double angle, double *s, double *c);
^

Where I mistake?
Thanks,
Gianluca.

[attachment deleted by admin]
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: sancio on 23 Dec 2006, 01:51:42 pm: I retried to compile with new libcurl, but unsuccessfully >:(
boinc compile fine (client too), but seti no.
It's a Slack problem? Or IPP 5.2 beta? I need IPP 5.1 or old?
May be I'll retry with IPP 5.1...
But if you can help me it's better ;D

Thanks,
goodbye.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: Simon on 23 Dec 2006, 10:11:38 pm: Hi Gianluca,

I've had those errors often as well. I usually "fixed" them, if nothing else helped, by editing the respective #ifdefs and #defines that redefine sincos and the other mathematical functions.

However, Michael Buckingham (http://lunatics.at/index.php?action=viewprofile;user=MichaelB) has offered to help porting the 2.0 sources to Linux, so maybe we will soon have new stuff to compile anyway ;)

Merry Christmas (http://lunatics.at/index.php/topic,155.msg2003/topicseen.html#msg2003)!
Simon.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: sancio on 24 Dec 2006, 12:21:01 pm: Thanks for the hint Simon and Merry Christmas to you too!!!
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: talaktalan on 24 Dec 2006, 10:03:46 pm: Hello and Merry Christmas!

Did anyone sucessfully install the IPP libraries on Ubuntu 6.10?

I was able to install the ICC compiler (*) but the installation of the IPP libs fails. The workaround for the MKL libs, which is given in the Ubunto knowledgebase, seems not to work for the IPP libs. Anyone had sucess here?

Thanks for any help.

Best regards

Alex

(*) With a few changes to the install scripts: /bin/bash instead of /bin/sh and alien instead of rpm.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: sancio on 25 Dec 2006, 10:07:25 pm: Quote from: Simon on 23 Dec 2006, 10:11:38 pm
Hi Gianluca,

I've had those errors often as well. I usually "fixed" them, if nothing else helped, by editing the respective #ifdefs and #defines that redefine sincos and the other mathematical functions.

Thanks,
Finally the compilation has gone successfully ;D

I commented line 44-48 of client/sincos.h
and added "tools/Makefile" to AC_CONFIG_FILES in configure.ac
(otherwise tools/Makefile is not updated and compile fail).
However I doubt that I need fakedata...

My first binary (SSE3/P4 ICC 9.1.045 IPP beta 5.2.011) seem to crunch fine.
In my tests it's the quickest with the exception of Josefs-WU where KWSN SSE2 is better.

For curiosity I will try to compile a SSE2 client in order to see if the improvements depend on the new version of ICC/IPP or on SSE3.

Code: [Select]
Current WU: testWU-4 --------------------------------------------------- Testing seti-sse3-p4... Running S@H client seti-sse3-p4.... Time for seti-sse3-p4: 261.35user 5.83system 4:54.54elapsed 90%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (1major+375869minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing crunch3r-sse3... Running S@H client crunch3r-sse3.... Time for crunch3r-sse3: 295.70user 3.71system 5:25.84elapsed 91%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (15major+375575minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing setiathome-5.15-kwsn-1.3-sse2-p4... Running S@H client setiathome-5.15-kwsn-1.3-sse2-p4.... Time for setiathome-5.15-kwsn-1.3-sse2-p4: 270.68user 3.81system 4:56.97elapsed 92%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (29major+20977minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing default-512... Running S@H client default-512.... Time for default-512: 500.46user 2.34system 9:33.83elapsed 87%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (34major+8620minor)pagefaults 0swaps --------------------------------------------------- Current WU: testWU-1 --------------------------------------------------- Testing seti-sse3-p4... Running S@H client seti-sse3-p4.... Time for seti-sse3-p4: 2713.52user 22.77system 50:32.40elapsed 90%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (49major+1448335minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing crunch3r-sse3... Running S@H client crunch3r-sse3.... Time for crunch3r-sse3: 3005.21user 22.43system 55:49.68elapsed 90%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (27major+1448340minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing setiathome-5.15-kwsn-1.3-sse2-p4... Running S@H client setiathome-5.15-kwsn-1.3-sse2-p4.... Time for setiathome-5.15-kwsn-1.3-sse2-p4: 2676.01user 14.39system 53:07.93elapsed 84%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (59major+40139minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing default-512... Running S@H client default-512.... Time for default-512: 4431.06user 20.63system 1:21:13elapsed 91%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (49major+13157minor)pagefaults 0swaps --------------------------------------------------- Current WU: testWU-6 --------------------------------------------------- Testing seti-sse3-p4... Running S@H client seti-sse3-p4.... Time for seti-sse3-p4: 48.45user 2.99system 0:54.73elapsed 93%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (47major+34454minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing crunch3r-sse3... Running S@H client crunch3r-sse3.... Time for crunch3r-sse3: 69.82user 0.39system 1:14.21elapsed 94%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (20major+34192minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing setiathome-5.15-kwsn-1.3-sse2-p4... Running S@H client setiathome-5.15-kwsn-1.3-sse2-p4.... Time for setiathome-5.15-kwsn-1.3-sse2-p4: 52.51user 2.16system 0:58.44elapsed 93%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (47major+15527minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing default-512... Running S@H client default-512.... Time for default-512: 93.22user 0.47system 1:59.16elapsed 78%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (5major+8383minor)pagefaults 0swaps --------------------------------------------------- Current WU: Josefs-WU --------------------------------------------------- Testing seti-sse3-p4... Running S@H client seti-sse3-p4.... Time for seti-sse3-p4: 483.40user 5.47system 8:50.17elapsed 92%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+161341minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing crunch3r-sse3... Running S@H client crunch3r-sse3.... Time for crunch3r-sse3: 542.71user 3.34system 9:52.80elapsed 92%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (10major+161055minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing setiathome-5.15-kwsn-1.3-sse2-p4... Running S@H client setiathome-5.15-kwsn-1.3-sse2-p4.... Time for setiathome-5.15-kwsn-1.3-sse2-p4: 463.96user 4.41system 8:27.15elapsed 92%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (32major+17910minor)pagefaults 0swaps --------------------------------------------------- --------------------------------------------------- Testing default-512... Running S@H client default-512.... Time for default-512: 802.40user 3.43system 14:44.12elapsed 91%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (49major+8768minor)pagefaults 0swaps ---------------------------------------------------
Code: [Select]
gtoso@marte:~/src/kwsn/seti_boinc/client/test/testData$ ../rescmp result.sah.default-512.Josefs-WU result.sah.seti-sse3-p4.Josefs-WU Result: these are strongly similar. gtoso@marte:~/src/kwsn/seti_boinc/client/test/testData$ ../rescmp result.sah.default-512.testWU-1 result.sah.seti-sse3-p4.testWU-1 Result: these are strongly similar. gtoso@marte:~/src/kwsn/seti_boinc/client/test/testData$ ../rescmp result.sah.default-512.testWU-4 result.sah.seti-sse3-p4.testWU-4 Result: these are strongly similar. gtoso@marte:~/src/kwsn/seti_boinc/client/test/testData$ ../rescmp result.sah.default-512.testWU-6 result.sah.seti-sse3-p4.testWU-6 Result: these are strongly similar. gtoso@marte:~/src/kwsn/seti_boinc/client/test/testData$
Quote from: Simon on 23 Dec 2006, 10:11:38 pm
However, Michael Buckingham (http://lunatics.at/index.php?action=viewprofile;user=MichaelB) has offered to help porting the 2.0 sources to Linux, so maybe we will soon have new stuff to compile anyway ;)

Very good news,
you tell me if I can help even if my C is much rusting.

Goodbye and sorry for my bad English :-(
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: Simon on 25 Dec 2006, 10:16:02 pm: Hi Gianluca,

tutto bene ;)

Happy you got it to compile, good work!

Regards,
Simon.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: sancio on 25 Dec 2006, 10:21:44 pm: Quote from: Sancio on 25 Dec 2006, 10:07:25 pm

My first binary (SSE3/P4 ICC 9.1.045 IPP beta 5.2.011) seem to crunch fine.
In my tests it's the quickest with the exception of Josefs-WU where KWSN SSE2 is better.

Emmh in testWU-1 is quicker KWSN, too.
I'm confused for the presence of elapsed time and not real time (as in bash builtin time)
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: michael37 on 25 Dec 2006, 11:31:54 pm: Simon,

Does the SSE3 variant uses the QxT (-xT) optimization flag? If yes, could you please update the Compiler options & seti #defines for optimiztion (http://lunatics.at/index.php?PHPSESSID=609410870520cdc99cddf26a121071a6&topic=52.msg628) document. If not, we need an option that uses the xT :)

In addition, I found this odd line in the Intel compiler guide: Use /QxW /QaxT (-xW –axT) to include other Intel 64 and like AMD* processors as well.. Did any experiment with these flags?

Thanks.

P.S. I started running the version 1.3 SSE2-optimized code on my dual-Xeon 5160 and it's surprisingly slow (~10,000 seconds for 0.4 ar/60 credit unit, 4 threads in parallel). Granted, this may be due to general slowness of dual-51XX family vs single socket Core 2 Duos (more info here (http://setiathome.berkeley.edu/forum_thread.php?id=36040)), but it has to be due to insufficiently optimized code.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: sancio on 26 Dec 2006, 11:21:43 am: Hi,
ICC is a frustrating compiler >:(
Your executable is static, my dynamic.
The -static flag cause me headache:
checking size of long int... configure: error: cannot compute sizeof (long int), 77

This is strange:
(conftest.cc is a piece of code copied from config.log that return the size of long int into file conftest.val)
Same error with flag -fast

Code: [Select]
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ icpc -o conftest -no-sox -O3 -pc64 -xP -axP -fp-model fast -no-prec-div -no-prec-sqrt -ipo4 -I/opt/intel/cc/9.1.045/include -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -I/usr/include -I/usr/include/openssl -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static-libcxa -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib conftest.cc -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -Wl,-Bdynamic -ldl -Wl,-Bdynamic -lm /usr/lib/libnsl.a -Wl,-Bdynamic -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -Wl,-Bdynamic -lpthread -Wl,-Bdynamic -lc -lippcore -lippsmerged IPO: performing single-file optimizations IPO: generating object file /tmp/ipo_icpcw7cWoK.o gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ./conftest gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ file ./conftest ./conftest: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ldd ./conftest linux-gate.so.1 => (0xffffe000) libimf.so => /opt/intel/cc/9.1.045/lib/libimf.so (0xb7dad000) libsvml.so => /opt/intel/cc/9.1.045/lib/libsvml.so (0xb7d4d000) libssl.so.0 => /usr/lib/libssl.so.0 (0xb7d15000) libcrypto.so.0 => /usr/lib/libcrypto.so.0 (0xb7c11000) libdl.so.2 => /lib/libdl.so.2 (0xb7c0d000) libm.so.6 => /lib/libm.so.6 (0xb7bea000) librt.so.1 => /lib/librt.so.1 (0xb7bd7000) libpthread.so.0 => /lib/libpthread.so.0 (0xb7b84000) libc.so.6 => /lib/libc.so.6 (0xb7a67000) libirc.so => /opt/intel/cc/9.1.045/lib/libirc.so (0xb7a26000) /lib/ld-linux.so.2 (0xb7feb000) gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ icpc -o conftest -no-sox -O3 -pc64 -xP -axP -fp-model fast -no-prec-div -no-prec-sqrt -ipo4 -I/opt/intel/cc/9.1.045/include -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -I/usr/include -I/usr/include/openssl -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static-libcxa -static -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib conftest.cc -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -Wl,-Bdynamic -ldl -Wl,-Bdynamic -lm /usr/lib/libnsl.a -Wl,-Bdynamic -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -Wl,-Bdynamic -lpthread -Wl,-Bdynamic -lc -lippcore -lippsmerged IPO: performing single-file optimizations IPO: generating object file /tmp/ipo_icpchnSplV.o gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ file ./conftest ./conftest: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ldd ./conftest /usr/bin/ldd: line 124: ./conftest: No such file or directory gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ./conftest -bash: ./conftest: No such file or directory gtoso@marte:~/src/kwsn/kwsn/seti_boinc$

[attachment deleted by admin]
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: Simon on 26 Dec 2006, 11:26:11 am: Hi,

I had exactly the same problem trying to use -static.

What I did to get the released executables to be static -

I used "statifier" (http://statifier.sourceforge.net/) with the --force-execve switch.

HTH,
Simon.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: Simon on 26 Dec 2006, 11:31:22 am: Quote from: michael37 on 25 Dec 2006, 11:31:54 pm
Simon,

Does the SSE3 variant uses the QxT (-xT) optimization flag? If yes, could you please update the Compiler options & seti #defines for optimiztion (http://lunatics.at/index.php?PHPSESSID=609410870520cdc99cddf26a121071a6&topic=52.msg628) document. If not, we need an option that uses the xT :)

In addition, I found this odd line in the Intel compiler guide: Use /QxW /QaxT (-xW –axT) to include other Intel 64 and like AMD* processors as well.. Did any experiment with these flags?
Yup - /QxT and /QaxT are for SSSE3 (Supplemental Streaming SIMD 3 Extensions), meaning they only run (and on Linux, only compile, for me) on Core 2-based systems.

Support for these was added in the 5.1.1 IPP version and the 9.1.028+ ICC one.

HTH,
Simon.

P.S.: I know the 1.3 code isn't that quick on Core 2s - guess why I put the 1.41 up instead? ;)
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: sancio on 26 Dec 2006, 11:52:30 am: Quote from: Simon on 26 Dec 2006, 11:26:11 am
Hi,

I had exactly the same problem trying to use -static.

What I did to get the released executables to be static -

I used "statifier" (http://statifier.sourceforge.net/) with the --force-execve switch.

Thanks very much ;D
But I'm not convinced...
it must be exist a clean way in order to obtain a real static binary,
than in theory it would have to also be faster.

But perhaps at this time it does not have much sense to strive on 1.3.
Some progress on 2.0?

PS: The same problems are taken place under Windows?
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: sancio on 26 Dec 2006, 03:03:11 pm: Quote from: Sancio on 26 Dec 2006, 11:21:43 am
Hi,
ICC is a frustrating compiler >:(
Your executable is static, my dynamic.
The -static flag cause me headache:
checking size of long int... configure: error: cannot compute sizeof (long int), 77

Code: [Select]
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ icpc -o conftest -no-sox -O3 -pc64 -xP -axP -fp-model fast -no-prec-div -no-prec-sqrt -ipo4 -I/opt/intel/cc/9.1.045/include -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -I/usr/include -I/usr/include/openssl -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static-libcxa -static -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib conftest.cc -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -Wl,-Bdynamic -ldl -Wl,-Bdynamic -lm /usr/lib/libnsl.a -Wl,-Bdynamic -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -Wl,-Bdynamic -lpthread -Wl,-Bdynamic -lc -lippcore -lippsmerged

Mmm...if I remove all -Wl,-Bdynamic it compile fine and static...
Now I think:
It's configure that insert in Makefiles this linker option,
why?
Is it wrong? Or is it's a ICC bug that should ignore it in a static compile?
Boh...

Maybe that removing this option from all Makefiles static compile works, but it's dirty.

I hope that this can be of useful.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: Simon on 26 Dec 2006, 03:17:47 pm: Thanks for the hint, I'll try that next time I compile on Linux.

I've long been annoyed that I couldn't compile a real static binary as-is, and agree it's most probably a configure problem (I don't think -static worked as is with GCC either, though I'm not sure).

The real solution would be to modify configure.ac and change the macros there, I believe.

Regards,
Simon.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: michael37 on 27 Dec 2006, 11:23:57 pm: Quote from: Simon on 26 Dec 2006, 11:31:22 am
Yup - /QxT and /QaxT are for SSSE3 (Supplemental Streaming SIMD 3 Extensions), meaning they only run (and on Linux, only compile, for me) on Core 2-based systems.

Support for these was added in the 5.1.1 IPP version and the 9.1.028+ ICC one.

HTH,
Simon.

P.S.: I know the 1.3 code isn't that quick on Core 2s - guess why I put the 1.41 up instead? ;)

Regarding 1.41 for Linux -- could you please make it so I can see the pre-released forum/downloads? I am not a builder (besides I use CentOS which is prohibitive to building Boinc), but I can test stuff. I've worked with Harold in the past.

Regarding -xT flag -- -xP also includes SSE3 instructions, but runs on single core EM64T Xeons. Most of my computers have these processors. You no longer have -xP Linux application, so I use -xN version. I've heard that benchmarks are pretty much the same on -xP and -xN versions.

Regarding -xW –axT flag -- that's not -xT -axT. It sounds like the code with -xW -axT flags should run on SSE3 Opterons and Athlon64s. Unfortunately, I no longer have one of these.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: sancio on 28 Dec 2006, 08:05:16 am: Quote from: Simon on 26 Dec 2006, 03:17:47 pm
Thanks for the hint, I'll try that next time I compile on Linux.

I've long been annoyed that I couldn't compile a real static binary as-is, and agree it's most probably a configure problem (I don't think -static worked as is with GCC either, though I'm not sure).

The real solution would be to modify configure.ac and change the macros there, I believe.

I think so, too.
I have done some tests:

I tried to manually link only seti_boinc with -static or to add -static to Makefiles removing any -Wl,-Bdynamic.
It produce these errors:

Code: [Select]
IPO Warning: unresolved : __intel_cpu_indicator Referenced in libimf.a(sqrt_stub.o) Referenced in libimf.a(sincos_stub.o) Referenced in libimf.a(exp_stub.o) Referenced in libimf.a(ceil_stub.o) Referenced in libimf.a(floor_stub.o) Referenced in libimf.a(log10_stub.o) Referenced in libimf.a(log_stub.o) Referenced in libimf.a(pow_stub.o) Referenced in libimf.a(atanf_stub.o) Referenced in libimf.a(cosf_stub.o) Referenced in libimf.a(sincosf_stub.o) Referenced in libimf.a(fmod_stub.o) Referenced in libimf.a(sin_stub.o) Referenced in libimf.a(cos_stub.o) IPO Warning: unresolved : __intel_cpu_indicator_init Referenced in libimf.a(sqrt_stub.o) Referenced in libimf.a(sincos_stub.o) Referenced in libimf.a(exp_stub.o) Referenced in libimf.a(ceil_stub.o) Referenced in libimf.a(floor_stub.o) Referenced in libimf.a(log10_stub.o) Referenced in libimf.a(log_stub.o) Referenced in libimf.a(pow_stub.o) Referenced in libimf.a(atanf_stub.o) Referenced in libimf.a(cosf_stub.o) Referenced in libimf.a(sincosf_stub.o) Referenced in libimf.a(fmod_stub.o) Referenced in libimf.a(sin_stub.o) Referenced in libimf.a(cos_stub.o) IPO: performing multi-file optimizations IPO: generating assembly file /tmp/icpc0Ktz4Jas_.s analyzeFuncs.cpp(849) : (col. 3) remark: LOOP WAS VECTORIZED. [...] /tmp/ipo_icpcMvqdAO.o(.text+0x1013a6): In function `lookup_group(char*, unsigned int&)': : warning: Using 'getgrnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /opt/intel/cc/9.1.045/lib/libimf.a(sincos_stub.o)(.text+0x2): In function `sincos': : undefined reference to `__intel_cpu_indicator' /opt/intel/cc/9.1.045/lib/libimf.a(sincos_stub.o)(.text+0x12): In function `sincos': : undefined reference to `__intel_cpu_indicator' /opt/intel/cc/9.1.045/lib/libimf.a(sincos_stub.o)(.text+0x21): In function `sincos': : undefined reference to `__intel_cpu_indicator_init' [...]
libirc contains __intel_cpu_indicator_init definition:
Code: [Select]
gtoso@marte:~/src/kwsn/seti_boinc/client$ nm /opt/intel/cc/9.1.045/lib/libirc.a |grep "T __intel_cpu_indicator_init" 00000184 T __intel_cpu_indicator_init gtoso@marte:~/src/kwsn/seti_boinc/client$ nm /opt/intel/cc/9.1.045/lib/libirc.so |grep "T __intel_cpu_indicator_init" 000114dc T __intel_cpu_indicator_init
Adding -lirc it seems to compile fine and static, I have not yet tested it.
However IPO warnings remain and it seems to depend on glibc version used to compile :-[
Maybe some source modification are needed...

Code: [Select]
gtoso@marte:~/src/kwsn/seti_boinc/client$ icpc -o seti_boinc -pthread -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -lirc -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib seti_boinc-main.o seti_boinc-analyzeFuncs.o seti_boinc-analyzeReport.o seti_boinc-analyzePoT.o seti_boinc-pulsefind.o seti_boinc-gaussfit.o seti_boinc-lcgamm.o seti_boinc-malloc_a.o seti_boinc-seti.o seti_boinc-seti_header.o seti_boinc-timecvt.o seti_boinc-s_util.o seti_boinc-version.o seti_boinc-worker.o seti_boinc-chirpfft.o seti_boinc-spike.o seti_boinc-progress.o seti_boinc-fft8g.o seti_boinc-gdata.o seti_boinc-schema_master.o seti_boinc-sqlrow.o seti_boinc-sqlblob.o seti_boinc-xml_util.o -L/home/gtoso/src/kwsn/boinc/api -lboinc_api -L/home/gtoso/src/kwsn/boinc/lib -lboinc -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -ldl -lm /usr/lib/libnsl.a -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -lpthread -lc -lippcore -lippsmerged IPO Warning: unresolved : __intel_cpu_indicator Referenced in libimf.a(sqrt_stub.o) Referenced in libimf.a(sincos_stub.o) Referenced in libimf.a(exp_stub.o) Referenced in libimf.a(ceil_stub.o) [...] fft8g.cpp(196) : (col. 5) remark: LOOP WAS VECTORIZED. /tmp/ipo_icpcg81LiJ.o(.text+0x1013a6): In function `lookup_group(char*, unsigned int&)': : warning: Using 'getgrnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking gtoso@marte:~/src/kwsn/seti_boinc/client$ file seti_boinc seti_boinc: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped gtoso@marte:~/src/kwsn/seti_boinc/client$ ldd seti_boinc not a dynamic executable gtoso@marte:~/src/kwsn/seti_boinc/client$
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: Simon on 28 Dec 2006, 11:28:06 am: Good work!

I'll try and incorporate this into the How-To and get the next versions compiled as fully static binaries.

Thanks for your help!
Simon.
Title: Re: How to make your own optimized Seti@Home client for Linux
Post by: sancio on 28 Dec 2006, 02:42:00 pm: Quote from: Simon on 28 Dec 2006, 11:28:06 am
Good work!

Thanks 8)
I hope that someone will succeed to make it better and in clean way.

Quote from: Simon on 28 Dec 2006, 11:28:06 am
I'll try and incorporate this into the How-To and get the next versions compiled as fully static binaries.

In attachment all my modifies.

Quote from: Simon on 28 Dec 2006, 11:28:06 am
Thanks for your help!
Simon.

Thanks to you and to all people that have contributed in any ways
and thanks to Alex Kan for the great mac client!
Gianluca.

[attachment deleted by admin]