How to make your own optimized Seti@Home client for Linux

[michael37]

Simon,

Does the SSE3 variant uses the QxT (-xT) optimization flag?  If yes, could you please update the Compiler options & seti #defines for optimiztion document.  If not, we need an option that uses the xT

In addition, I found this odd line in the Intel compiler guide: Use /QxW /QaxT (-xW –axT) to include other Intel 64 and like AMD* processors as well..  Did any experiment with these flags?

Thanks.

P.S. I started running the version 1.3 SSE2-optimized code on my dual-Xeon 5160 and it's surprisingly slow (~10,000 seconds for 0.4 ar/60 credit unit, 4 threads in parallel).  Granted, this may be due to general slowness of dual-51XX family vs single socket Core 2 Duos (more info here), but it has to be due to insufficiently optimized code.

[sancio]

Hi,
ICC is a frustrating compiler
Your executable is static, my dynamic.
The -static flag cause me headache:
checking size of long int... configure: error: cannot compute sizeof (long int), 77

This is strange:
(conftest.cc is a piece of code copied from config.log that return the size of long int into file conftest.val)
Same error with flag -fast

Code: [Select]

gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ icpc -o conftest -no-sox -O3 -pc64 -xP -axP -fp-model fast -no-prec-div -no-prec-sqrt -ipo4 -I/opt/intel/cc/9.1.045/include -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -I/usr/include -I/usr/include/openssl -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib  -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static-libcxa -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib conftest.cc  -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -Wl,-Bdynamic -ldl -Wl,-Bdynamic -lm /usr/lib/libnsl.a -Wl,-Bdynamic -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -Wl,-Bdynamic -lpthread -Wl,-Bdynamic -lc -lippcore -lippsmerged
IPO: performing single-file optimizations
IPO: generating object file /tmp/ipo_icpcw7cWoK.o
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ./conftest
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ file ./conftest
./conftest: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ldd ./conftest
        linux-gate.so.1 =>  (0xffffe000)
        libimf.so => /opt/intel/cc/9.1.045/lib/libimf.so (0xb7dad000)
        libsvml.so => /opt/intel/cc/9.1.045/lib/libsvml.so (0xb7d4d000)
        libssl.so.0 => /usr/lib/libssl.so.0 (0xb7d15000)
        libcrypto.so.0 => /usr/lib/libcrypto.so.0 (0xb7c11000)
        libdl.so.2 => /lib/libdl.so.2 (0xb7c0d000)
        libm.so.6 => /lib/libm.so.6 (0xb7bea000)
        librt.so.1 => /lib/librt.so.1 (0xb7bd7000)
        libpthread.so.0 => /lib/libpthread.so.0 (0xb7b84000)
        libc.so.6 => /lib/libc.so.6 (0xb7a67000)
        libirc.so => /opt/intel/cc/9.1.045/lib/libirc.so (0xb7a26000)
        /lib/ld-linux.so.2 (0xb7feb000)
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ icpc -o conftest -no-sox -O3 -pc64 -xP -axP -fp-model fast -no-prec-div -no-prec-sqrt -ipo4 -I/opt/intel/cc/9.1.045/include -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -I/usr/include -I/usr/include/openssl -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib  -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static-libcxa -static -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib conftest.cc  -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -Wl,-Bdynamic -ldl -Wl,-Bdynamic -lm /usr/lib/libnsl.a -Wl,-Bdynamic -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -Wl,-Bdynamic -lpthread -Wl,-Bdynamic -lc -lippcore -lippsmerged
IPO: performing single-file optimizations
IPO: generating object file /tmp/ipo_icpchnSplV.o
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ file ./conftest
./conftest: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ldd ./conftest
/usr/bin/ldd: line 124: ./conftest: No such file or directory
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ./conftest
 -bash: ./conftest: No such file or directory
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$


[attachment deleted by admin]

[Simon]

Hi,

I had exactly the same problem trying to use -static.

What I did to get the released executables to be static -

I used "statifier" (http://statifier.sourceforge.net/) with the --force-execve switch.

HTH,
Simon.

[Simon]

Simon,

Does the SSE3 variant uses the QxT (-xT) optimization flag?  If yes, could you please update the Compiler options & seti #defines for optimiztion document.  If not, we need an option that uses the xT

In addition, I found this odd line in the Intel compiler guide: Use /QxW /QaxT (-xW –axT) to include other Intel 64 and like AMD* processors as well..  Did any experiment with these flags?

Yup - /QxT and /QaxT are for SSSE3 (Supplemental Streaming SIMD 3 Extensions), meaning they only run (and on Linux, only compile, for me) on Core 2-based systems.

Support for these was added in the 5.1.1 IPP version and the 9.1.028+ ICC one.

HTH,
Simon.

P.S.: I know the 1.3 code isn't that quick on Core 2s - guess why I put the 1.41 up instead?

[sancio]

Hi,

I had exactly the same problem trying to use -static.

What I did to get the released executables to be static -

I used "statifier" (http://statifier.sourceforge.net/) with the --force-execve switch.

Thanks very much
But I'm not convinced...
it must be exist a clean way in order to obtain a real static binary,
than in theory it would have to also be faster.

But perhaps at this time it does not have much sense to strive on 1.3.
Some progress on 2.0?

PS: The same problems are taken place under Windows?

[sancio]

Hi,
ICC is a frustrating compiler
Your executable is static, my dynamic.
The -static flag cause me headache:
checking size of long int... configure: error: cannot compute sizeof (long int), 77

Code: [Select]

gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ icpc -o conftest -no-sox -O3 -pc64 -xP -axP -fp-model fast -no-prec-div -no-prec-sqrt -ipo4 -I/opt/intel/cc/9.1.045/include -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -I/usr/include -I/usr/include/openssl -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib  -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static-libcxa -static -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib conftest.cc  -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -Wl,-Bdynamic -ldl -Wl,-Bdynamic -lm /usr/lib/libnsl.a -Wl,-Bdynamic -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -Wl,-Bdynamic -lpthread -Wl,-Bdynamic -lc -lippcore -lippsmerged


Mmm...if I remove all -Wl,-Bdynamic it compile fine and static...
Now I think:
It's configure that insert in Makefiles this linker option,
why?
Is it wrong? Or is it's a ICC bug that should ignore it in a static compile?
Boh...

Maybe that removing this option from all Makefiles static compile works, but it's dirty.

I hope that this can be of useful.

[Simon]

Thanks for the hint, I'll try that next time I compile on Linux.

I've long been annoyed that I couldn't compile a real static binary as-is, and agree it's most probably a configure problem (I don't think -static worked as is with GCC either, though I'm not sure).

The real solution would be to modify configure.ac and change the macros there, I believe.

Regards,
Simon.

[michael37]

Yup - /QxT and /QaxT are for SSSE3 (Supplemental Streaming SIMD 3 Extensions), meaning they only run (and on Linux, only compile, for me) on Core 2-based systems.

Support for these was added in the 5.1.1 IPP version and the 9.1.028+ ICC one.

HTH,
Simon.

P.S.: I know the 1.3 code isn't that quick on Core 2s - guess why I put the 1.41 up instead?

Regarding 1.41 for Linux -- could you please make it so I can see the pre-released forum/downloads?  I am not a builder (besides I use CentOS which is prohibitive to building Boinc), but I can test stuff.  I've worked with Harold in the past. 

Regarding -xT flag --  -xP also includes SSE3 instructions, but runs on single core EM64T Xeons.  Most of my computers have these processors.  You no longer have -xP Linux application, so I use -xN version.  I've heard that benchmarks are pretty much the same on -xP and -xN versions.

Regarding -xW –axT flag -- that's not -xT -axT.  It sounds like the code with -xW -axT flags should run on SSE3 Opterons and Athlon64s.  Unfortunately, I no longer have one of these.

[sancio]

Thanks for the hint, I'll try that next time I compile on Linux.

I've long been annoyed that I couldn't compile a real static binary as-is, and agree it's most probably a configure problem (I don't think -static worked as is with GCC either, though I'm not sure).

The real solution would be to modify configure.ac and change the macros there, I believe.

I think so, too.
I have done some tests:

I tried to manually link only seti_boinc with -static or to add -static to Makefiles removing any -Wl,-Bdynamic.
It produce these errors:

Code: [Select]

IPO Warning: unresolved : __intel_cpu_indicator
        Referenced in libimf.a(sqrt_stub.o)
        Referenced in libimf.a(sincos_stub.o)
        Referenced in libimf.a(exp_stub.o)
        Referenced in libimf.a(ceil_stub.o)
        Referenced in libimf.a(floor_stub.o)
        Referenced in libimf.a(log10_stub.o)
        Referenced in libimf.a(log_stub.o)
        Referenced in libimf.a(pow_stub.o)
        Referenced in libimf.a(atanf_stub.o)
        Referenced in libimf.a(cosf_stub.o)
        Referenced in libimf.a(sincosf_stub.o)
        Referenced in libimf.a(fmod_stub.o)
        Referenced in libimf.a(sin_stub.o)
        Referenced in libimf.a(cos_stub.o)
IPO Warning: unresolved : __intel_cpu_indicator_init
        Referenced in libimf.a(sqrt_stub.o)
        Referenced in libimf.a(sincos_stub.o)
        Referenced in libimf.a(exp_stub.o)
        Referenced in libimf.a(ceil_stub.o)
        Referenced in libimf.a(floor_stub.o)
        Referenced in libimf.a(log10_stub.o)
        Referenced in libimf.a(log_stub.o)
        Referenced in libimf.a(pow_stub.o)
        Referenced in libimf.a(atanf_stub.o)
        Referenced in libimf.a(cosf_stub.o)
        Referenced in libimf.a(sincosf_stub.o)
        Referenced in libimf.a(fmod_stub.o)
        Referenced in libimf.a(sin_stub.o)
        Referenced in libimf.a(cos_stub.o)
IPO: performing multi-file optimizations
IPO: generating assembly file /tmp/icpc0Ktz4Jas_.s
analyzeFuncs.cpp(849) : (col. 3) remark: LOOP WAS VECTORIZED.
[...]
/tmp/ipo_icpcMvqdAO.o(.text+0x1013a6): In function `lookup_group(char*, unsigned int&)':
: warning: Using 'getgrnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/opt/intel/cc/9.1.045/lib/libimf.a(sincos_stub.o)(.text+0x2): In function `sincos':
: undefined reference to `__intel_cpu_indicator'
/opt/intel/cc/9.1.045/lib/libimf.a(sincos_stub.o)(.text+0x12): In function `sincos':
: undefined reference to `__intel_cpu_indicator'
/opt/intel/cc/9.1.045/lib/libimf.a(sincos_stub.o)(.text+0x21): In function `sincos':
: undefined reference to `__intel_cpu_indicator_init'
[...]

libirc contains __intel_cpu_indicator_init definition:

Code: [Select]

gtoso@marte:~/src/kwsn/seti_boinc/client$ nm /opt/intel/cc/9.1.045/lib/libirc.a |grep "T __intel_cpu_indicator_init"
00000184 T __intel_cpu_indicator_init
gtoso@marte:~/src/kwsn/seti_boinc/client$ nm /opt/intel/cc/9.1.045/lib/libirc.so |grep "T __intel_cpu_indicator_init"
000114dc T __intel_cpu_indicator_init

Adding -lirc it seems to compile fine and static, I have not yet tested it.
However IPO warnings remain and it seems to depend on glibc version used to compile
Maybe some source modification are needed...

Code: [Select]

gtoso@marte:~/src/kwsn/seti_boinc/client$ icpc -o seti_boinc -pthread -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -lirc -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib   seti_boinc-main.o seti_boinc-analyzeFuncs.o seti_boinc-analyzeReport.o seti_boinc-analyzePoT.o seti_boinc-pulsefind.o seti_boinc-gaussfit.o seti_boinc-lcgamm.o seti_boinc-malloc_a.o seti_boinc-seti.o seti_boinc-seti_header.o seti_boinc-timecvt.o seti_boinc-s_util.o seti_boinc-version.o seti_boinc-worker.o seti_boinc-chirpfft.o seti_boinc-spike.o seti_boinc-progress.o seti_boinc-fft8g.o seti_boinc-gdata.o seti_boinc-schema_master.o seti_boinc-sqlrow.o seti_boinc-sqlblob.o seti_boinc-xml_util.o   -L/home/gtoso/src/kwsn/boinc/api -lboinc_api -L/home/gtoso/src/kwsn/boinc/lib -lboinc    -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -ldl -lm /usr/lib/libnsl.a -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -lpthread -lc -lippcore -lippsmerged
IPO Warning: unresolved : __intel_cpu_indicator
        Referenced in libimf.a(sqrt_stub.o)
        Referenced in libimf.a(sincos_stub.o)
        Referenced in libimf.a(exp_stub.o)
        Referenced in libimf.a(ceil_stub.o)
[...]
fft8g.cpp(196) : (col. 5) remark: LOOP WAS VECTORIZED.
/tmp/ipo_icpcg81LiJ.o(.text+0x1013a6): In function `lookup_group(char*, unsigned int&)':
: warning: Using 'getgrnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
gtoso@marte:~/src/kwsn/seti_boinc/client$ file seti_boinc
seti_boinc: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
gtoso@marte:~/src/kwsn/seti_boinc/client$ ldd seti_boinc
        not a dynamic executable
gtoso@marte:~/src/kwsn/seti_boinc/client$ 


[Simon]

Good work!

I'll try and incorporate this into the How-To and get the next versions compiled as fully static binaries.

Thanks for your help!
Simon.

[sancio]

Good work!

Thanks 
I hope that someone will succeed to make it better and in clean way.

I'll try and incorporate this into the How-To and get the next versions compiled as fully static binaries.

In attachment all my modifies.

Thanks for your help!
Simon.

Thanks to you and to all people that have contributed in any ways
and thanks to Alex Kan for the great mac client!
Gianluca.


[attachment deleted by admin]