+- +-
Say hello if visiting :) by Gecko
11 Jan 2023, 07:43:05 pm

Seti is down again by Mike
09 Aug 2017, 10:02:44 am

Some considerations regarding OpenCL MultiBeam app tuning from algorithm view by Raistmer
11 Dec 2016, 06:30:56 am

Loading APU to the limit: performance considerations by Mike
05 Nov 2016, 06:49:26 am

Better sleep on Windows - new round by Raistmer
26 Aug 2016, 02:02:31 pm

Author Topic: How to make your own optimized Seti@Home client for Linux  (Read 29478 times)

Offline michael37

  • Knight o' The Round Table
  • ***
  • Posts: 137
Re: How to make your own optimized Seti@Home client for Linux
« Reply #15 on: 25 Dec 2006, 11:31:54 pm »
Simon,

Does the SSE3 variant uses the QxT (-xT) optimization flag?  If yes, could you please update the Compiler options & seti #defines for optimiztion document.  If not, we need an option that uses the xT :)

In addition, I found this odd line in the Intel compiler guide: Use /QxW /QaxT (-xW –axT) to include other Intel 64 and like AMD* processors as well..  Did any experiment with these flags?

Thanks.

P.S. I started running the version 1.3 SSE2-optimized code on my dual-Xeon 5160 and it's surprisingly slow (~10,000 seconds for 0.4 ar/60 credit unit, 4 threads in parallel).  Granted, this may be due to general slowness of dual-51XX family vs single socket Core 2 Duos (more info here), but it has to be due to insufficiently optimized code.

sancio

  • Guest
Re: How to make your own optimized Seti@Home client for Linux
« Reply #16 on: 26 Dec 2006, 11:21:43 am »
Hi,
ICC is a frustrating compiler >:(
Your executable is static, my dynamic.
The -static flag cause me headache:
checking size of long int... configure: error: cannot compute sizeof (long int), 77

This is strange:
(conftest.cc is a piece of code copied from config.log that return the size of long int into file conftest.val)
Same error with flag -fast

Code: [Select]
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ icpc -o conftest -no-sox -O3 -pc64 -xP -axP -fp-model fast -no-prec-div -no-prec-sqrt -ipo4 -I/opt/intel/cc/9.1.045/include -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -I/usr/include -I/usr/include/openssl -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib  -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static-libcxa -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib conftest.cc  -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -Wl,-Bdynamic -ldl -Wl,-Bdynamic -lm /usr/lib/libnsl.a -Wl,-Bdynamic -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -Wl,-Bdynamic -lpthread -Wl,-Bdynamic -lc -lippcore -lippsmerged
IPO: performing single-file optimizations
IPO: generating object file /tmp/ipo_icpcw7cWoK.o
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ./conftest
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ file ./conftest
./conftest: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ldd ./conftest
        linux-gate.so.1 =>  (0xffffe000)
        libimf.so => /opt/intel/cc/9.1.045/lib/libimf.so (0xb7dad000)
        libsvml.so => /opt/intel/cc/9.1.045/lib/libsvml.so (0xb7d4d000)
        libssl.so.0 => /usr/lib/libssl.so.0 (0xb7d15000)
        libcrypto.so.0 => /usr/lib/libcrypto.so.0 (0xb7c11000)
        libdl.so.2 => /lib/libdl.so.2 (0xb7c0d000)
        libm.so.6 => /lib/libm.so.6 (0xb7bea000)
        librt.so.1 => /lib/librt.so.1 (0xb7bd7000)
        libpthread.so.0 => /lib/libpthread.so.0 (0xb7b84000)
        libc.so.6 => /lib/libc.so.6 (0xb7a67000)
        libirc.so => /opt/intel/cc/9.1.045/lib/libirc.so (0xb7a26000)
        /lib/ld-linux.so.2 (0xb7feb000)
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ icpc -o conftest -no-sox -O3 -pc64 -xP -axP -fp-model fast -no-prec-div -no-prec-sqrt -ipo4 -I/opt/intel/cc/9.1.045/include -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -I/usr/include -I/usr/include/openssl -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib  -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static-libcxa -static -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib conftest.cc  -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -Wl,-Bdynamic -ldl -Wl,-Bdynamic -lm /usr/lib/libnsl.a -Wl,-Bdynamic -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -Wl,-Bdynamic -lpthread -Wl,-Bdynamic -lc -lippcore -lippsmerged
IPO: performing single-file optimizations
IPO: generating object file /tmp/ipo_icpchnSplV.o
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ file ./conftest
./conftest: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ldd ./conftest
/usr/bin/ldd: line 124: ./conftest: No such file or directory
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ ./conftest
 -bash: ./conftest: No such file or directory
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$


[attachment deleted by admin]
« Last Edit: 28 Dec 2006, 05:15:50 am by Sancio »

Offline Simon

  • Ni!
  • Knight who says 'Ni!'
  • *****
  • Posts: 1045
    • Is it a bird? Is it a plane? No...its-the.net!
Re: How to make your own optimized Seti@Home client for Linux
« Reply #17 on: 26 Dec 2006, 11:26:11 am »
Hi,

I had exactly the same problem trying to use -static.

What I did to get the released executables to be static -

I used "statifier" (http://statifier.sourceforge.net/) with the --force-execve switch.

HTH,
Simon.

Offline Simon

  • Ni!
  • Knight who says 'Ni!'
  • *****
  • Posts: 1045
    • Is it a bird? Is it a plane? No...its-the.net!
Re: How to make your own optimized Seti@Home client for Linux
« Reply #18 on: 26 Dec 2006, 11:31:22 am »
Simon,

Does the SSE3 variant uses the QxT (-xT) optimization flag?  If yes, could you please update the Compiler options & seti #defines for optimiztion document.  If not, we need an option that uses the xT :)

In addition, I found this odd line in the Intel compiler guide: Use /QxW /QaxT (-xW –axT) to include other Intel 64 and like AMD* processors as well..  Did any experiment with these flags?
Yup - /QxT and /QaxT are for SSSE3 (Supplemental Streaming SIMD 3 Extensions), meaning they only run (and on Linux, only compile, for me) on Core 2-based systems.

Support for these was added in the 5.1.1 IPP version and the 9.1.028+ ICC one.

HTH,
Simon.

P.S.: I know the 1.3 code isn't that quick on Core 2s - guess why I put the 1.41 up instead? ;)

sancio

  • Guest
Re: How to make your own optimized Seti@Home client for Linux
« Reply #19 on: 26 Dec 2006, 11:52:30 am »
Hi,

I had exactly the same problem trying to use -static.

What I did to get the released executables to be static -

I used "statifier" (http://statifier.sourceforge.net/) with the --force-execve switch.

Thanks very much ;D
But I'm not convinced...
it must be exist a clean way in order to obtain a real static binary,
than in theory it would have to also be faster.

But perhaps at this time it does not have much sense to strive on 1.3.
Some progress on 2.0?

PS: The same problems are taken place under Windows?

sancio

  • Guest
Re: How to make your own optimized Seti@Home client for Linux
« Reply #20 on: 26 Dec 2006, 03:03:11 pm »
Hi,
ICC is a frustrating compiler >:(
Your executable is static, my dynamic.
The -static flag cause me headache:
checking size of long int... configure: error: cannot compute sizeof (long int), 77

Code: [Select]
gtoso@marte:~/src/kwsn/kwsn/seti_boinc$ icpc -o conftest -no-sox -O3 -pc64 -xP -axP -fp-model fast -no-prec-div -no-prec-sqrt -ipo4 -I/opt/intel/cc/9.1.045/include -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib -I/usr/include -I/usr/include/openssl -I/opt/intel/ipp/5.2_beta/ia32/include -I/opt/intel/ipp/5.2_beta/ia32/tools/staticlib  -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static-libcxa -static -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib conftest.cc  -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -Wl,-Bdynamic -ldl -Wl,-Bdynamic -lm /usr/lib/libnsl.a -Wl,-Bdynamic -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -Wl,-Bdynamic -lpthread -Wl,-Bdynamic -lc -lippcore -lippsmerged


Mmm...if I remove all -Wl,-Bdynamic it compile fine and static...
Now I think:
It's configure that insert in Makefiles this linker option,
why?
Is it wrong? Or is it's a ICC bug that should ignore it in a static compile?
Boh...

Maybe that removing this option from all Makefiles static compile works, but it's dirty.

I hope that this can be of useful.

Offline Simon

  • Ni!
  • Knight who says 'Ni!'
  • *****
  • Posts: 1045
    • Is it a bird? Is it a plane? No...its-the.net!
Re: How to make your own optimized Seti@Home client for Linux
« Reply #21 on: 26 Dec 2006, 03:17:47 pm »
Thanks for the hint, I'll try that next time I compile on Linux.

I've long been annoyed that I couldn't compile a real static binary as-is, and agree it's most probably a configure problem (I don't think -static worked as is with GCC either, though I'm not sure).

The real solution would be to modify configure.ac and change the macros there, I believe.

Regards,
Simon.

Offline michael37

  • Knight o' The Round Table
  • ***
  • Posts: 137
Re: How to make your own optimized Seti@Home client for Linux
« Reply #22 on: 27 Dec 2006, 11:23:57 pm »
Yup - /QxT and /QaxT are for SSSE3 (Supplemental Streaming SIMD 3 Extensions), meaning they only run (and on Linux, only compile, for me) on Core 2-based systems.

Support for these was added in the 5.1.1 IPP version and the 9.1.028+ ICC one.

HTH,
Simon.

P.S.: I know the 1.3 code isn't that quick on Core 2s - guess why I put the 1.41 up instead? ;)

Regarding 1.41 for Linux -- could you please make it so I can see the pre-released forum/downloads?  I am not a builder (besides I use CentOS which is prohibitive to building Boinc), but I can test stuff.  I've worked with Harold in the past. 

Regarding -xT flag --  -xP also includes SSE3 instructions, but runs on single core EM64T Xeons.  Most of my computers have these processors.  You no longer have -xP Linux application, so I use -xN version.  I've heard that benchmarks are pretty much the same on -xP and -xN versions.

Regarding -xW –axT flag -- that's not -xT -axT.  It sounds like the code with -xW -axT flags should run on SSE3 Opterons and Athlon64s.  Unfortunately, I no longer have one of these.


sancio

  • Guest
Re: How to make your own optimized Seti@Home client for Linux
« Reply #23 on: 28 Dec 2006, 08:05:16 am »
Thanks for the hint, I'll try that next time I compile on Linux.

I've long been annoyed that I couldn't compile a real static binary as-is, and agree it's most probably a configure problem (I don't think -static worked as is with GCC either, though I'm not sure).

The real solution would be to modify configure.ac and change the macros there, I believe.

I think so, too.
I have done some tests:

I tried to manually link only seti_boinc with -static or to add -static to Makefiles removing any -Wl,-Bdynamic.
It produce these errors:

Code: [Select]
IPO Warning: unresolved : __intel_cpu_indicator
        Referenced in libimf.a(sqrt_stub.o)
        Referenced in libimf.a(sincos_stub.o)
        Referenced in libimf.a(exp_stub.o)
        Referenced in libimf.a(ceil_stub.o)
        Referenced in libimf.a(floor_stub.o)
        Referenced in libimf.a(log10_stub.o)
        Referenced in libimf.a(log_stub.o)
        Referenced in libimf.a(pow_stub.o)
        Referenced in libimf.a(atanf_stub.o)
        Referenced in libimf.a(cosf_stub.o)
        Referenced in libimf.a(sincosf_stub.o)
        Referenced in libimf.a(fmod_stub.o)
        Referenced in libimf.a(sin_stub.o)
        Referenced in libimf.a(cos_stub.o)
IPO Warning: unresolved : __intel_cpu_indicator_init
        Referenced in libimf.a(sqrt_stub.o)
        Referenced in libimf.a(sincos_stub.o)
        Referenced in libimf.a(exp_stub.o)
        Referenced in libimf.a(ceil_stub.o)
        Referenced in libimf.a(floor_stub.o)
        Referenced in libimf.a(log10_stub.o)
        Referenced in libimf.a(log_stub.o)
        Referenced in libimf.a(pow_stub.o)
        Referenced in libimf.a(atanf_stub.o)
        Referenced in libimf.a(cosf_stub.o)
        Referenced in libimf.a(sincosf_stub.o)
        Referenced in libimf.a(fmod_stub.o)
        Referenced in libimf.a(sin_stub.o)
        Referenced in libimf.a(cos_stub.o)
IPO: performing multi-file optimizations
IPO: generating assembly file /tmp/icpc0Ktz4Jas_.s
analyzeFuncs.cpp(849) : (col. 3) remark: LOOP WAS VECTORIZED.
[...]
/tmp/ipo_icpcMvqdAO.o(.text+0x1013a6): In function `lookup_group(char*, unsigned int&)':
: warning: Using 'getgrnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/opt/intel/cc/9.1.045/lib/libimf.a(sincos_stub.o)(.text+0x2): In function `sincos':
: undefined reference to `__intel_cpu_indicator'
/opt/intel/cc/9.1.045/lib/libimf.a(sincos_stub.o)(.text+0x12): In function `sincos':
: undefined reference to `__intel_cpu_indicator'
/opt/intel/cc/9.1.045/lib/libimf.a(sincos_stub.o)(.text+0x21): In function `sincos':
: undefined reference to `__intel_cpu_indicator_init'
[...]

libirc contains __intel_cpu_indicator_init definition:
Code: [Select]
gtoso@marte:~/src/kwsn/seti_boinc/client$ nm /opt/intel/cc/9.1.045/lib/libirc.a |grep "T __intel_cpu_indicator_init"
00000184 T __intel_cpu_indicator_init
gtoso@marte:~/src/kwsn/seti_boinc/client$ nm /opt/intel/cc/9.1.045/lib/libirc.so |grep "T __intel_cpu_indicator_init"
000114dc T __intel_cpu_indicator_init

Adding -lirc it seems to compile fine and static, I have not yet tested it.
However IPO warnings remain and it seems to depend on glibc version used to compile :-[
Maybe some source modification are needed...

Code: [Select]
gtoso@marte:~/src/kwsn/seti_boinc/client$ icpc -o seti_boinc -pthread -L/opt/intel/cc/9.1.045/lib -L/opt/intel/ipp/5.2_beta/ia32/lib -lirc -limf -lippsmerged -lippvmmerged -lippchmerged -lippcore -lsvml -i-static -static -L/usr/lib -nodefaultlibs -L/opt/intel/ipp/5.2_beta/ia32/lib   seti_boinc-main.o seti_boinc-analyzeFuncs.o seti_boinc-analyzeReport.o seti_boinc-analyzePoT.o seti_boinc-pulsefind.o seti_boinc-gaussfit.o seti_boinc-lcgamm.o seti_boinc-malloc_a.o seti_boinc-seti.o seti_boinc-seti_header.o seti_boinc-timecvt.o seti_boinc-s_util.o seti_boinc-version.o seti_boinc-worker.o seti_boinc-chirpfft.o seti_boinc-spike.o seti_boinc-progress.o seti_boinc-fft8g.o seti_boinc-gdata.o seti_boinc-schema_master.o seti_boinc-sqlrow.o seti_boinc-sqlblob.o seti_boinc-xml_util.o   -L/home/gtoso/src/kwsn/boinc/api -lboinc_api -L/home/gtoso/src/kwsn/boinc/lib -lboinc    -lssl -lcrypto /usr/lib/libcrypto.a /usr/lib/libssl.a -ldl -lm /usr/lib/libnsl.a -lrt /usr/lib/libz.a /usr/lib/libjpeg.a /usr/lib/libstdc++.a -Wl,-Bstatic -lgcc_eh -lpthread -lc -lippcore -lippsmerged
IPO Warning: unresolved : __intel_cpu_indicator
        Referenced in libimf.a(sqrt_stub.o)
        Referenced in libimf.a(sincos_stub.o)
        Referenced in libimf.a(exp_stub.o)
        Referenced in libimf.a(ceil_stub.o)
[...]
fft8g.cpp(196) : (col. 5) remark: LOOP WAS VECTORIZED.
/tmp/ipo_icpcg81LiJ.o(.text+0x1013a6): In function `lookup_group(char*, unsigned int&)':
: warning: Using 'getgrnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
gtoso@marte:~/src/kwsn/seti_boinc/client$ file seti_boinc
seti_boinc: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
gtoso@marte:~/src/kwsn/seti_boinc/client$ ldd seti_boinc
        not a dynamic executable
gtoso@marte:~/src/kwsn/seti_boinc/client$ 

Offline Simon

  • Ni!
  • Knight who says 'Ni!'
  • *****
  • Posts: 1045
    • Is it a bird? Is it a plane? No...its-the.net!
Re: How to make your own optimized Seti@Home client for Linux
« Reply #24 on: 28 Dec 2006, 11:28:06 am »
Good work!

I'll try and incorporate this into the How-To and get the next versions compiled as fully static binaries.

Thanks for your help!
Simon.

sancio

  • Guest
Re: How to make your own optimized Seti@Home client for Linux
« Reply #25 on: 28 Dec 2006, 02:42:00 pm »
Good work!

Thanks  8)
I hope that someone will succeed to make it better and in clean way.

I'll try and incorporate this into the How-To and get the next versions compiled as fully static binaries.

In attachment all my modifies.

Thanks for your help!
Simon.

Thanks to you and to all people that have contributed in any ways
and thanks to Alex Kan for the great mac client!
Gianluca.


[attachment deleted by admin]

 

Welcome, Guest.
Please login or register.
 
 
 
Forgot your password?
Members
Total Members: 97
Latest: ToeBee
New This Month: 0
New This Week: 0
New Today: 0
Stats
Total Posts: 59559
Total Topics: 1672
Most Online Today: 25
Most Online Ever: 983
(20 Jan 2020, 03:17:55 pm)
Users Online
Members: 0
Guests: 19
Total: 19
Powered by EzPortal