[Split] PowerSpectrum Unit Test

[PatrickV2]

Not sure if you're looking for this, but below my results on my 8800GTX, 260.99 drivers:

Device: GeForce 8800 GTX, 1350 MHz clock, 731 MB memory.
Compiled with CUDA 3020.
Stock GetPowerSpectrum():
     64 threads:       17.8 GFlops    7.1 GB/s 1183.3ulps


GetPowerSpectrum() mod 1:
     32 threads:       14.2 GFlops    5.7 GB/s 1183.3ulps
     64 threads:       17.8 GFlops    7.1 GB/s 1183.3ulps
    128 threads:       17.8 GFlops    7.1 GB/s 1183.3ulps
    256 threads:       17.6 GFlops    7.0 GB/s 1183.3ulps


GetPowerSpectrum() mod 2:
     32 threads:        6.8 GFlops    2.7 GB/s 1183.3ulps
     64 threads:        6.2 GFlops    2.5 GB/s 1183.3ulps
    128 threads:        9.1 GFlops    3.7 GB/s 1183.3ulps
    256 threads:        8.0 GFlops    3.2 GB/s 1183.3ulps

Regards, Patrick.

[_heinz]

starting PowerSpectrum2
.
-device 0
Device: GeForce GTX 470, 810 MHz clock, 1248 MB memory.
Compiled with CUDA 3020.
Stock GetPowerSpectrum():
     64 threads:       20.6 GFlops    8.2 GB/s   0.0ulps


GetPowerSpectrum() mod 1:
     32 threads:       12.5 GFlops    5.0 GB/s   0.0ulps
     64 threads:       20.5 GFlops    8.2 GB/s   0.0ulps
    128 threads:       27.6 GFlops   11.0 GB/s   0.0ulps
    256 threads:       29.9 GFlops   12.0 GB/s   0.0ulps


GetPowerSpectrum() mod 2:
     32 threads:       14.4 GFlops    5.8 GB/s   0.0ulps
     64 threads:       28.3 GFlops   11.3 GB/s   0.0ulps
    128 threads:       42.4 GFlops   16.9 GB/s   0.0ulps
    256 threads:       42.5 GFlops   17.0 GB/s   0.0ulps


-device 1
Device: GeForce GTX 470, 810 MHz clock, 1249 MB memory.
Compiled with CUDA 3020.
Stock GetPowerSpectrum():
     64 threads:       20.6 GFlops    8.3 GB/s   0.0ulps


GetPowerSpectrum() mod 1:
     32 threads:       12.6 GFlops    5.0 GB/s   0.0ulps
     64 threads:       20.5 GFlops    8.2 GB/s   0.0ulps
    128 threads:       27.5 GFlops   11.0 GB/s   0.0ulps
    256 threads:       30.1 GFlops   12.0 GB/s   0.0ulps


GetPowerSpectrum() mod 2:
     32 threads:       14.4 GFlops    5.8 GB/s   0.0ulps
     64 threads:       28.4 GFlops   11.4 GB/s   0.0ulps
    128 threads:       42.2 GFlops   16.9 GB/s   0.0ulps
    256 threads:       41.1 GFlops   16.4 GB/s   0.0ulps


.
Done
modify:
@Jason, woundering about you get 20 GFlops more with 256 threads than mine GTX470
have you source for me to compile with 2011XE Compiler ?

[arkayn]

I tried running it on my 460 but the program always crashes on the end of 128/beginning of 256 threads in mod 2.

Never see any results.

[Claggy]

Here's my 9800GTX+ result, like Richard's 9800GTX+ it's a factory overclocked example, but by XFX:

Device: GeForce 9800 GTX/9800 GTX+, 1900 MHz clock, 496 MB memory.
Compiled with CUDA 3020.
Stock GetPowerSpectrum():
     64 threads:       16.1 GFlops    6.5 GB/s 1183.3ulps


GetPowerSpectrum() mod 1:
     32 threads:       15.1 GFlops    6.1 GB/s 1183.3ulps
     64 threads:       16.1 GFlops    6.5 GB/s 1183.3ulps
    128 threads:       16.0 GFlops    6.4 GB/s 1183.3ulps
    256 threads:       15.9 GFlops    6.3 GB/s 1183.3ulps


GetPowerSpectrum() mod 2:
     32 threads:        6.2 GFlops    2.5 GB/s 1183.3ulps
     64 threads:        8.2 GFlops    3.3 GB/s 1183.3ulps
    128 threads:        8.3 GFlops    3.3 GB/s 1183.3ulps
    256 threads:        8.1 GFlops    3.2 GB/s 1183.3ulps

Claggy

[Claggy]

Here's my 128Mb 8400M GS's result, while it's not got enough RAM for Seti, it at least gives you some figures for very slow GPU's:

Device: GeForce 8400M GS, 800 MHz clock, 114 MB memory.
Compiled with CUDA 3020.
Stock GetPowerSpectrum():
     64 threads:        1.2 GFlops    0.5 GB/s 1183.3ulps


GetPowerSpectrum() mod 1:
     32 threads:        1.2 GFlops    0.5 GB/s 1183.3ulps
     64 threads:        1.2 GFlops    0.5 GB/s 1183.3ulps
    128 threads:        1.2 GFlops    0.5 GB/s 1183.3ulps
    256 threads:        1.2 GFlops    0.5 GB/s 1183.3ulps


GetPowerSpectrum() mod 2:
     32 threads:        0.7 GFlops    0.3 GB/s 1183.3ulps
     64 threads:        0.7 GFlops    0.3 GB/s 1183.3ulps
    128 threads:        0.7 GFlops    0.3 GB/s 1183.3ulps
    256 threads:        0.6 GFlops    0.2 GB/s 1183.3ulps

Claggy

[_heinz]

run it twice on the ION
~~~~~~~~~~~~~~~~~
starting PowerSpectrum2
.

Device: ION, 1100 MHz clock, 242 MB memory.
Compiled with CUDA 3020.
Stock GetPowerSpectrum():
     64 threads:        1.9 GFlops    0.8 GB/s 1183.3ulps


GetPowerSpectrum() mod 1:
     32 threads:        1.3 GFlops    0.5 GB/s 1183.3ulps
     64 threads:        1.9 GFlops    0.7 GB/s 1183.3ulps
    128 threads:        1.9 GFlops    0.8 GB/s 1183.3ulps
    256 threads:        1.9 GFlops    0.8 GB/s 1183.3ulps


GetPowerSpectrum() mod 2:
     32 threads:        1.0 GFlops    0.4 GB/s 1183.3ulps
     64 threads:        1.0 GFlops    0.4 GB/s 1183.3ulps
    128 threads:        0.9 GFlops    0.4 GB/s 1183.3ulps
    256 threads:        0.8 GFlops    0.3 GB/s 1183.3ulps



Device: ION, 1100 MHz clock, 242 MB memory.
Compiled with CUDA 3020.
Stock GetPowerSpectrum():
     64 threads:        1.9 GFlops    0.8 GB/s 1183.3ulps


GetPowerSpectrum() mod 1:
     32 threads:        1.3 GFlops    0.5 GB/s 1183.3ulps
     64 threads:        1.9 GFlops    0.8 GB/s 1183.3ulps
    128 threads:        1.9 GFlops    0.8 GB/s 1183.3ulps
    256 threads:        1.9 GFlops    0.8 GB/s 1183.3ulps


GetPowerSpectrum() mod 2:
     32 threads:        1.0 GFlops    0.4 GB/s 1183.3ulps
     64 threads:        1.0 GFlops    0.4 GB/s 1183.3ulps
    128 threads:        0.9 GFlops    0.4 GB/s 1183.3ulps
    256 threads:  &nbqp;     0.8 GFlops    0.3 GB/s 1183.3ulps


.
Done

[SciManStev]

This is what I got on my 480's with 260.99

Device: GeForce GTX 480, 810 MHz clock, 1503 MB memory
Compiled with CUDA 3020
Stock GetPowerSpectrum<> mod 1:
     64 threads:       27.6 GFlops   11.1 GB/s   0.0ulps

GetPowerSpectrum<> mod 1:
     32 threads:       17.5 GFlops   7.0 GB/s    0.0ulps
     64 threads:       27.5 GFlops&nb!`; 11.0 GB/s    0.0ulps
    128 threads:       36.4 GFlops  14.6 GB/s    0.0ulps
    256 threads:       39.6 GFlops  15.8 GB/s    0.0ulps

GetPowerSpectrum<> mod 2:
     32 threads:       20.2 GFlops   8.1 GB/s    0.0ulps
     64 threads:       39.7 GFlops  15.9 GB/s    0.0ulps
    128 threads:       64.1 GFlops  25.6 GB/s    0.0ulps
    256 threads:       64.3 GFlops  25.7 GB/s    0.0ulps

Steve

I edited the data as the first time I was crunching.

[Jason G]

modify:
@Jason, woundering about you get 20 GFlops more with 256 threads than mine GTX470
have you source for me to compile with 2011XE Compiler ?

GTX480 has wider memory bus IIRC.  Also they're GPU Kernels Heinz, so CPU host side won't make any difference here (Unless Intel started messing with Cuda binaries  ) After some work, this will lead to a set of optimisation strategies for other kernels throughout, rather than 1 specific piece of useful code

I'm looking at this (almost pure) memory bound computation (powerspectrum), as a way to see what optimisation strategies work on different cards with that type of operation.  This way I can learn to make kernels that choose the best memory access strategy internally by compute capability.

So far it looks like Mod2 is winning on Fermi (apart from whatever is causing arkayn's problems)  Prior Gen 200 series seem to like Mod1 better, so I suspect there is some memory pattern issue for me to look at in Mod2 with respect to prior gen cards.  Earlier G80-G92 cards could be even more memory subsystem constrained, or need even more special treatment of access patterns, by the looks of things.

@Arkayn, not sure what would cause that, but on my 480 that's where things start to get 'a bit warm'  ... Is there a possibility of temperature issues ? Try cranking the fan perhaps.  [Edit:]  Probably pushing the 2.1 (GTX 460) architecture limits in Mod2.  I'll look into that for mod3.

Steve's WINNING! (Just  ) -

Plenty of data for me to chew on.  Will be thinking about mod3.

Jason

[Jason G]

Here's my 128Mb 8400M GS's result, while it's not got enough RAM for Seti, it at least gives you some figures for very slow GPU's:

Nice!  Another stubborn GPU 

[Jason G]

Not sure if you're looking for this, but below my results on my 8800GTX, 260.99 drivers:

Exactly what I'm looking for, thanks.

[glennaxl]

Device: GeForce 9800 GT, 1750 MHz clock, 500 MB memory.
Compiled with CUDA 3020.
Stock GetPowerSpectrum():
     64 threads:       13.6 GFlops    5.4 GB/s 1183.3ulps


GetPowerSpectrum() mod 1:
     32 threads:       12.1 GFlops    4.9 GB/s 1183.3ulps
     64 threads:       13.7 GFlops    5.5 GB/s 1183.3ulps
    128 threads:       13.5 GFlops    5.4 GB/s 1183.3ulps
    256 threads:       13.4 GFlops    5.3 GB/s 1183.3ulps


GetPowerSpectrum() mod 2:
     32 threads:        5.3 GFlops    2.1 GB/s 1183.3ulps
     64 threads:        7.0 GFlops    2.8 GB/s 1183.3ulps
    128 threads:        7.1 GFlops    2.8 GB/s 1183.3ulps
    256 threads:        6.8 GFlops    2.7 GB/s 1183.3ulps

[Jason G]

If anyone's wondering what this figure is:

... 1183.3ulps ...

It's a measure of the precision against a CPU double precision reference power spectrum.

Fermi's get 0ulps total deviation (most accurate) because they default to IEEE-754 compliance, whereas earlier gen consistently get 1183.3 because they use a fast single precision implementation by default.

I can either use special intrinsic functions on the older cards to force compliance, at a speed penalty, or allow the Fermi's to use the faster (less accurate) computation.  Will see.  1183.3 'Units of Least Precision' isn't much total deviation from double precision reference over the 1048576 point data set used in multibeam. 

an ulp is defined here as:

const float ulp =  1.192092896e-07f;

... about  0.00000012 ... and there'd be some of that amount of variation from double precision CPU reference scattered throughout the dataset.

Jason

[Jason G]

@Arkayn:  I looked through some results I have, and I have a GTX460 set that ran to completion @ stock speeds (Using driver 263.06).  Might be pushing the memory OC a bit on yours ?

[arkayn]

I think it is at 800/1600 right now, runs Collatz just fine at that speed.

I just took it down to stock speed as well as the lowest setting that Afterburner allowed and it still crashed the program.

This is on a XP-64 pro machine though.

Driver is the 263.06, do I need the toolkit installed as well?

[Jason G]

Driver is the 263.06, do I need the toolkit installed as well?

Nope, It's definitely something weird.  Bear in mind that those upper kernels are pushing Fermi's memory subsystem harder than any boinc science app has to date that I know of, so I doubt Collatz or any other existing app would be a fair comparison ( except maybe Furmark, which is just a savage thing to do to a graphics card )

If it runs this at stock OK, but not at 800/1600, then it might be Collatz stable, but is unlikely to be future X series stable.  My current feeling is that the memory frequency is the culprit, rather than the core.

(If it doesn't run correctly at stock either, then more guessing to do  )

[Later:]  At this stage I'm assuming some sort of bug in Mod2, so don;t go pulling things to bits just yet