Optimized AP 5.03 for Windows and %age blanked

[WinyterKnight]

Have now done 5 AP tasks with Opti 5.03 on my Q6600, and noticed crunch time is either 12 hrs (2 tasks) or 14+ hrs (3 tasks).
Looking further I see that the 12 hr tasks have 0% blanked, the others have 76% or more blanked.
Is there any explanation for this or is it the case that the noise inserted in the blanked portions is still not right.

I haven't done any 5.03's with default app, as it wasn't tested on Beta, and I have been trying to clear the 5.00's on main.

[Raistmer]

Have now done 5 AP tasks with Opti 5.03 on my Q6600, and noticed crunch time is either 12 hrs (2 tasks) or 14+ hrs (3 tasks).
Looking further I see that the 12 hr tasks have 0% blanked, the others have 76% or more blanked.
Is there any explanation for this or is it the case that the noise inserted in the blanked portions is still not right.

I haven't done any 5.03's with default app, as it wasn't tested on Beta, and I have been trying to clear the 5.00's on main.

Check if your non-blanked fast results have overflows (30 single or 30 repeated pulses or both) ?

[WinyterKnight]

Task 1171357741 reports
    single pulses: 1
repetitive pulses: 1
  percent blanked: 0.00

Task 1171358497 reports
    single pulses: 3
repetitive pulses: 1
  percent blanked: 0.00


So no, there are no overflows.

[Raistmer]

Task 1171357741 reports
    single pulses: 1
repetitive pulses: 1
  percent blanked: 0.00

Task 1171358497 reports
    single pulses: 3
repetitive pulses: 1
  percent blanked: 0.00


So no, there are no overflows.

Hm very interesting... There were reports that blanking itself takes pretty small amount of time...

[Richard Haselgrove]

Jason and I both saw that slow-processing effect for heavily-blanked WUs in early testing - Raistmer, review the pre-release thread for 13/14 Feb.

[Raistmer]

Jason and I both saw that slow-processing effect for heavily-blanked WUs in early testing - Raistmer, review the pre-release thread for 13/14 Feb.

Ok, thanks! Missed that probably.

[WinyterKnight]

Glad I'm not the only one seeing this on tasks with lots of blanking.
I've got 2 more 5.03's crunching both at over 50% completed, one is proceeding at over 8%/hr, so approx 100% complete in 12 hrs,  the other is only going a just about 7%/hr so probably over 14 hrs.

[Jason G]

This variation is a known effect of Berkeley's insertion of noise shaping envelopes into the radar blanking.  Nothing to worry about.  Luck of the draw.  I see this in about 10% of WUs only, and the time penalty for 90%+ blanking seems to be about 20-30% here.  Once we're more confident with validations, we can streamline this fairly radically.

Jason

[WinyterKnight]

This variation is a known effect of Berkeley's insertion of noise shaping envelopes into the radar blanking.  Nothing to worry about.  Luck of the draw.  I see this in about 10% of WUs only, and the time penalty for 90%+ blanking seems to be about 20-30% here.  Once we're more confident with validations, we can streamline this fairly radically.

Jason

Glad to know it is not the optmised app. And that Berkeley haven't got the correct shaped noise envelope yet.

I know the problems, I spent years injecting noise into comms channels hunting for distortion, intermodulation products, harmonic distortion, etc, and if the noise is not the correct shape you are just wasting your time.

[Josef W. Segur]

This variation is a known effect of Berkeley's insertion of noise shaping envelopes into the radar blanking.  Nothing to worry about.  Luck of the draw.  I see this in about 10% of WUs only, and the time penalty for 90%+ blanking seems to be about 20-30% here.  Once we're more confident with validations, we can streamline this fairly radically.

Jason

Glad to know it is not the optmised app. And that Berkeley haven't got the correct shaped noise envelope yet.

I know the problems, I spent years injecting noise into comms channels hunting for distortion, intermodulation products, harmonic distortion, etc, and if the noise is not the correct shape you are just wasting your time.

The impression I get is the shaped noise is probably good, knowing when to inject or not in the raw data still uncertain. In any case, the slowdown is related to the internal 'blanking' in AP_v5. As that's based on a more general noise detection approach, it probably can be triggered by other RFI sources in addition to the particular radar which is the main problem. Josh Von Korff does estimate that generating the needed patterns is a small fraction of processing time but his reference is stock processing rate, in the optimized code it's effectively multiplied by the reductions that have been found for other parts of the code. A 20% time variance in optimized might be only 5% or so in stock.

Josh seems to agree in principle that the internal blanking should be just that; skip processing the parts which have been judged bad. I think we can get that implemented for the next version. If so the more internal blanking needed the faster the app will run.
                                                                               Joe

[WinyterKnight]

Thought I would report the first of my Opt AP5.03 tasks has validated against a X9770 host that uses stock application.
The task was one with 76.6% blanked, hence 14h:10m, his time was 52h:20m.