Curious about the BatchPreprocessing script operation

[astropixel]

I'm curious... I have run 3 old image sets through BatchPreprocessing without MRS reverting to k-sigma during integration. The same image sets produced these errors frequently when manually preprocessing (dslr_raw).

Some issues I couldn't resolve with one image set My imaging system such as it is, scheduled half a session of 30 second shots, instead of 180 seconds. Consequently, the light data is useless, but scaling factors are quite evident against these 30 second frames using the preprocessing script. Not so following the dslr_raw workflow, MRS reverting to k-sigma at the slightest hint of stress.

So what is the difference? A newer computer with more memory and a faster processor, maybe - does MRS revert to k-sigma because of memory issues?

Anyway, it would be nice to know whether optimizations have been employed in the script.

[georg.viehoever]

I'm curious... I have run 3 old image sets through BatchPreprocessing without MRS reverting to k-sigma during integration. The same image sets produced these errors frequently when manually preprocessing (dslr_raw).
...

That's also what I observe. Something must be wrong with the good old dslr_raw workflow documented here http://pixinsight.com/forum/index.php?topic=2570.msg19019#msg19019 . Never got real good results with it, but the BatchProcessing script usually produces decent results. Some magic ingredient?

Georg

[astropixel]

I struggled with it to the point of frustration. It worked 60% of the time But the other 40% consumed 90% of my time.

Perhaps something implicit in the process, not explicitly stated???

Rowland.

[ruediger]

I looked through the source files and checked the master dark that is generated from the BatchProcessing script with ImageStatistics.

The major difference is, that the master dark keeps the bias pedestal as opposed to the "normal workflow", where the bias gets subtracted from the dark files during ImageCalibration. When using Canon DSLR, this bias subtraction truncates part of the dark histogram's left side to zero.

Could this be the only reason for a more stable noise evaluation? I'm still struggling with this problem on some of my stacks, but never used the BatchProcessing script so far.

Rüdiger

[astropixel]

In that case, retaining the bias pedestal would seem to be a workaround. Is that not possible with the DSLR_ workflow? Subtraction of the bias pedestal was raised in terms of increasing it if required.

Precisely, where does the DSLR workflow go wrong in this regard? to better understand the issue.

[Ignacio]

I looked through the source files and checked the master dark that is generated from the BatchProcessing script with ImageStatistics.

The major difference is, that the master dark keeps the bias pedestal as opposed to the "normal workflow", where the bias gets subtracted from the dark files during ImageCalibration. When using Canon DSLR, this bias subtraction truncates part of the dark histogram's left side to zero.

Could this be the only reason for a more stable noise evaluation? I'm still struggling with this problem on some of my stacks, but never used the BatchProcessing script so far.

Rüdiger

I think that's right. I experienced the same inconsistencies when manually reducing data from short duration subs (2 min) taken with a canon dslr. This was evidenced by an optimized dark scaling factor that was too high when calibrating lights (expecting <1 and got >3), or failing to correlate at all.

I avoid the problem by integrating an uncalibrated master dark, and then checking the "calibrate" master dark option during calibration of lights.

This is less of a problem for longer subs (say 5 min), as dark current builds up, and substracting the bias pedestal produces few pixels with negative numbers.

I also experimented with a pedestal in building a calibrated master dark, but could not get consistent results.

Ignacio

[Cleon_Wells]

Rowland, I second what Rudiger and Ignacio said, I think this goes back to my problems trying to integrate cal_Dark Canon T1i subs.

http://pixinsight.com/forum/index.php?topic=3663.msg25252#msg25252

I can always get a MasterDark to work with  by integrating un _cal Dark subs  and checking the Cal MasterDark.
Cleon

[astropixel]

Yep. This is one of those things that was mentioned from time to time, but never elaborated so well as in Ignacio and Rudigers posts. I get it now. I think I posted this before using the BP script.

I tried scaling a master dark composed of 10 x 2.5 minute frames against a set of 3.5 minute lights and the script didn't flinch.