ImageCalibration Overscan with Optic Black Area

[smccully]

The new QHY600 Mono Fullframe CMOS camera includes two overscan regions, one being an Optic Black Area and the other the overscan. From QHYCCD's documentation

https://www.qhyccd.com/index.php?m=content&c=index&a=show&catid=30&id=260

The optic black area and overscan area is usually used for precise calibration and for the calibration without the bias frame and dark frame, or other scientific using. Due to it is an non-image area ,  QHYCCD does not guranttee the signal quality in the overscan area. If you do not use this area, you can select the option "ignore overscan area" in the ASCOM driver or select a ROI of effective area range in SharpCAP.

My understanding, is that currently PixInsight is not able to read/differentiate the two regions. The new IMX455 Sensors stand to be the new go to high end AstroPhotography sensor, it would be good if PixInsight could take advantage of all the capabilities.

[Juan Conejero]

Our ImageCalibration tool supports up to 4 user-definable overscan regions. You can define them also with the WBPP script (Bias section).

[smccully]

Thank you for the reply, yes but my understanding is that the Overscan region will not take into the affect the Optic Black Area (Dark Substract) in the QHY600?

From this thread, my understanding is that the Optic Black Area of the QHY600 would not be correctly handled?

https://pixinsight.com/forum/index.php?topic=14711.msg87847#msg87847

Regarding your questions:
1. I guess that either could be used, but definitely not both at the same time. The value computed from the optically black area would additionally contain the average dark current, and the value computed from the overscan area would not.
2. One could think of it this way.
3. Using PixInsight's Overscan calibration cannot be taken as a replacement for applying a MasterDark. Since Overscan calibration only subtracts a constant value from all pixels in the corresponding target region, fixed pattern noise is not at all corrected by this procedure.
4. In case of a drifting bias offset, the calibration result will be more precise. If the bias offset of your camera is stable, using Overscan calibration in PixInisght will not improve the calibration result.

[smccully]

Maybe a checkbox to define an overscan region as an "Optic Black Area", though i can't say if this term is used colloquially by other manufacturers.

[smccully]

So I started looking into this, first at the Source Code to see what PixInsight was doing here. It appears that PixInsight takes the median of the Overscan rectangle and subtracts this from the Bias, Dark, Flats and then the target image when Calibration and Overscan parameters are on.

I then tried figuring out what exactly would be difference if there, or the difference between this and a Dark substract. After looking into a few reference documentation on Overscan calibration there appears to be two methods for Overscan Calibration.

One the same method being used by PixInsight, which subtracts the median of the Overscan rectangle.

The second, uses a Column by Column and/or Row by Row subtraction.

As a relatively non-knowledgeable, layperson on the subject. There appears to be some benefit towards the second approach, I currently plan to try and see if I can compare the two approaches on my dataset. I would be interested to know if this approach is something PixInsight has considered previously, or any reason that they would not want to take this approach?

http://www.stsci.edu/files/live/sites/www/files/home/hst/instrumentation/stis/documentation/_documents/stis_dhb.pdf

From Section

3.4.4 BLEVCORR: Large Scale Bias & Overscan Subtraction

Both the serial (physical) and virtual (parallel) overscans are used for the overscan bias level determination. A line-by-line subtraction is performed in the following way. An initial value of the electronic bias level, or overscan, is determined for each line of the image, using the serial and parallel overscans, and a function, currently a straight line, is fit to these values as a function of image line. This determines the slope of the bias level across the image at each line. The actual overscan value subtracted from an image line is the value of the linear fit at that image line. The initial value for each line is found by taking the median of a predetermined subset of the trailing serial overscan pixels. Currently, that region includes most of the trailing overscan region, however the first pixel and last three pixels are skipped, as they have been shown to be subject to problems; pixels flagged as bad in the input data quality flag are also skipped. The region used changes as a function of the binning or sub-array (see Table 3.2). The mean value of all overscan levels is computed and the mean is written to the output SCI extension header as MEANBLEV.

[smccully]

Do let me know if I have misunderstood or represented anything here. Any feedback is appreciated.

[bulrichl]

Hi,

I cannot follow your wish and doubt that it makes sense. In this thread:
https://pixinsight.com/forum/index.php?topic=14711
I already described to you, what PixInsight's Overscan calibration executes and for what it is intended: for compensating a drifting bias offset.

You are citing a publication in which specialized scientific sensors are used:
"This  handbook  describes  data  from  the  Space  Telescope  Imaging  Spectrograph (STIS) onboard the Hubble Space Telescope (HST), and how to manipulate, calibrate,and analyze those data."

And later:
"1.1.1  The STIS Detectors
STIS uses three large format (1024 × 1024 pixel) detectors:
• A Scientific Image Technologies (SITe) CCD, called the STIS/CCD, with 0.05 arcsec square pixels, covering a nominal 52 × 52 arcsec square field ofview (FOV), operating from ~2000 to 10300 Å.
• A Cs2Te MAMA detector, called the STIS/NUV-MAMA, with 0.025 arcsecsquare pixels, and a nominal 25 × 25 arcsec square FOV, operating in the near-ultraviolet from 1650 to 3100 Å.
• A solar blind CsI MAMA, called the STIS/FUV-MAMA, with 0.025 arcsecpixels, and a nominal 25 × 25 arcsec square FOV, operating in the far-ultraviolet from 1150–1700 Å."

These detectors probably have little in common with your QHY600 camera which uses a Sony IMX455 CMOS sensor; the first one in the publication is a CCD, and the latter two are something very different:
"The MAMAs: The two MAMA detectors are photon counting detectors that provide atwo-dimensional ultraviolet imaging capability."

To me it is unclear, whether the Overscan calibration that is decribed in the cited article for the special CCD sensor is applicable at all to a CMOS sensor. This question has to be answered by the manufacturer of the sensor and the camera respectively. QHY doesn't give any explanations how to correctly perform the Overscan calibration -- probably they don't know it. The right addressee for asking this question is Sony. It is highly unlikely that you can simply adapt the cited procedure to your CMOS sensor. So please clarify with the manufacturer which procedure is recommended for your CMOS sensor before making any assumptions.

Bernd

[smccully]

I cannot follow your wish and doubt that it makes sense. In this thread:
https://pixinsight.com/forum/index.php?topic=14711
I already described to you, what PixInsight's Overscan calibration executes and for what it is intended: for compensating a drifting bias offset.

The point of this thread was to ask, what it would take to expand the current Overscan Calibration.

To me it is unclear, whether the Overscan calibration that is decribed in the cited article for the special CCD sensor is applicable at all to a CMOS sensor. This question has to be answered by the manufacturer of the sensor and the camera respectively. QHY doesn't give any explanations how to correctly perform the Overscan calibration -- probably they don't know it. The right addressee for asking this question is Sony. It is highly unlikely that you can simply adapt the cited procedure to your CMOS sensor. So please clarify with the manufacturer which procedure is recommended for your CMOS sensor before making any assumptions.

Well, the point was not to use the exact procedure. But there appears to be a distinct difference between the two overscan regions in the QHY600 that they should be treated differently. The procedure that may make the most sense to me, was a Row by Row subtraction. But this is an overall fair point. I will find inquire with QHYCCD on what if any overscan calibration technique they can recommend or find out from Sony directly.

[bulrichl]

The point of this thread was to ask, what it would take to expand the current Overscan Calibration.

You are asking this question without knowing whether such an expansion would improve your image calibration result.

Well, the point was not to use the exact procedure. But there appears to be a distinct difference between the two overscan regions in the QHY600 that they should be treated differently. The procedure that may make the most sense to me, was a Row by Row subtraction. But this is an overall fair point. I will find inquire with QHYCCD on what if any overscan calibration technique they can recommend or find out from Sony directly.

Yes, this should be the first step.

Bernd