Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - bulrichl

Pages: [1] 2 3 ... 35
1
Announcements / Re: Cameron Leger's SubframeSelector
« on: 2020 February 23 01:24:32 »
Did you read the Reference Documentation (Process Explorer/ImageInspection/SubframeSelector)? If the answer is yes, you will have to put concrete questions.

The board 'Announcement' ("This board will be used to announce new features and releases.") is definitely the wrong place to put such questions, they rather belong to 'General'.

Bernd

3
General / Re: Local normalization problem
« on: 2020 February 20 03:18:43 »
Instead of doing the image calibration and registration tasks manually, you can use the WeightedBatchPreprocessing script, which automates them. However, knowing the entire data reduction process and understanding how it works at each stage is absolutely necessary if you want to control your data. Automation should never come without previous knowledge of the tasks performed.
My guide probably should be prefaced by this statement. I could not express it better.

Bernd

4
General / Re: Problem with Overscan Inconsistent Geometry
« on: 2020 February 18 03:29:53 »
The only effect of enabling 'Ignore overscan area' in the camera driver is that the outputted image will be cropped. In case of the QHY600, the full image will be cropped by 24 pixels at the left side and by 34 pixels at the bottom.

Yes, I also read the part of the QHY600 manual that covers optically black and overscan region in the section "Image Area Layout". It seems very general and vague to me. They don't give any hints about how to use these areas of the sensor for the calibration.

Generally, the optically black area is a region on the sensor that is equipped with photosites that are covered with a nontransparent layer, and the overscan area does not contain any photosites. So the intensities of the optically black area contain the constant bias offset + dark current + read noise, whereas the intensities in the overscan area contain only the constant bias offset + read noise.

Since PixInsight subtracts the median of the source region (i.e.: a constant value) from the target region, Overscan calibration cannot be taken as a replacement for applying a MasterDark.

As I wrote before, if you decide to use Overscan correction, this must be applied as well to all calibration frames in use, otherwise the calibration result will be wrong. The only purpose of Overscan calibration is to correct for a drifting bias offset over time.

Regarding your questions:
1. I guess that either could be used, but definitely not both at the same time. The value computed from the optically black area would additionally contain the average dark current, and the value computed from the overscan area would not.
2. One could think of it this way.
3. Using PixInsight's Overscan calibration cannot be taken as a replacement for applying a MasterDark. Since Overscan calibration only subtracts a constant value from all pixels in the corresponding target region, fixed pattern noise is not at all corrected by this procedure.
4. In case of a drifting bias offset, the calibration result will be more precise. If the bias offset of your camera is stable, using Overscan calibration in PixInisght will not improve the calibration result.

Bernd

5
General / Re: Problem with Overscan Inconsistent Geometry
« on: 2020 February 17 04:20:58 »
I am confused by the provided information about the overscan settings that you used for the source regions. As I stated above, in ImageCalibration, ALL of the rectangles are defined by the coordinates x, y of the upper left point and width and height of the rectangle. This does not only apply to the image and target region but also applies to the source region!

In my post above I just tried to figure out which settings could work based on the single raw light frame, "M84-0092-B.fit", that was uploaded by you. However, my conclusion might be wrong. The necessary information about how to correctly set up the overscan calibration should be contained in the documentation of your camera. I am, however, unclear whether it is reasonable at all to apply an Overscan correction for this camera model which seems to be a QHY600 (with a CMOS sensor).

I want to explain the Overscan correction in PixInsight in more detail.

Here is an excerpt of the mouse-over text of 'Overscan #1' in section 'Overscan':
Quote
... A source overscan region is used to compute an overscan correction value as the median of all source region pixels that will be subtracted from all pixels in the corresponding target region.
So the sole purpose of this function is to correct for a drifting bias offset over time. If the bias offset of your camera is stable, you don't need Overscan calibration.

Generally, if you decide to use the Overscan correction, you need to
- recapture all the calibration frames with overscan enabled in the camera driver and
- and apply overscan-calibration to each of the individual calibration frames in use when preparing the master calibration files.

Definitely you must not use Overscan calibration for the calibration of the light frames, and at the same time continue using older master calibration files that are produced without overscan calibration -- this approach will yield a wrong calibration result.

Bernd

6
General / Re: Problem with Overscan Inconsistent Geometry
« on: 2020 February 15 05:05:17 »
I am unclear whether your issues are solved or not.

I took a look at your uploaded light frame. This image has a width of 9600 and a height of 6422 pixels. In STF, I set the parameters to Shadows Clipping = 0.008 and Midtones Balance = 0.010. Then one realizes black borders at the left side (width 24 pixels) and at the lower side (height 34 pixels). In Image calibration, the rectangles are defined by the coordinates x, y of the upper left point, and width and height of the rectangle.

So image and target region are:
x = 24, y = 0, w = 9576, h = 6388

I tried the overscan calibration with two different Source region settings:
1. x = 24, y = 6388, w = 9576, h =   34 (i.e. the lower region)
2. x =  0, y =    0, w =   24, h = 6388 (i.e. the left region)


After overscan calibration, the image has a width of 9576 and a height of 6388 pixels since the black borders are cropped. It made no difference which Source region was used for the overscan calibration, the statisitcs of the calibrated images were almost identical.

However, in the FITS headers of the calibrated images, the rectangles used by Overscan calibration are defined differently: by the upper left and the lower right point coordinates:

Image and target region:
x1 = 24, y1 = 0, x2 = 9600, y2 = 6388

Source region:
1. x1 = 24, y1 = 6388, x2 = 9600, y2 = 6422
2. x1 =  0, y1 =    0, x2 =   24, y2 = 6388


This might have been somewhat confusing.

Bernd

7
Bug Reports / Re: Double Click in XISF-file starts PI installer
« on: 2020 February 14 08:54:14 »
Strange. According to Juan, this issue should be fixed already, see https://pixinsight.com/forum/index.php?topic=14665 , reply #2.

Bernd

9
General / Re: WBPP-Problem with image calibration
« on: 2020 February 12 14:30:10 »
I am using SGP for capturing with the ASCOM driver for my ZWO ASI cameras. There is nothing wrong with the ASCOM driver. However, it is generally not a good idea to use either different acquisition software or different camera drivers for light frames and calibration frames of one project.

Bernd

10
General / Re: WBPP-Problem with image calibration
« on: 2020 February 11 00:41:12 »
You are right, the brightness of the dark frames should not vary by much. In order to check a large set of dark frames, it will be best to use Blink and its function 'Series analysis report': you load the dark frames in Blink and click on the column chart symbol. Check 'Write text file', choose an output folder and then click 'OK'. Blink will write the file "Statistics.txt" to the specified folder which contains the statistics of the files loaded in Blink.

Bernd

11
General / Re: WBPP-Problem with image calibration
« on: 2020 February 09 03:20:19 »
Up to now I always used manual preprocessing with PixInsight's individual processes. So this was a premiere for me: I tried WBPP with a project of mine (camera: ZWO ASI071MC Pro) and used the individual calibration frames (not using finished master calibration frames) for this test.

I seleceted no frames in the section 'Bias', 110 dark frames and 100 flat-darks in the section 'Darks', 40 flats in the section 'Flats' and (for this test) only 5 light frames in the section 'Lights'. 'Calibrate with flat darks' was enabled in the tab 'Flats'. 'Calibrate only' was enabled in the tab 'Lights'. In 'Global Options', 'CFA images' was enabled and 'Optimize dark frames' was disabled (see appended screen section "WBPP_settings.JPG"). Clicking in 'Diagnostics' resulted in the warning: "** Warning: No bias frames have been selected.". This warning was ignored: the process was started and 'Continue' confirmed when the warning appeared again.

WBPP integrated the dark frames to the MasterDark and the flat-darks to the MasterFlat-Dark, calibrated the flat frames with the MasterFlat-Dark and integrated the calibrated flat frames. Then the 5 light frames were calibrated with the MasterDark and the MasterFlat, not using dark frame optimization. Finally the calibrated light frames were debayered. The whole procedure took 9 min 10.2 s. Smart report:

************************************************************
Group of 110 Dark frames
BINNING  : 1
Exposure : 300.00s

Rejection method auto-selected: Generalized Extreme Studentized Deviate
Integration completed: master file C:/Users/Bulrich/Downloads/test1/master/masterDark-BINNING_1-EXPTIME_300.xisf
************************************************************

************************************************************
Group of 100 Dark frames
BINNING  : 1
Exposure : 0.01s

Rejection method auto-selected: Generalized Extreme Studentized Deviate
Integration completed: master file C:/Users/Bulrich/Downloads/test1/master/masterDark-BINNING_1-EXPTIME_0.0055.xisf
************************************************************

************************************************************
Group of 40 Flat frames
BINNING  : 1
NoFilter
Exposure : 0.01s

[Calibration data]
  Master bias: none
  Master dark: C:/Users/Bulrich/Downloads/test1/master/masterDark-BINNING_1-EXPTIME_0.0055.xisf
  Master flat: none
Calibration completed
Rejection method auto-selected: Generalized Extreme Studentized Deviate
Integration completed: master file C:/Users/Bulrich/Downloads/test1/master/masterFlat-BINNING_1-FILTER_NoFilter.xisf
************************************************************

************************************************************
Group of 5 Light frames
BINNING  : 1
NoFilter
Exposure : 300.00s

[Calibration data]
  Master bias: none
  Master dark: C:/Users/Bulrich/Downloads/test1/master/masterDark-BINNING_1-EXPTIME_300.xisf
  Master flat: C:/Users/Bulrich/Downloads/test1/master/masterFlat-BINNING_1-FILTER_NoFilter.xisf
Calibration completed
Debayer completed
************************************************************

The master calibration files and the calibrated and debayered light frames were virtually identical with the files previously obtained by manually performing the processes. The minuscule differences resulted from different pixel rejection settings for the integration of the dark frames and flat-darks.

In summary: WBPP performed the preparation of the master calibration files and the light calibration flawlessly, the calibration result is correct. The only criticism I have is that the warning "** Warning: No bias frames have been selected." is out of place with my calibration approach (not pre-calibrated MasterDark, no use of bias frames or a MasterBias and no application of dark frame optimization).

So I am unsure why you obtained an unusable MasterDark from WBPP initially. I can only imagine that your settings in WBPP were not appropriate. Please closely compare your settings with mine.

Also very important: if you used the "masterDark-BINNING_1-EXPTIME_0.35.xisf" (mean = 781.7, median = 781.6; so here the same issue as with the MasterDark applies!) for the calibration of the flat frames, this would produce an incorrect MasterFlat which would result in an incorrect flat field correction. So the whole procedure for the preparation of the MasterFlat has to be repeated as well.

Bernd

12
General / Re: WBPP-Problem with image calibration
« on: 2020 February 08 14:14:47 »
Thank you for uploading the new files. Currently I am sitting at my notebook for the acquisition of images and have no access to your old data. I will take a close look tomorrow and report.

Definitely the new MasterDark has a mean = 334.3, median = 334.4, and the new integration is barely clipped (count % R: 99.84, G: 99.85, B: 99.86), so the new MasterDark matches the light frames pretty well, and the calibration procedure was successfull. The peaks in the histogram of the integration using the new calibration result look very clean. In the image I see a small brightening at the upper right corner. Otherwise the  integration using the new calibration result looks very well to me, I guess it is a great result for 50 x 5 s = 4 min 10 s.

More about it tomorrow. I am particularly curious about the camparison with the integration you obtained after manual calibration with the old Master calibration files, but wrong settings for the ImageCalibration, and of course about the reason why WBPP produced an unusable MasterDark.

Bernd

13
General / Re: WBPP-Problem with image calibration
« on: 2020 February 08 00:57:41 »
Could you please upload the new MasterDark and the new integration (following my suggestion) as well? I would love to take a look at it.

Bernd

14
General / Re: WBPP-Problem with image calibration
« on: 2020 February 07 07:54:40 »
It is very strange that the mean of the MasterDark is considerably higher than the mean of the light subframe. The filter is irrelevant in this regard. It doesn't look like light leakage to me either. The most likely explanation for your problem is different offset setting.

In this thread https://www.cloudynights.com/topic/672344-gain-and-sub-exposure-calculation-spreadsheet-for-the-zwo-asi183-and-294/page-2 #42, offset calculation is explained. For the Panasonic MN34230, a value of (offset setting * 16 ADU) is added during A/D conversion. The factor of 16 is the scaling factor for the scaling from 12 bit (intensity range from 0 to 4096) to 16 bit (intensity range from 0 to 65535) that the camera driver applies.

It is very likely that for the acquisition of the light frames, an offset of 21 was used, leading to a median of the light subframe of 336. However, I assume that for the dark frames an offset of about 50 was set. So my suggestion is: take new dark frames making sure that the offset is 21 this time. I guess these new dark frames will match your light frames then. Check the new dark frames with histogram and Statistics in PixInsight. If the mean / median suit to the values of the light frames, make a new MasterDark and please follow the recommendation in the last paragraph of my preceding post.

Bernd

15
General / Re: WBPP-Problem with image calibration
« on: 2020 February 07 03:47:43 »
Hi Dale,

the MasterDark has mean = 842.3, median = 842.8. The uncalibrated light subframe has mean = 345.1, median = 336. This cannot not work at all. The darks don't match the lights. Did you use the same acquisition software for capturing  light and dark frames?

According to the FITS headers, both the dark frames and the light subframe were exposed for 5 s, and EGAIN was 1.00111 (which corresponds to gain setting of 139 for the ASI1600). I also can imagine that different offset settings were used for the acquisition of light and dark frames. You'll need to repeat the dark frames with the same acquisition software and the same offset setting as used for the light frames.

However, there are further issues. The manually (with ImageCalibration) calibrated subframes may look wonderful to you, but they are not. In this calibration, the (faulty) MasterDark and MasterFlat were used, in the 'Master Dark' section the option 'Calibrate' was disabled and the option 'Optimize' was enabled. Thus dark frame optimization was applied, the Dark scaling factor was 0.063. These settings are wrong and yield an incorrect calibration result.

When you got matching dark frames, please stick to manual calibration with the ImageCalibration process for now and take a look at https://pixinsight.com/forum/index.php?topic=14663 , reply #9 for the recommended approach and settings for pre-processing.

Bernd

Pages: [1] 2 3 ... 35