Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - bulrichl

Pages: [1] 2 3 ... 29
1
General / Re: Low gain darks not read by PI
« on: Today at 09:58 »
Yes, for the integration of darks, flat-darks and bias frames the setting 'Normalization' in the 'Pixel Rejection (1)' section has to be set to 'No Normalization', see Vicent Peris's excellent tutorial:

https://www.pixinsight.com/tutorials/master-frames/

Why did you say that an outlier-dark is no problem? Something obviously went wrong during the capture of dark 022, and I definitely would exclude it from the integration.

Bernd

2
General / Re: Low gain darks not read by PI
« on: Today at 09:18 »
With 'No Pixel Rejection', these darks are integrated flawlessly. However when I set e.g. 'Percentile Clipping', the error message appears.

On further inspection it turns out that dark 022 has a totally different median (= 4720 ADUs) than darks 020 and 021 (= 620 and 624 ADUs respectively). I guess this is the culprit. You could load all darks with 'Blink', output a Seies Analysis Report and take a look whether there are any more outliers. Exclude the deviating darks from the integration. Hope that helps.

Bernd

3
General / Re: Low gain darks not read by PI
« on: Today at 07:38 »
Then I guess it is advisable to upload a set of 3 dark frames which produce this error message in order that Juan can take a look at it.

Bernd

4
In order to report this you need 3 threads in the category "Announcements" which by the way is reserved for the developer of PixInsight?

This was a bug in SGP which was fixed in the SGP beta v3.1.0.211 in August 2019, see e.g. https://pixinsight.com/forum/index.php?topic=13796 . When you search in PixInsight Forum, you will find numerous threads about it.

Bernd

5
General / Re: Low gain darks not read by PI
« on: Today at 06:58 »
Did you also try the integration of the darks with 'Evaluate Noise' disabled?

Bernd

6
Lights were created with N.I.N.A: 1.10.0.10 (gain 120, offset 30), flat darks and flats are created with APT v.3.7.1 (gain 120, offset not written to the FITS header). We had it in another thread just recently:

It is not recommended to use different capturing software for light frames and calibration frames!

---

FlatDarks captured at 25th November 2019 (folder "/flat darks/"):

The flat darks in the folder "/flat darks/" have a median of about 77 ADUs which is not plausible at all. Presumably these frames were captured with a different (higher) offset setting than most of the other frames. Unfortunately the offset is not visible for the frames captured with APT.

Accordingly the MasterFlatDark "dark-BINNING_1-EXPTIME_3.5.xisf" has a median of 77.257.

All of these files (flat darks of 25th November 2019, MasterFlatDark and the integration of the calibrated light frames) are unusable and should be deleted.

---

FlatDarks captured at 27th November 2019 (folder "/flat darks 2019-11-27 L-enhance fw 120 sweatshirt GOOD peaks good 2.5sec (WORKS)/"):

The first file in the folder "/flat darks 2019-11-27 L-enhance fw 120 sweatshirt GOOD peaks good 2.5sec (WORKS)/", "F_5-Lenhance_2019-11-27_18-42-16_Bin1x1_2.5s__-10C.fit", is a flat frame, NOT a flat dark. It belongs to the folder "/flats-2019-11-27 L-enhance fw 120 sweatshirt GOOD peaks good 2.5sec done (WORKS)/".

The remaining 35 flat darks have a median of 1916 ADUs which is in the expected range.

---

Correct dark frames and consequently a correct MasterDark are missing.

Additional to using the incorrectly captured flat darks and the missing dark frames there are mistakes using the script Batch Preprocessing:

==================================================================================

[2019-12-04 20:08:04] ************************************************************
[2019-12-04 20:08:04] * Begin integration of dark frames
[2019-12-04 20:08:04] ************************************************************
[2019-12-04 20:08:04]
[2019-12-04 20:08:04] ImageIntegration: Global context
[2019-12-04 20:08:04]
[2019-12-04 20:08:04] Opening files:
[2019-12-04 20:08:04]
[2019-12-04 20:08:04] B:/Data/astronomy/Calibration Captures/Dark Flats/2019-11-25 L-enhance fw hyperstar 1.25 120 sweatshirt 3.5sec GOOD/F_5_2019-11-25_20-46-31_Bin1x1_3.5s__-10C.fit
[2019-12-04 20:08:04] 36 FITS keywords extracted.
[2019-12-04 20:08:04] Reading FITS image: 16-bit integers, 1 channel(s), 4144x2822 pixels: done
[2019-12-04 20:08:04] Computing image statistics: done
[2019-12-04 20:08:05] Weight          :     1.00000

...

[2019-12-04 20:08:32] ************************************************************
[2019-12-04 20:08:32] * End integration of dark frames
[2019-12-04 20:08:32] ************************************************************
[2019-12-04 20:08:32]
[2019-12-04 20:08:32] * Writing master dark frame:
[2019-12-04 20:08:32] B:/Data/astronomy/Captures/M42 Orion Nebula M42 hyperstar fw L-ehance/2019-11-25 30s 120 gain/PPoutput no darks/master/dark-BINNING_1-EXPTIME_3.5.xisf
[2019-12-04 20:08:32] Writing image 'integration': w=4144 h=2822 n=1 Gray Float32
[2019-12-04 20:08:32] 186 FITS keyword(s) embedded.
[2019-12-04 20:08:32] 16 image properties embedded.
[2019-12-04 20:08:32] Writing image 'rejection_low': w=4144 h=2822 n=1 Gray Float32
[2019-12-04 20:08:32] Writing image 'rejection_high': w=4144 h=2822 n=1 Gray Float32
[2019-12-04 20:08:33]
[2019-12-04 20:08:33] * Searching for a master flat dark frame with exposure time = 30s -- not found.
[2019-12-04 20:08:33]
[2019-12-04 20:08:33] * Calibration of flat frames skipped -- no flat darks found.
[2019-12-04 20:08:33]
[2019-12-04 20:08:33] ************************************************************
[2019-12-04 20:08:33] * Begin integration of flat frames
[2019-12-04 20:08:33] ************************************************************

...

[2019-12-04 20:09:18] ************************************************************
[2019-12-04 20:09:18] * End integration of flat frames
[2019-12-04 20:09:18] ************************************************************
[2019-12-04 20:09:18]
[2019-12-04 20:09:18] * Writing master flat frame:
[2019-12-04 20:09:18] B:/Data/astronomy/Captures/M42 Orion Nebula M42 hyperstar fw L-ehance/2019-11-25 30s 120 gain/PPoutput no darks/master/flat-FILTER_L-enhance-BINNING_1.xisf
[2019-12-04 20:09:18] Writing image 'integration': w=4144 h=2822 n=1 Gray Float32
[2019-12-04 20:09:18] 244 FITS keyword(s) embedded.
[2019-12-04 20:09:18] 16 image properties embedded.
[2019-12-04 20:09:18] Writing image 'rejection_low': w=4144 h=2822 n=1 Gray Float32
[2019-12-04 20:09:18] Writing image 'rejection_high': w=4144 h=2822 n=1 Gray Float32
[2019-12-04 20:09:18] Writing image 'slope_map': w=4144 h=2822 n=1 Gray Float32
[2019-12-04 20:09:20]
[2019-12-04 20:09:20] * Searching for a master dark frame with exposure time = 30s -- best match is 3.5s
[2019-12-04 20:09:20]
[2019-12-04 20:09:20] ************************************************************
[2019-12-04 20:09:20] * Begin calibration of light frames
[2019-12-04 20:09:20] ************************************************************

==================================================================================

According to this section of the logfile generated by BPP there are several flaws:

1) The flat darks are taken as dark frames. The "MasterDark" in fact is the MasterFlatDark.
2) Surprisingly the script searches for a MasterFlatDark with an exposure time of 30 s (this is the exposure time of the LIGHT frames, the flat frames had an exposure time of 3.5 s!). It does not find one and skips the ccalibration of the flat frames. The uncalibrated flat frames are then integrated and the result is used as the MasterFlat.
3) Then the script searches for a MasterDark with an exposure time of 30 s, finds the one with 3.5 s and takes this.

So there is big confusion. Obviously this is caused by a wrong selection of the sort of frames (light, dark, flat, flat-dark). Normally this has to be set in the capturing software and the rest is done automatically. Alternatively one can denominate the files accord to their sort of frame  (e.g. L for light, D for dark, F for Flat) and select them manually in BPP with 'Add Custom'.

The bottom line is: the flat frames were not calibrated at all. For the calibration of the light frames the MasterFlatDark made from the incorrect flat darks (instead of a MasterDark) and the "MasterFlat" (without subtraction of the bias offset) were used. The result is - not surprisingly - rubbish.

I strongly recommend you to perform the preparation of the master calibration files and the calibration of the light frames at least once separately.step by step (not with BPP but with the processes ImageIntegration, ImageCalibration, Debayer, StarAlignment). See my guide https://pixinsight.com/forum/index.php?topic=11968 for a detailed description and explanation of the preprocessing.

Bernd

7
General / Re: Tool Documentation
« on: 2019 December 04 14:47:02 »
I know there is documentation on imageintegration but not showing for me.

Did you once take a look at the PixInsight website? Obviously not. That seems to be demanded too much. This whole discussion is ridiculous.

Documentation of ImageIntegration:
https://pixinsight.com/doc/tools/ImageIntegration/ImageIntegration.html

Bernd

8
I use a ASI294MC Pro by myself, so I know the camera and how the calibration files should look like.

1. In the data that you uploaded I don't find 'flat darks' (this is the right term for darkframes that are captured with the exposure time of the flat frames, in your case 4.25 s). The 20 files in the folder "Dark Flats" in fact are flat frames! Please remove the files in that folder and provide the flat darks.

2. I have never seen dark frames that show such uneven intensities. I integrated the dark frames to a MasterDark (screen section appended). The intensity in the center of the MasterDark is 2860 ADUs, and at the corners 2400 - 2650 ADUs -- this is not plausible at all. Either what Adam described in his EDIT note applies or you got light incidence during dark frame acquisition.

3. The minimum intensity in the first light frame (see appended screen section) is 2084 ADUs, the minimum in the MasterDark is 2360 ADUs. When the MasterDark is subtracted from this light frame, negative values result for 25 % of the pixels. Predominantly the red channel is affected by the clipping. This is the reason for the error message "zero or insignificant signal detected" during ImageIntegration. Certainly the present dark frames are completely unusable.

It is always a good idea to check some calibration files, light frames and calibrated light frames with ImagesStatistics and HistogramTransformation. Incorrect image calibration cannot be ironed out in post-processing.

Bernd

9
General / Re: Image Calibration Woes..Whats Going Wrong
« on: 2019 November 29 08:19:58 »
This is an interesting case.

Because the dark frames were captured at -25 °C and the light frames at -30 °C, I performed the calibration of the light frames with dark frame optimization, i.e ImageCalibration with MasterDark and MasterBias (in the Master Dark section: both options 'Calibrate' and 'Optimize' enabled). Nevertheless the calibrated light frames were severely clipped. So I applied an output pedestal (100, 150 and 200). An output pedestal of 150 does not last to avoid the clipping of the calibrated light frames.

Whereas the individual calibrated files look passable when calibrated with a pedestal of 200, this doesn't seem to solve the underlying problem. After registration and integration, the result is not correct with ImageIntegration's default setting 'Subtract Pedestals' enabled. However, when this setting is disabled, strongly varying results regarding pixel rejection are obtained, depending on the used value of the output pedestal. This doesn't seem reasonable at all.

Particularly strange is that the mean or median value of the uncalibrated light frames are quite different: it looks like the bias offset (which seems reasonable in image 1) is decreasing from image to image. This effect is especially pronounced from image 1 to 2, see appended plot. I guess this is the real problem. I tried to circumvent this effect by adding the difference median(light frame 1) - median(current light frame) to each uncalibrated light frame in PixelMath. Then these "corrected" light frames were calibrated as above with an output pedestal of 100. The result is shown in the appended screen section.

For me it looks as if the bias offset of this camera (a FLI MicroLine ML16200) is not stable. However, there could be a different cause, what about RBI (anti-ghosting) technology? I don't know a lot about this topic, perhaps other users of a similar camera can chime in.

Rob's notes are to be considered as well, the most important question probably is whether dark and light frames were captured with the same acquisition software (and the same settings except the differing temperature).


Bernd

10
This error message is not at all related to the WBPP script. The reason is: Sequence Generator Pro (SGP) used a wrong format for these FITS keywords. This bug in SGP has been fixed in v3.1.0.383 (beta version).

See:
https://forum.mainsequencesoftware.com/t/again-fits-keywords-sitelat-sitelong/10998

and release notes for v3.1.0.383 (11/23/2019):
http://mainsequencesoftware.com/Releases

Bernd

11
That's great. And sorry, Juan, I did not intend to confuse the forum with my suggestions.

Bernd

12
It seems that LibRaw will maintain the changed cropping behavior of version 201910 snapshot. Thus the workaround that I suggested above (expecting that they revert to the original cropping behavior) is not reasonable.

In order to use old MasterCalibration files that were prepared with PixInsight versions < 1.8.8, a better workaround would be: crop (remove) the first row of the Master Calibration files (MasterDark and MasterFlat) and save them as new files. These modified Master Calibration files could then be used furthermore with PixInsight versions starting from 1.8.8.

---

The 600D is not the only camera model that is concerned; so far as I know the following Canon models had an uneven number of rows as well, and I guess it will be similar:

EOS 1D Mark IV
EOS 5D Mark II
EOS 7D
EOS 60D
EOS 550D
EOS 600D
EOS 1200D

Bernd

13
Hi Jürgen,

this seems to be a bug in the new LibRaw version, 201910 snapshot, that is used by PixInisght 1.8.8. Since cropping the first row changes the CFA pattern:

R G          G B
G B          R G

the displayed CFA pattern is not wrong. However, the main problem is that these light frames cannot be calibrated with old Master Calibration files (different image geometry).

A workaround would be: load the CR2 files and crop them with parameters: left margin 0, top margin 1, right margin 0, bottom margin 0. This adds an empty first row to the image. Thereby the CFA pattern becomes RGGB again and the image geometry is consistent with the previously generated Master calibration files. The cropping can be done for an Image Container.

Note that in this case the Bayer/mosaic pattern has to be set manually for the Debayer process, 'Auto' doesn't work with this workaround.

Bernd


14
General / Re: Pink stars after integration - DSLR data
« on: 2019 November 12 09:13:11 »
Hi Magnus,

yes, I missed the calibrated, not cosmetically corrected file.

In the calibrated frame the CFA0 (red) channel again looks quite different than the rest. The histogram peak is much broader (much more noise), and there are horizontal structures that are not visible in the other channels.

I also tried to calibrate only with the MasterDark (no MasterBias, no MasterFlat). Then the histogram peak of the CFA0 channel comes out much more narrow. So I guess that (at least part of) the issue is related to the quality of your MasterFlat. There is little signal in the CFA0 channel compared to the other channels. The horizontal structures are already weakly visible in the CFA0 channel of the light frame calibrated with only the MasterDark.

How many flat frames were used for the preparation of the MasterFlat? Maybe you have to use a larger number of flat frames.

Furthermore it stands out that the flat frames are underexposed. The red channel of the MasterFlat is by far (by factor 4 - 5!) the weakest, so this channel is concerned the most:

            MF_CFA0     MF_CFA1     MF_CFA2     MF_CFA3
count (%)   99.98763    99.99954    99.99933    99.99942
count (px)  5041956     5042557     5042546     5042551
mean        824.647     4391.798    4371.928    3515.716   <-
median      833.568     4445.423    4427.996    3557.699   <-
variance    3410.628    78454.199   75361.941   54379.268
stdDev      58.401      280.097     274.521     233.194
avgDev      54.944      269.616     264.493     227.833
MAD         51.681      247.832     243.658     217.574
minimum     0.441       266.044     54.010      51.771
maximum     1794.439    4951.006    4945.752    4051.001



I suggest you to make new flat frames with exposure time x 2.5, same light source.

Presumably you took sky flats? Such strong discrepancies between the color channels can cause trouble in the preprocessing of images captured by OSC cameras. Maybe a much better result would be obtainable by using a light source with white light (plus more flat frames).

Bernd

15
Announcements / Re: PixInsight 1.8.8 Released
« on: 2019 November 12 01:59:13 »
The parameter "-- with printer" has to be added to Target in the Shortcut Tab of Pixinsight, see https://pixinsight.com/forum/index.php?topic=13904 , reply #2 for Mac, reply #6 for Windows.

Bernd

Pages: [1] 2 3 ... 29