PixInsight Forum (historical)
PixInsight => General => Topic started by: andythilo on 2020 January 22 23:50:50
-
Hi
Been shooting for only a few months. But I'm having issues with my stacking. The light master below is from 45 x 180sec frames with 40 x flats, 40 x darks and 40 x flat darks. All frames at the same temperature and gain, flats are 4 sec as recommended for the ASI294MC Pro and taken with a flat field generator @ ADU 27000 approx.
https://1drv.ms/u/s!Ari3AWpbmLZ0gvwwSaNxwdSfJkmVJw?e=HYwv1Z
Setup is Esprit 80 with dedicated flattener, ASI 294MC Pro cooled to -20. Dew heater on both lens and guide camera.
If anyone could have a look and see what you think I'd be really grateful :). I can upload the all the individual frames if it helps.
-
I should add that every other frame was dithered and a 2x drizzle applied in PI to help with the undersampling of the Esprit 80/ASI294 combo.
Here's a link to the individual files - https://1drv.ms/u/s!Ari3AWpbmLZ0gvwxyfTENADAlBHSOg?e=lCnmgj
-
Before I start downloading an 1.7 GB rar file, could you please be more specific about the problem you are facing?
-
Before I start downloading an 1.7 GB rar file, could you please be more specific about the problem you are facing?
Hi, basically I have a light patch on the left side, although running the light frames through ccdinspector it seems I have a tilt, probably from my rotator as I didn't have this tilt in light frames before I fitted it, also tested in ccdinspector. The other issue is background mess. Hard to explain, it's not noise, maybe high level cloud but I ran it this morning using WBPP and it used 42 out of 45.
I guess I'd just like someone to try stacking it and seeing if they know what the issue is, whether is a viewing conditions issue, user error or equipment error. Sorry I know it's rather vague :(.
Thanks
Andy
-
Hi Andy,
the issue is definitely caused by the flat frames. You need to capture new flat frames.
It is conspicious that the red channel is very weak compared to the other channels. These are the mean values of the CFA channels of the MasterFlat:
CFA0 (red): 3109
CFA1 (green): 37843
CFA2 (green): 37840
CFA3 (blue): 23713
I suppose that you took sky flats -- this would explain why the red channel is that weak. Speaking from my own experience with the ASI294, there is no need to take long exposed flat frames. I took my flat frames with a not dimmed flatfield box and an exposure time of 0.003 s. However, it is not clear to me whether the weakness of the red channel is the actual problem.
By the way, why did you prepare the Master Calibration files with DeepSkyStacker?
Bernd
-
Hi
Hi Andy,
the issue is definitely caused by the flat frames. You need to capture new flat frames.
It is conspicious that the red channel is very weak compared to the other channels. These are the mean values of the CFA channels of the MasterFlat:
CFA0 (red): 3109
CFA1 (green): 37843
CFA2 (green): 37840
CFA3 (blue): 23713
I suppose that you took sky flats -- this would explain why the red channel is that weak. Speaking from my own experience with the ASI294, there is no need to take long exposed flat frames. I took my flat frames with a not dimmed flatfield box and an exposure time of 0.003 s. However, it is not clear to me whether the weakness of the red channel is the actual problem.
By the way, why did you prepare the Master Calibration files with DeepSkyStacker?
Bernd
Hi
Ignore the masters in the folders, I tried DSS and it puts them in there and I forget to delete them. I had my L-Enhance filter in for all frames. Flats were taken with a flat field generator with the voltage dropped to give me a 4sec sub, directly after the light frames. Flat darks the same but with the cap on the scope.
If you look at the stacked master light I linked, it's virtually all red, until I ABE or DBE.
I'll try not dimming the flat field generator but was told it was better for the 294 to run the subs >2 secs.
-
Hi Andy,
I guess you misunderstood what I wrote. Independently from the MasterFlat generated by DSS, your flat subframes (FITS format) are very weak in the red channel, and the same will be true for your MasterFlat, whether generated by DSS or by PixInsight. The very weak red channel in the MasterFlat will cause the red channel in the calibrated light frames to be shifted to high values. See the appended histograms of a flat subframe (left), uncalibrated light frame (middle) and your stacked master light (right). For better comparability, the flat frame and the uncalibrated light frame are debayered as well. Please note the strong shift of the red peak in the histograms of the the stacked master light compared to the histogram of the uncalibrated light frame.
You can easily evaluate the calibrated light frames or your stacked master light before applying DBE or ABE: simply disable 'Link RGB channels' in ScreenTransferFunction and apply an Auto Stretch to the image. Then the strong red color cast is gone.
As I wrote, I am NOT shure that the weakness of the red channel in the MasterFlat is the actual issue. So I am NOT telling you that the problem will be gone with using an undimmed flatfield box and short exposure time. However, it is absolutely clear that the flat frames at hand are causing your problem -- they are unusable. You can check that by repeating the image calibration without the MasterFlat. In this special case, a better result is to be expected. Of course I am not recommending to omit the flat frames as a solution. Bottom line is: you have to capture new flat frames.
In the last post you wrote that you used an L-enhance (dual band pass) filter. There is a thread in CloudyNights about problems with an ASI294 and similar filters:
https://www.cloudynights.com/topic/661685-word-of-warning-asi294mc-pro-and-opt-triad-and-nb/
Your problem seems to be related.
Bernd
-
Hi Andy,
I guess you misunderstood what I wrote. Independently from the MasterFlat generated by DSS, your flat subframes (FITS format) are very weak in the red channel, and the same will be true for your MasterFlat, whether generated by DSS or by PixInsight. The very weak red channel in the MasterFlat will cause the red channel in the calibrated light frames to be shifted to high values. See the appended histograms of a flat subframe (left), uncalibrated light frame (middle) and your stacked master light (right). For better comparability, the flat frame and the uncalibrated light frame are debayered as well. Please note the strong shift of the red peak in the histograms of the the stacked master light compared to the histogram of the uncalibrated light frame.
You can easily evaluate the calibrated light frames or your stacked master light before applying DBE or ABE: simply disable 'Link RGB channels' in ScreenTransferFunction and apply an Auto Stretch to the image. Then the strong red color cast is gone.
As I wrote, I am NOT shure that the weakness of the red channel in the MasterFlat is the actual issue. So I am NOT telling you that the problem will be gone with using an undimmed flatfield box and short exposure time. However, it is absolutely clear that the flat frames at hand are causing your problem -- they are unusable. You can check that by repeating the image calibration without the MasterFlat. In this special case, a better result is to be expected. Of course I am not recommending to omit the flat frames as a solution. Bottom line is: you have to capture new flat frames.
In the last post you wrote that you used an L-enhance (dual band pass) filter. There is a thread in CloudyNights about problems with an ASI294 and similar filters:
https://www.cloudynights.com/topic/661685-word-of-warning-asi294mc-pro-and-opt-triad-and-nb/
Your problem seems to be related.
Bernd
Hi Bernd
Interesting topic, read through it. So it's possible either a issue with the sensor itself, or incorrect flat process (exposure/ADU?). My ADU was set to 27000 for the APT flats Aid. I'll try a stack later without the flats (and dark flats)? I'll also try taking some new flats with the flat field generator at full (12V) brightness. Is my ADU correct?
I'm going to try some tests without my rotator in as well as when I check my lights with and without the rotator, there was a clear tilt to stacked lights with the rotator. Of course I can't reprocess without the rotator as it won't match up to the lights.
Thanks for your advise so far, much appreciated :) BTW how did you show the CFA values for each colour? Is that in Pixinsight? If so I'll test that on my other flat sets that used different filters and also on the new set @ 12v.
Andy
-
Ok well I've processed without flats. So my 45 lights, and 40 darks. DSS on the left (forgot to turn off drizzle) and PI's BPP on the right. Weird how BPP didn't remove the amp glow properly but DSS did. However PI did a better job of removing the left side gradient which I think is from sensor tilt as seen in CCD Inspector.
Both images received DBE subtraction and SNCR (green). In the PI image, DBE allowed points to be placed on the gradient, DSS's image didn't.
-
Interesting topic, read through it. So it's possible either a issue with the sensor itself, or incorrect flat process (exposure/ADU?). My ADU was set to 27000 for the APT flats Aid. I'll try a stack later without the flats (and dark flats)? I'll also try taking some new flats with the flat field generator at full (12V) brightness. Is my ADU correct?
A mean of 27000 ADU should be fine. It is the large discrepancy between the weak red channel and the rest of the channels that might pose a problem. That means a light source with more red light possibly could improve the situation. However (as I wrote before), I am not shure whether such a measure is the solution of your problem.
Thanks for your advise so far, much appreciated :) BTW how did you show the CFA values for each colour? Is that in Pixinsight? If so I'll test that on my other flat sets that used different filters and also on the new set @ 12v.
Data of an OSC camera in raw format (CFA) can be split into the 4 CFA channels with the SplitCFA process. Then you apply Statistics to each of the images generated by SplitCFA.
Bernd
-
Ok well I've processed without flats. So my 45 lights, and 40 darks. DSS on the left (forgot to turn off drizzle) and PI's BPP on the right. Weird how BPP didn't remove the amp glow properly but DSS did. However PI did a better job of removing the left side gradient which I think is from sensor tilt as seen in CCD Inspector.
If BPP didn't remove the amp glow, the settings for ImageCalibration were wrong. When I tried ImageCalibration on your data applying only the MasterDark, the amp glow was removed completely.
Bernd
-
Ok well I've processed without flats. So my 45 lights, and 40 darks. DSS on the left (forgot to turn off drizzle) and PI's BPP on the right. Weird how BPP didn't remove the amp glow properly but DSS did. However PI did a better job of removing the left side gradient which I think is from sensor tilt as seen in CCD Inspector.
If BPP didn't remove the amp glow, the settings for ImageCalibration were wrong. When I tried ImageCalibration on your data applying only the MasterDark, the amp glow was removed completely.
Bernd
Can you remember what settings you used? I did a quick and dirty edit with the DSS stack and am much happier with the colours.
-
Tried again with Linear Fit selected for both Darks and Lights. The amp glow is still there :(.
-
Tried again with Linear Fit selected for both Darks and Lights. The amp glow is still there :(.
???
I referred to ImageCalibration. When you tried 'Linear Fit' I guess you referred to ImageIntegration?
I didn't use BPP but performed the image calibration with the ImageCalibration process (Section 'MasterDark': both options 'Calibrate' and 'Optimize' disabled; Section 'MasterFlat': option 'Calibrate' disabled). BPP will yield the same result whith the correct settings.
Bernd
-
Hi Bernd
Ok so I followed the manual integration process from the Inside Pixinsight book. Much better result and zero amp glow :). Here's my finished result fully processed with final framing in lightroom. Maybe I don't need flats at all?
I'm going to have a play with various flat settings (exposure times/ADU values) over the weekend and see how they impact the stacked image. Thanks for all your help :)
Andy
(https://live.staticflickr.com/65535/49433337752_4b0744db3a_b.jpg) (https://flic.kr/p/2ijfM5L)NGC 2174 Monkey Head Nebula V2 (https://flic.kr/p/2ijfM5L) by Andy Thilo (https://www.flickr.com/photos/andythilo/), on Flickr
-
Hi Andy,
the background is somewhat dark from my point of view (slightly too aggressive black point adjustment?), but yes, this image is much better than the results before.
Whether it is worthwhile to optimize the capturing of flat frames or not is up to you. Generally, the flatfield correction is very important and it is not recommended to skip this step. Hopefully you will get some good advice in the CloudyNights thread for the capturing of useful flat frames for this special configuration (ASI294 and dual band pass filter).
I still don't understand what ImageIntegration settings have to do with the removal of amp glow. The amp glow has to be removed in the ImageCalibration step, it has to be removed from each of the light subframes. The critical point is 1. to use a matched MasterDark (gain, offset, temperature and exposure time) and 2. to disable 'Optimize'. You can check whether image calibration has made it by inspecting the individual calibrated light subframes. If this step is successful, there will be no amp glow in the integration result.
Bottomline: it is not possible to remove amp glow by tweaking the ImageIntegration settings, this has to be done before, at the image calibration stage.
Bernd
-
Hi Bernd
I've never manually integrated in PI, I've only ever used BPP. For some reason BPP wasn't correctly removing the amp glow in this case, I tried it twice. Once I did it manually i.e created master dark, calibrated with the lights, cosmetic correction, debayered, star aligned and final integration to produce the master light it worked with zero amp glow.
My darks are from my library and matched to the lights, both 180secs, gain 120, sensor @ -20.
With regards to cloudynights, I've had no responses but I'll do my own tests of flats at various exposures/ADU values and see how they integrate. To that end, if I create a batch of different masterflats, I'll do the SplitCFA and Statistics and see what they look like.
Cheers
Andy
-
Ok well either I'm doing something very wrong, or there's a problem with my flat field generator. So I created some new flats, all at gain 120, camera cooled to -20 and focus in same position as when I took the lights.I also created a fresh batch of BIAS files to use with the tests.
I took 30 subs each with the generator at full brightness (approx 12V), dimmed to 8V and dimmed to 5V. All at ADU values of 20000, 22000, 25000, 27000 & 30000.
I then did some stacking in DSS and PI.
All processing in PI for comparison purposes was identical, ABE and SNCR. This was just to be able to get an idea on image quality.
In DSS I ran the same sequence with all 3 voltages @ ADU3000. All 3 resulting images looked virtually identical. So I then processed in PI. I then ran the same without flats.
In PI, I did a full manual integration as per the book Inside Pixinsight, and also used the BPP script. Again with and without flats.
6 images below show the following:
Top - DSS. Left with flats, right without flats
Middle - PI BPP. Left with flats, right without flats
Bottom - PI Full Manual Integration. Left with flats, right without flats
Again all 6 processed in PI with the Automatic Background Extractor and SNCR (green).
Conclusion, my flats are broke lol. My images are 100% cleaner with no flats and manual integration without flats the clear winner with DSS a close 2nd. For some reason BPP isn't removing my amp glow. No idea why, it used to! So I'm going to send this to the flat field generator supplier and manufacturer to advise. This is the generator I use:
https://www.astroshop.eu/flatfield-masks/gerd-neumann-jr-aurora-flat-field-panel-160mm-12v/p,46378
(https://live.staticflickr.com/65535/49439427497_7674bf759c_b.jpg) (https://flic.kr/p/2ijMZmn)PI_DSS_Flats_Noflats_Compare (https://flic.kr/p/2ijMZmn) by Andy Thilo (https://www.flickr.com/photos/andythilo/), on Flickr
-
The difference between the results of PI BPP and PI manual preprocessing is obviously due to your wrong settings in the 'Global Options' section of BPP: 'Use master bias' has to be unchecked, both with and without a MasterFlat.
I will not speculate about the cause of the fact that your MasterFlat doesn't correct properly, there are some more possibilities except the flat field generator. The ASI294 is a very touchy camera. Using the ASI294 I had severe problems with the "colored background artifact" (see the lengthy thread https://www.cloudynights.com/topic/596025-zwo-asi294-mc-pro/ , examples of deep integrations are in posts #105 and #112. The images show STF auto stretched integration results without any further processing). I could not manage these issues appropriately and finally threw in the towel and purchased an ASI071 which I now use exclusively. I didn't use a dual band pass filter though -- this could still complicate things.
However, in my view your results without applying a MasterFlat don't look bad at all. Certainly you should manage to remove amp glow completely. This is possible with BPP and correct settings as well as with manual preprocessing (see above).
At any rate, your issues are NOT caused by PixInsight, so I would like to finish my answers for now.
Bernd
-
Hi Bernd
I’m not for a second blaming PI. The issues I’m having with BPP are not important, rather just just included as a query. I don’t ever use masters with BPP and leave the boxes unchecked. Why it’s suddenly not removing amp glow, I’ve no idea.
I’ll keep looking for a solution, thanks for all your help.
Andy
-
Just to update, I had a play with WBPP this morning and with optimise dark frames disabled, it removed the amp glow fully.
With regards to the flats, I'm in touch with Gerd and funnily enough, they sell a LPR filter which combats the issues you stated at the start -
https://www.gerdneumann.net/english/astrofotografie-parts-astrophotography/aurora-flatfield-panels/aurora-accessories-aurora-zubehoer/lpr-konversionsfilter-fur-100mm-und-160mm-aurora.html
However I still think there's a problem with my panel because of the dark streak and mottling in the flat images.
-
As a general rule, when using a camera with a CMOS sensor that exhibits amp glow, dark frame optimization should NOT be used, otherwise the amp glow will not be removed completely by subtracting the MasterDark. That means, the option 'Optimize' has to be disabled in section 'Master Dark' of ImageCalibration (of course the same holds for the option 'Optimize dark frames' in 'Global Options' of BPP or WBPP). I had told you my settings for ImageCalibration already in reply #13 of this thread and therefore did not suppose that you used different settings.
Since it is such a small investment, it would not hurt trying the LPR conversion filter, but i am not optimistic. If you decide to purchase it, I would be interested to see a MasterFlat with it and a full calibration result (with MasterDark and the new MasterFlat). Good luck!
Bernd
-
It was my darks. My darks had really bad light leaks, highlighted to me by someone on Cloudy Nights. I retook my 180s darks by putting the camera in my fridge with it's cap on and halfway in it's bag.
Result:-
-
OK, I was on the wrong track concentrating only on the flats. I am glad that you were able to solve this issue, and that the correct calibration procedure (using dark frames, flat frames and flat-darks) results in a flat integration result -- just as it should be.
Bernd