PixInsight Forum

PixInsight => Image Processing Challenges => Topic started by: CepheidSoul on 2019 June 12 12:28:34

Title: Flats Correction on M81
Post by: CepheidSoul on 2019 June 12 12:28:34
Hello Team!

I'm fairly new to the PI game. I started my first image in 2016 with PI but have only done 3 or 4 pictures using the software in total. On this one, I am having a real hard time in pre-processing. When using flats to calibrate my light frames, I seem to get clean subs for my the L channel, but all my colour channels have this ghosting as if the Flat is inverted against the image or overcorrecting somehow. Here's what I've tried so far:

Creating new Master Flats using lower or higher skewed ADUs of the flat sub frames
Creating new Master Flats without the bias or master darks (in case they were messing them up somehow)
Adjusting the image integration for the master flat using different normalizations (this was the only thing that seemed to make an impact on the final version but even then wasn't quite right)

I have the raw data and the flats linked below. The flats are twilight flats.

I have added two sample subs as an example. One of from the L Channel and another from R

Thanks everyone! I'd appreciate any help

Title: Re: Flats Correction on M81
Post by: Greg Schwimer on 2019 June 12 14:04:02
Hi Matthew,

I've seen the "inverted flats" look before, usually when something goes wrong with the darks during calibration.

In taking a look at your data I found a few things of interest:

 - the binning between your L flats and lights is different. These will need to be the same geometry in order to work. You *could* fudge it and up/down scale if you wanted but to get the best outcome they should be the same.

 - Your darks appear to have an object in them at the center (possibly M81 core?). I would think maybe this is RBI ghosting, but blinking through them the object moves, almost as if it is dithered. How did you capture them? I'm curious if your camera uses an electronic shutter?

 - Your bias master does not have this ghosting so that's good

 - I ran everything through BatchPreProcessing (BPP) using defaults and got a similar result to you.

 - Ran BPP again without the master bias - same result

 - Ran BPP again with 'Optimze dark frames' disabled <- this seems to fix your problem

So, give the calibration routine you're following at try with dark optimization disabled and see how that works for you. If dark optimzation doesn't work for you maybe consider building a master dark library for each temp/exposure you might image at.

Result below (RGB only).

- Greg
Title: Re: Flats Correction on M81
Post by: CepheidSoul on 2019 June 13 10:48:54

Thank you so much for taking the time to look at this! I tried to replicate what you did in BPP and got it to work. Same with manual image calibration and integration.

It does look like not using the dark optimization does introduce a lot of noise into the masters. So trying to see if I can find a work around for that or determine if there is something that I did to the master dark that is resulting in the flats working the way they do.

You're right about the blob at the centre of the dark frames. The camera at the observatory uses a mechanical shutter so I am wondering if some of the light from the target is coming through. We have had issues with the shutter sticking in the past and perhaps it requires adjustment again.

I also noticed that in the light master that the donuts look "raised" as if they too are overcorrected. I am still trying to determine how to repair that too. Would appreciate any thoughts.

Wow, each of these is a such a journey!

Thank you so much again!

Title: Re: Flats Correction on M81
Post by: Greg Schwimer on 2019 June 13 11:34:23
I'm happy to help. Astrophotography is indeed a journey and the learning curve is always present.

Dark optimization usually "just works" but sometimes it doesn't as you're finding. The best solution is to use exposure and temperature matched darks so you don't even need it. Looking at the fits headers it seems that you have this. If you're seen a noise difference between when it is used and not used it may need adjustment. In this case you'll need to try adjusting the settings in ImageCalibration re: dark optimization. I've not done this before - perhaps someone else has something to add on this point.

Before you mess with those settings you can try calibrating the flats w/ just the bias or use darks specific to the flats to match temp/exposure. If the flats are short enough using a master bias to calibrate them may be all you need. It depends on the camera. I'd say give it a try and see what you get.

You might also try using the CosmeticCorrection tool with a master dark as a reference. This will help clean up lots of things in the lights like hot/cold pixels and bad columns/rows. I do see some dark columns in your lights so it may be worth investigating this as a part of your process.

I've had raised donuts myself. Unfortunately not the tasty kind. ;)   In most cases this is caused by a shift in the projection of the dust on the sensor, whether by just movement of the dust, inaccuracies in the filter wheel, rotator, difference in focus, etc. The CloneStamp tool may come in handy here. Some people use clever formulae in PixelMath.

Share the final when you're happy with it!

- Greg

Title: Re: Flats Correction on M81
Post by: CepheidSoul on 2019 June 13 11:42:23
Thanks again! And yes, once I've got it somewhere I'm happy with. I will definitely post it!