Flats Over-correcting?

[iksose7]

Hi, i have just entered the CCD world and am having a little difficulty with flats. They seem to be over correcting my images. I am not sure if this is a problem with the flats exposure or if its me missing something when stacking. This is something i never had a problem with when using a DSLR.

At the moment i am just using the BatchPreProcessing script. The single flat frames themselves look good. Any ideas are welcome, i will provide as much info as i can to help iron out the issue. Below i have included a master flat, image with flat correction and image without flat correction.

Thanks
Callum

[msmythers]

Callum,

A quick look at your flat issue with the supplied images I don't see the problem your seeing. First I took your jpg's and did a SampleFormatConversion to 32 bit and saved those images. Then I used ImageCalibration and got a basic flat calibrated image.

Here is a screen dump showing the images and ImageCalibration.


Mike

[iksose7]

Hi Mike, thanks for looking

It looks like the image hasnt been over-corrected which is good but it doesnt appear to have removed the large dust bunnies in the image? The galaxy also appears to have lost all contrast.

Any ideas why that has happened?

Callum

[msmythers]

Callum,

I would say some is because I started with 8-bit jpg's. Not seeing the original subs before making a master it's very hard to know. I was just concerned with the over corrected flat. You can look at an original flat sub and see if the dust shows.


Mike

[iksose7]

Thanks Mike,

Its getting late here, i'll run them through ImageIntegration tomorrow with the FITS and see what kind of result i can get. Would be good to get to the root of the issue.

Callum

[iksose7]

I have been trying and i just cant get a result like you have unless i use the jpegs as you have. Cant get my head around it. I am not very clued up on the technical workings of PI so dont know why it works like this.

Any other ideas would be great!

Callum

[jkmorse]

Callum,

I will be interested in your result.  I have been running into similar issues that are not plaguing others and I fear it may be a configuration issue on certain machines.  That may be why you are seeing the problem and Mike is not, using the same raw data.  Please post your follow-on tests.

Best,

Jim

[jkmorse]

Callum,

We posted at the same time.  I am planning to do some detailed testing to see what is causing this at least on my machine and will post if I find anything of interest.

If interested, I do my processing under Windows 8.1, using a Samsung 880Z laptop with 16Gb of Ram.

Best,

Jim

[msmythers]

Callum

My suggestion at this time would be to upload the fit files to a site like dropbox or google drive and share them. Then myself and others can also try to see the problem.


Mike

[iksose7]

Hi Jim,

I wouldnt of thought our hardware would cause issues with the stack, but i could be wrong. Look forward to seeing what you find if you get to the root of the issue.

Callum

[iksose7]

Hi Mike,

Here is the link to the Google Drive files: https://drive.google.com/folderview?id=0B9LTQzt_Fnanfmh4VW9MNUZ1eE1zVUpCQVN3OFNrbU80NGp5WDNNZE5YRWVpR3JRUTNEd0U&usp=sharing

Hopefully that works, its not as easy as it should be to share a file like this with Drive. let me know if you run into any difficulties attempting to download.

Callum

[msmythers]

Callum

I am not a CCD person so I don't know anything about acquisition of subs. The lights that you supplied in your Google drive folder seem to have already been calibrated with flats maybe. I'm not sure. There was a text file with the file name of L_2015-05-12_01-50-21.stackinfo.txt . I decided to use the Blink tool and first remove all light frames that had bad star shapes. That left me with 10 subs. Then I opened all 10 light subs in PI and applied the same AutoSTF to each sub. The brightness is all different for many of the subs. Also the subs look flat like they have already been calibrated with flats.

I then opened all the flats. I did not need to apply STF. 14 of the flats are very similar. 4 are darker. 2 are just white. The ones that are not white look like normal flats, lighter in the center then the edges.

No matter how I calibrate the lights using these flats I get what you got. If I don't use the flats I get a reasonable image that is easily processed using the normal tools like DBE/ABE and so on.

It seems to me the lights have had some other processing done. Again I am not a CCD person so I have no idea.

Here are screen captures of the flats and the lights.the M101 image was procced using the BatchPreprocessing script with the 4 darker lights subs, the master dark and bias and no flats. I used ABE and then HistogramTransformation.

Hopefully others can tell you what is going on.

Mike

[iksose7]

Hi Mike, thanks for the effort!

The single light frames are uncalibrated, calibrated images are moved to a separate folder and labeled accordingly. You can still see all the hot pixels ect in the single subs. I am lucky enough to have a fairly flat field that doesnt really need flats for correcting if it wasnt for removing dust bunnies! I must of accidentally left 6 test flats in the folder, thats what those bright and dark frames are.

The brightness difference of the light frames is just caused by the actual darkness of the night sky, i have always just thrown all the subs into a stack regardless of the brightness. As for star shapes, this is not going to be the final image, this is just to see what the data looked like, the plan is to sink around 50 hours into this image if i can sort the issues out.

I wonder if its the flats themselves? Being new to CCD imaging i have found taking flats to be extremely difficult compared to DSLR. I use a white screen and tshirt method, always have. But with the CCD i have to fold the white tshirt over twice, then put a black tshirt over and set my flat panel to its lowest brightness otherwise the frames are pure white, even with 1 second exposures! Its like the CCD is too sensitive. Can anybody comment on this? I have never heard of anyone using a black tshirt for flats, but i just cant do it any other way.

Callum

 

[NGC7789]

I am lucky enough to have a fairly flat field that doesnt really need flats

I would say this is the crux of the issue. Your flats indicate a system that is anything but flat. They show significant drop off from the center to the edge. If in fact you have a flat field then your flats are off and you are not really measuring the flatness of your system. My guess is your light source is not even. There is a simple way to confirm this. Take two sets of flats. With one set rotate the light source or the camera 90 degrees relative to the other set. Treat one set as lights and calibrate using the other set. If your light source is even this "flattened flat" should be a smooth grey with no (or very little) variation across the frame.

[iksose7]

The more i think about it the more i think the flats are off. Looking at the single light subs its hard to see any serious gradients which makes me think i do in fact have a reasonably flat field. When the images are stacked however a gradient does become a bit more noticeable in each corner but nothing that even a quick dose of ABE cant deal with.

I dont see how the light source could be un-even though. The light had to make it through 3 layers of fabric, 2 white, 1 black. I would of thought it would disperse the light evenly enough?