running into Flat subtraction issue

[HadesZ]

Hello
May be you could try calibrating using the method I described in this link:
http://pixinsight.com/forum/index.php?topic=9312.msg59752#msg59752

At least it gives good results for me on my QHY8L.


Regards
Geert

I will try this to see if it replicates the same issue.  but to be honest, I am NOT willing to deal with this everytime I need new flats, or deal with this for every image im trying to process. this is not acceptable to have to deal with. and I would really like to know.. -- is it something to do with the way the QHY10 writes its images/fits , or is it something I am doing wrong in my processing? I never ran into issues like this when I used my DSLR.

im looking for the reason why its happening, JUST as much if not MORE than how to fix it.. 

[HadesZ]

It is a known issue with the QHY10. I suggest using division with DBE to correct this.
Cheers,

Jean Guy Moreau
Quebec

Also, when I try this, DBE will not place, or let me manually place samples in the corners, because the minimum weights in said corners are below 0.000 ( it wont even give me a weight because its so low)

[pfile]

i think you misunderstand how it works - just increase the tolerance to like 6 or 7 and you can place the sample...

the tolerance is not feedback, it's a setting...

rob

[HadesZ]

i think you misunderstand how it works - just increase the tolerance to like 6 or 7 and you can place the sample...

the tolerance is not feedback, it's a setting...

rob

Thanks for this Rob,  I'll have to play with this to see how low I can while still getting rid of the brightness in the corners.

I set tolerance to 7 , corner brightness was gone but the over subtraction of dust motes were still there and very visible. This brings up the question of, if I need to bump tolerance that high anyways, is using a flat even worth it if it's not getting my dust motes out?

And , any suggestions on how high I could generally go with tolerance before it starts knocking out good nebulosity?

Thanks

[pfile]

well, 7 might be overkill. when you look at the DBE interface you'll see a blown-up version of the currently selected sample. you can then adjust the tolerance until there are few black pixels (all black = no pixels are considered, all white = all pixels in sample are considered).

for that first pass if you set very few samples i think you can avoid destroying real nebulosity. it's ok to use DBE 2 or even 3 times; on the 2nd and 3rd passes of course you should reduce the tolerance though.

on your original problem it might help to post links to a bias, a dark a flat and a light and maybe someone can take a a look.

rob

[HadesZ]

thanks again for the help,

below is a link to a dropbox folder where I have the full integration, a single light frame, and master bias, flat, dark.

if anyone is willing to take a look, see if you run into the same issue.. or know of a way to fix the issue and would be willing to try it. id much appreciate your time.

https://www.dropbox.com/sh/ujqptwx7dqcosgu/AACkTY_Topl1MGAfYKgJKCyxa?dl=0


-Tyler

[pfile]

can you post an uncalibrated flat sub?

is the master flat bias and/or dark subtracted?

rob

[bulrichl]

Hi Tyler,

I am inspecting your frames and have some questions:

1) When you calibrate your flat frames with the setup shown in your first screenshot, what is the output to the console?
2) What is your setup for the calibration of the light frames (best is again a screenshot)? Please indicate the console output also for this calibration.

By the way: your integrated image of M 38, M 36 is flipped vertically. What is your setting regarding coordinate origin in Format Explorer/FITS? Did you change this setting lately?

Bernd

[HadesZ]

Rob, there is an uncalibrated flat sub in that folder, the master flat is bias and dark calibrated. I also checked Optimize during this process on the dark

Bernd, I've never changed any of those settings, I went into format explorer to DBL check a coordinate origin setting and did not see one? ( Srry , I may sound confused here, I've never messed with that, ) I do know that the QHY10 mirror and flips (both I think) , but I've never touched orrientstion untill after stacking so figured as long as all my subs and calibration frames are raw orientation from camera I'd be good.


Again sorry, but how would I get u the Output to Console?
, And lights were calibrated with Master bias, dark and flat,  with Optimize and calibrate ONLY checked for the darks.

[bulrichl]

Bernd, I've never changed any of those settings, I went into format explorer to DBL check a coordinate origin setting and did not see one? ( Srry , I may sound confused here, I've never messed with that, ) I do know that the QHY10 mirror and flips (both I think) , but I've never touched orrientstion untill after stacking so figured as long as all my subs and calibration frames are raw orientation from camera I'd be good.

I think in Format Explorer/FITS you should change the setting of 'Coordinate origin' to 'lower left corner'. Then the images are not flipped vertically.

Again sorry, but how would I get u the Output to Console?

When a process in Pixinsight has finished, you can look into the Process Console. There you will find the description about what was done and whether an error occurred. Most processes output important parameters that should be inspected after the process terminated.

, And lights were calibrated with Master bias, dark and flat,  with Optimize and calibrate ONLY checked for the darks.

OK, I think that is the key point here. I calibrated your light frame in two ways:

1) No dark frame optimization
Master Bias unchecked;
Master Dark checked, calibrate not checked, optimization not checked;
Master Flat checked, calibrate not checked.

2) With dark frame optimization
Master Bias checked, calibrate not checked;
Master Dark checked, calibrate checked, optimize checked;
Master Flat checked, calibrate not checked.

Calibration 1) looked OK to me. Calibration 2) (the way you did it) looked strange because of an uneven background color. In Calibration 2) the following warning appeared in the console:
"** Warning: No correlation between the master dark and target frames (channel 0).".
That means, the dark frame optimization algorithm did not find a correlation between the light frame and the Master Dark and as result, no dark frame subtraction was performed at all. That is not what you want. The reason why dark frame optimization doesn't find a correlation between the light frame and the MasterDark is presumably that your sensor has very few hot pixels.

Please try the following:

- Calibrate your flats using only Master Bias, no Master Dark.

For the calibration of the light frames try both of the following posibilities and compare the differences.:
- calibrate your light frames according to 1) or
- calibrate your light frames using Master Bias and Master Flat, but no Master Dark.

Hope that helps!

Bernd

[pfile]

Rob, there is an uncalibrated flat sub in that folder, the master flat is bias and dark calibrated. I also checked Optimize during this process on the dark.

well, the flat filename was FLAT 1_2sec_1x1__frame1_c.xisf which indicates it had been calibrated... and i assume the master flat was made from calibrated subs, so...

rob

[bulrichl]

I am intrigued by the following observation: when I want to rotate the light frame "M36 + M38_60sec_1x1__frame6.fit", I get the note:

"The image contains an astrometric solution that will be deleted by the geometric transformation. This side effect could be irreversible. Proceed?".

I have not seen this note before. Does anyone know what it means?

Bernd

[HadesZ]

Rob, there is an uncalibrated flat sub in that folder, the master flat is bias and dark calibrated. I also checked Optimize during this process on the dark.

well, the flat filename was FLAT 1_2sec_1x1__frame1_c.xisf which indicates it had been calibrated... and i assume the master flat was made from calibrated subs, so...

rob

sigh.. your right,,,  sorry about this, i uploaded a non calibrated one,  I thought I was in the right folder when I selected that but I was not.

and Bernd I am going to try those suggestions and see how it goes! especially now since you may have been using an already calibrated flat frame since I was an idiot and uploaded one that was already calibrated.


I want to say thanks again for the time you guys are putting into helping me out!

[pfile]

I am intrigued by the following observation: when I want to rotate the light frame "M36 + M38_60sec_1x1__frame6.fit", I get the note:

"The image contains an astrometric solution that will be deleted by the geometric transformation. This side effect could be irreversible. Proceed?".

I have not seen this note before. Does anyone know what it means?

Bernd

it means that the file has WCS coordinates in the fits header, and by rotating the image the solution becomes invalid... so PI deletes the WCS data from the fits header.

rob

[bulrichl]

Yet one important note regarding orientation:

I recommended to change the setting of 'Coordinate origin' in Format Explorer/FITS from the default ('Upper left corner (up-bottom)') to 'Lower left (bottom-up)' so the images are not vertically flipped any more. However, if you decide to do so, you must not use the old Masters any longer. All Masters have to be built new from the original QHY10 FIT files after this change.
---

When I inspected yout MasterBias and MasterDark it catched my eye that both show a checkerboard pattern. In the histogram, two peaks show up, in the MasterBias at 756 (R + B channels) and 799 ADUs (G channels), in the MasterDark at 850 (R + B channels) and 890 ADUs (G channels).

I don't have a CCD camera but only a DSLR, and my bias and dark frames show only one peak.

My question: Is a difference of about 40 ADUs between R + B channels and G channels regular? In the dark it makes no difference what filter is ahead of the sensor element, so it must be just a different bias setting of the corresponding sensor elements - but why should the manufacturer do that? I don't know.

Bernd