Author Topic: FLATS !  (Read 8817 times)

Offline TobiasLindemann

  • PixInsight Addict
  • ***
  • Posts: 126
    • View Profile
    • Trackingstation
Re: FLATS !
« Reply #15 on: 2016 July 20 04:38:43 »
Paul,
do you have used your original darks that matches the lights too? If not have it a try. Just throw all darks in the BPPS, PI sorts them and uses the right ones for the lights and the flats. I think the color is produced by calibrating the lights with the (wrong) short exposed flatdarks.
But now I don't  think any more this is the main problem. Is it possible to upload your data somewhere (Dropbox)?

Greetings
Tobias

Offline gvanhau

  • PixInsight Old Hand
  • ****
  • Posts: 345
    • View Profile
Re: FLATS !
« Reply #16 on: 2016 July 20 13:49:04 »
Hello

I think that you can't use BPP with QHY8L, because it acts different when taking short exposures than when taking long ones.  (On short exposures it takes 2 images for each half of the image, on long exposures (>4s) it takes only one image, wich has some amp glow that builds up during readout of the second half of the image.

In my experience, what I do is the following:
I take the flats with very short exposures, and calibrate them only with bias frames.

I then do a linear fit of the master flat using this procedure:
http://pixinsight.com/forum/index.php?topic=9312.msg59752#msg59752

Finally I calibrate the lights only with darks and the master flat obtainded from the above procedure.

This workflow works almost all the time for me.

Regards
Geert
Geert Vanhauwaert

Offline pscammp

  • PixInsight Addict
  • ***
  • Posts: 116
    • View Profile
Re: FLATS !
« Reply #17 on: 2016 July 20 17:48:24 »
Tobias,
    I will try this as soon as I get a chance and see what happens then report back, Thank you

I will load up x5 of all my different types of frames to my Dropbox, again, as soon as I get time,
you professionals may Identify what the real problem is where as I can not.

gvanhau,
    I will also give that a go and also report back too

I cant thank you enough for all the help your giving me guy's, it's so frustrating but somehow,
someway, with your help I know we'll get there in the end.

Many Regards
Paul



Offline pscammp

  • PixInsight Addict
  • ***
  • Posts: 116
    • View Profile
Re: FLATS !
« Reply #18 on: 2016 July 21 04:01:20 »
Tobias,

Tried what you suggested by loading both types of Darks into BPP and the result was pretty much the same
as before but there is no evidence of the extra Orange colour the images were showing before so you were
right about that part.

When I ran this test I used Flats with an ADU of 8000 instead of 17,000 in case it made a difference but the
illumination in the corners looks almost identical as when using the 17,000 ADU Flats. At the bottom of the
image is a screenshot of the 'Master' folder created by BPP so you can see what it produced.

See attached pic and compare it to the 'Test_1' image.....8000 ADU v 17,000 ADU

I'm currently loading up x5 pics of all my image types to Dropbox....This may take some time  LOLOL

Regards
Paul

Offline pscammp

  • PixInsight Addict
  • ***
  • Posts: 116
    • View Profile
Re: FLATS !
« Reply #19 on: 2016 July 21 04:07:39 »
Ok,
   So here I will add public links to the files as I get them loaded up, hopefully I can modify this
post over time to add them so they are all in one post. I have compressed them on Win 7 using
the Zip function so hopefully it's a lossless compression, x5 of each type should suffice I hope.

Bias Frames (zero second as suggested by QHY8L manual):
https://dl.dropboxusercontent.com/u/4938813/Bias.zip


Dark Frames (25 seconds to match Light frames):
https://dl.dropboxusercontent.com/u/4938813/Darks.zip


Flat Dark Frames (10.71 seconds to match Flats exposure time):
https://dl.dropboxusercontent.com/u/4938813/FlatDarks_30000.zip


Flat Frames (30,000 ADU @ 10.71 seconds to avoid dual readout mode below 4 seconds):
https://dl.dropboxusercontent.com/u/4938813/Flat_30000.zip


Light Frames (25 sec exposure time):
https://dl.dropboxusercontent.com/u/4938813/M13_Lights.zip


Well there you go, finally got them up.

So hope you can help me identify the problem

Many Thanks for looking
Paul
 
« Last Edit: 2016 July 28 11:05:17 by pscammp »

Offline gvanhau

  • PixInsight Old Hand
  • ****
  • Posts: 345
    • View Profile
Re: FLATS !
« Reply #20 on: 2016 July 21 07:23:11 »
Hello Paul

The zero secconds bias frames will not work with long exposures, because the readount mode is different.
If you take flats over 4 secconds you have to take flatDarks.
Note that more than 4 secconds flats have amp-glow  and bias of zero seconds do not have amp glow.

I this case, you have to integrate the set of flatDarks (without calibration) and then calibrate the flats only using the resulting master flatDark.

Regards
Geert
Geert Vanhauwaert

Offline jkmorse

  • PixInsight Padawan
  • ****
  • Posts: 931
  • Two questions, Mitch . .
    • View Profile
    • Jim Morse Astronomy
Re: FLATS !
« Reply #21 on: 2016 July 21 11:30:06 »
Paul,

I think there is some confusion on what ADU count does for flats.  Whether you have 8000 counts, 17,000 counts or 32,000 counts, so long as you get above the floor (best to be above 15,000) and so long as you stay linear (32,000 or less for most CCDs that have antiblooming gates), you are going to get pretty much the same result when you use any of those those flats to pre-process your images.  The key is that flats are divided out, not subtracted and they serve to "flatten" out the light variations in your CCD/CMOS.  Changing the ADU doesn't change the math.  The more important question is what do your flats look like?  Also, how are you shooting your flats?  That's an important piece of the puzzle too.  I will take a look at the flats you posted when I get to a computer that can read them (or if you could just attach one here, that would be great).

Best,

Jim   
Really, are clear skies, low wind and no moon that much to ask for? 

New Mexico Skies Observatory
Apogee Aspen 16803
Planewave CDK17 - Paramount MEII
Planewave IFR90 - Astrodon LRGB & NB filters
SkyX - MaximDL - ACP

http://www.jimmorse-astronomy.com
http://www.astrobin.com/users/JimMorse

Offline pscammp

  • PixInsight Addict
  • ***
  • Posts: 116
    • View Profile
Re: FLATS !
« Reply #22 on: 2016 July 21 18:02:17 »
jkmorse,

To take the Flats I use the following electroluminescence panel:

http://www.earlsmann.co.uk/lighting-components/electroluminescent-products/el-panels/

The inverter which comes with it is powered directly from a 75Ah leisure battery. It gets switched on around
5 mins before I begin acquiring the Flats and stays on until I'm finished. To get the Flats exposure time above
4 seconds (to avoid dual readout mode) I have about 8 sheets of plain white paper over the panel.

gvanhau,

Thanks, I'll give it a go as soon as I can

Regards
Paul

 

Offline TobiasLindemann

  • PixInsight Addict
  • ***
  • Posts: 126
    • View Profile
    • Trackingstation
Re: FLATS !
« Reply #23 on: 2016 July 22 01:26:51 »
Paul,
sorry I have no real solution, but you can use the following, until you have solved the problem. This is not very "PI-like" and I would not recommend this if I would see another way.
It seems like your flat has too much contrast. To lower it you can use the CurvesTransformation with a s-curve as you see in the picture below. For the shadows I used an input of 25 and an output of 35, for the highlights I used an input of 75 and an output of 65. Just apply the transformation on your masterflat and use the result as a new masterflat in the BPPS. As you see in the picture the overcorrection is not perfect, but much better with the corrected flat. Maybe you can optimize it with other in- and outputs in the CurvesTransformation.

Greetings
Tobias

Offline pscammp

  • PixInsight Addict
  • ***
  • Posts: 116
    • View Profile
Re: FLATS !
« Reply #24 on: 2016 July 22 05:52:33 »
Many Thanks Tobias,

Looks great Tobias, At this point 'almost' anything will work for me just though pure frustration.

Unfortunately I'm not quite out of the woods yet, tried the CurvesTransformation but as you'll see from the attached pic, it didn't work for me. So, I have to be doing something wrong along the way. Following is a step by step of what I did, hopefully you will see what I did wrong.

The FlatDarks I used were the same duration as the normal Flats at 10.71 sec each which were the 17,000 ADU set.

1. Integrated all FlatDarks with the following settings: Average, No Normalisation, Don't Care (All Weights = 1), Winsorized Sigma Clipping, No Normalisation, High 3.0 Low 3.0 - Saved

2. Calibrated all Flats using Master FlatDark - Used ImageCalibration with Master FlatDark in 'Master Bias' box with 'Calibrate' unchecked All other options were unchecked too

3. Integrated Calibrated Flats using following settings: Average, Multiplicative, Don't Care (All Weights = 1), Percentile Clipping, Equalize Fluxes, Low = 0.010 High = 0.010

4. Apply CurvesTransformation to Master Flat:  25/35 & 75/65  to resulting master Flat - Saved

5. Open BPP - Optimize Dark Frames set to disabled or Amp Glow wont be removed,

a) No Bias frames were loaded
b) Loaded Darks (full length Darks which match Light Frames in duration)
c) Load Curves modified Master Flat and checked 'Use Master Flat' - Blue star appears
d) Load all Light frames

Image registration 'Pixel Interpolation' set to Auto
Image Integration set to Average and Winsorized Sigma Clipping - High = 3.0 & Low = 3.0
Darks set to Average and Winsorized Sigma Clipping - High = 3.0 & Low = 3.0

Run

Result is attached

Where did I go wrong ?

Regards
Paul
 

Offline TobiasLindemann

  • PixInsight Addict
  • ***
  • Posts: 126
    • View Profile
    • Trackingstation
Re: FLATS !
« Reply #25 on: 2016 July 22 06:46:34 »
Hm, sorry I do not see any mistake. If I would do it manually, I would do the same.
The only different to my workflow is, that I used the BPPS from the beginning. My procedure was the following:
1. Load all your data in the script with standard settings (except optimize dark is unchecked and the calibrate only box is checked) and run it.
2. Open the resulting masterflat and do the CurvesTransformation (an in- and output of 0->0.125 and 1->0.875 as you see in the picture will do a slightly better job, than my last try with the s-curve)
3. Open the script again and load your lights and the resulting masterdarks and biases of your last session and the new corrected masterflat / check masterdark, masterflat and masterbias
4. Run the script with registration and integration
Tobias

Offline gvanhau

  • PixInsight Old Hand
  • ****
  • Posts: 345
    • View Profile
Re: FLATS !
« Reply #26 on: 2016 July 22 17:22:53 »
Hello Paul

I did a test on your images, using my workflow.
If you are interested, I can share the proyect

On the left image I integrated calibrated ligths with normal master flat. On the Right is the image integrated using the Adapted master Flat.

Regards
Geert Vanhauwaert

Offline pscammp

  • PixInsight Addict
  • ***
  • Posts: 116
    • View Profile
Re: FLATS !
« Reply #27 on: 2016 July 23 13:54:36 »
gvanhau,

Go for it....The More the merrier !

Tobias,
Still trying to get time to test, sorry, been a hectic weekend, really sorry.

Cant thank you both enough, your so kind

Regards
Paul

Offline pscammp

  • PixInsight Addict
  • ***
  • Posts: 116
    • View Profile
Re: FLATS !
« Reply #28 on: 2016 July 24 06:45:09 »
Tobias,

Quote:
3. Open the script again and load your lights and the resulting masterdarks and biases of your last session and the new corrected masterflat / check masterdark, masterflat and masterbias

Sill have no time to try this out but I do have one question ref step 3. When you are referring to the Lights, is this the lights which were calibrated, registered etc in the first BPP run or my original untouched lights ? The first BPP run produced Lights with the _c suffix and another set with _c_d_r suffix.

Thanks
Paul

Offline TobiasLindemann

  • PixInsight Addict
  • ***
  • Posts: 126
    • View Profile
    • Trackingstation
Re: FLATS !
« Reply #29 on: 2016 July 24 06:48:11 »
Paul,
I meant the original lights, sorry for the confusion.

Tobias