Author Topic: Issues creating DSLR RAW FLATS  (Read 1650 times)

Offline kcharters

  • Newcomer
  • Posts: 2
Issues creating DSLR RAW FLATS
« on: 2018 December 01 04:53:57 »
Hello. I've been imaging for just over a year using an unmodified Nikon D5200. Up until now I have been happily processing my bias, dark, flat and light images all as RGB and getting some reasonable results. However, after further research it seems that I should be using the PURE RAW settings for DSLR RAW images in the Format Explorer, but so far I have had no success at all in producing a master flat. I've been puzzling over this for some days now and I'm convinced I'm doing something wrong so I started examining some of the individual images. I opened 1 each of a bias, dark, flat and light images as both RGB Debayered and the same images as Pure Raw. When I compare the RGB Debayered and Pure RAW images for the BIAS, DARK and LIGHT Frames you can see similar structures between each. However, the Pure RAW FLAT image appears to have lost the structure that is clearly seen in the RBG Debayered version. I have attached a screen shot of Pixinsight with all the files compared side by side. Can anyone shed any light on what I'm doing wrong? I've been through several online tutorials on pre-processing DSLR images and I also have Warren Keller's 'Inside Pixinsight' book but I'm still at a loss and this is now driving me to distraction.

Thanks in advance for your help.

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Re: Issues creating DSLR RAW FLATS
« Reply #1 on: 2018 December 01 08:56:20 »
Hi <???>,

I jabe some basic questions that I would need answering before trying to get to the bottom of your issues:

Quote
Up until now I have been happily processing . . .
What do you mean by this - processing inside PixInsight, or processing with some external software?

Quote
. . . my bias, dark, flat and light images all as RGB
First question - presumably you are letting the camera produce the RGB image from its own 'internal' RAW grey-scale data obtained from the imagiing sensor?
Second question - are you using Dark frames exposed for the same lemgth of time as for your Lights?
Third question - are you using FlatDark frames exposed for the same lemgth of time as for your Flats?
Fourth question (really more of a statement) - obviously, using a DSLR means you have very poor control of the thermal behaviour of the camera electronics

Quote
using (the) PURE RAW settings for DSLR RAW images
Yes - by doing this, you should be getting the 'pure' data, as obtained direct from the camera sensor itself, with no in-camera processing whatsoever. Any processing of the image by the camera software must, unfortunately, be considered as ' >:D black magic  >:D ' - simply because the process is not publicised by the DSLR vendor. Given the opportunity, PixInsight always prefers to start yhe calibration process with RAW data.

Quote
no success at all in producing a master flat
Well a couple of things might be working against you here. First, as already menrioned, are you calibrating the Flat with an appropriate MasterFlatDark (as opposed to your MasterLightDark), and associated with this then mas to be the issue of even needing MasterBias frames at all (they are only needed if your exposure times for Lights and Darks - or Flats and FlatDarks - differ).
But, perhaps more importantly is what the camera internal software is going to do when you ask it to take images with very low ADU vaues and to then process these, internally, from a CFA-based greyscale format into a deBayered RGB format. It may well be a process that the vendor gave no consideration for, and so the internal software just has to 'do its best' - applying some arbitrary set of rules in the hope that the results will be acceptable and, more importantly, repeatable between calibration frame types. It is almost certain that the reslts will be neither acceptable, nor repeatable.

Quote
Can anyone shed any light on what I'm doing wrong?
Probably nothing, but perhaps everything ;)
So much depends on those critical steps of image acquisition - where detailed information presented for analysis and critique is vital. Presenting PixInsight with detailed image data sets for calibration is, fundamentally, the single most important step that you can take.

Hope this helps.
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline kcharters

  • Newcomer
  • Posts: 2
Re: Issues creating DSLR RAW FLATS
« Reply #2 on: 2018 December 01 10:15:41 »
Hi Niall, thanks for your response. So far all my processing has been done using Pixinsight. Within the Format Explorer and the DSLR-RAW section I have selected 'Pure RAW'. Dark frames (50) were taken immediately after the Light frames were taken at the same ISO setting and the same exposure time. I have 200 Bias frames, and 50 Flats. I do not use Flat Darks. The issue (at least to me!) appears to be that when I load the Nikon RAW (.NEF) frames into Pixinsight, all the Flat frames are identical and don't appear to contain any useful data, that's even before calibration and stacking!. I've been following this tutorial:

https://www.lightvortexastronomy.com/tutorial-pre-processing-calibrating-and-stacking-images-in-pixinsight.html#Section3

I've attached another screen shot showing the same .NEF flat file opened in Pixinsight as Pure RAW alongside one openedas RGB Debayered and zoomed right in to the bottom right corner of each image, but this just doesn't look right to me. On the Pure RAW image each block of 4 pixels (RGGB) look identical across the entire frame but, as you can see, there is a lot more information in the RGB Debayered image.

I am currently blindly following the above tutorial and have got as far as creating a MAster Flat so I will continue with the process of calibrationg, cosmetically correcting and registering the Light frames using my Super bias, Master Dark and Master Flat, then debayer the calibrated, CC'd and aligned Light frames and then integrate them to see what the results are. I will post agagin when I have done this

Offline STEVE333

  • PixInsight Addict
  • ***
  • Posts: 231
    • sk-images
Re: Issues creating DSLR RAW FLATS
« Reply #3 on: 2018 December 02 18:20:41 »
I've been following this tutorial:

https://www.lightvortexastronomy.com/tutorial-pre-processing-calibrating-and-stacking-images-in-pixinsight.html#Section3

I am currently blindly following the above tutorial and have got as far as creating a MAster Flat so I will continue with the process of calibrationg, cosmetically correcting and registering the Light frames using my Super bias, Master Dark and Master Flat, then debayer the calibrated, CC'd and aligned Light frames and then integrate them to see what the results are. I will post agagin when I have done this

Thanks for the reference. It has helped me to create a better MasterFlat for my DSLR.

I never realized that the BYEOS Histogram is useless for estimating the proper exposure for taking Flats.

Hope your efforts are rewarded with success.

Steve
Telescopes:  WO Star71 ii, ES ED102 CF
Camera:  Canon T3 (modified)
Filters:  IDAS LPS-D1, Triad Tri-Band, STC Duo-Narrowband
Mount:  CEM40 EC
Software:  BYEOS, PHD2, PixInsight

http://www.SteveKing.Pictures/