Guide to Preprocessing of Raw Data with PixInsight

bulrichl

Well-known member
The current version was revised on March 7, 2024.

This is the structure of the text:

Preface

A Camera Hardware and Control Software

A1 Setup and Properties of Digital Cameras used for Astrophotography
A2 Quality Settings, Camera Drivers, Image Acquisition Software and File Formats
A3 Metadata

B Calibration Frames, Master Calibration Files and Image Calibration
B1 Why Do We Perform Image Calibration?
B2 Types of Calibration Frames
B3 Settings and Conditions for Image Acquisition of Calibration Frames
B4 Generation of Master Calibration Files
B5 Light Frame Calibration with PixInsight's ImageCalibration Process
B6 Inspection and Judgement of Calibration Results

C Post-Calibration Preprocessing Steps
C1 Linear Defect Correction (LDC) [Only if linear defects are detected]
C2 Cosmetic correction
C3 Debayer [Only for OSC Cameras]
C4 Subframe Weighting with SubframeSelector
C5 Registration
C6 Local Normalization
C7 Integration
C8 Fast Integration
C9 Drizzle Integration [Recommended]
C10 Cropping

D Automation of Preprocessing: WeightedBatchPreprocessing (WBPP) Script

E Appendix

E1 Types of Resources for PixInsight
E2 Resources for Processes and Scripts that Are Mentioned in this Guide
E3 List of Abbreviations

Endnotes

References




Bernd
 

Attachments

  • 20240307_PreprocessingOfRawImageDataWithPixInsight.pdf
    719.2 KB · Views: 157
Last edited:
Great guide - very useful, can I ask what debayer pattern you're using for your ASI294 ? I've been using RGGB but I'm not entirely sure it's correct as it's giving me really noisy blotchy backgrounds even after DBE.
 
Very informative and helpful, thanks.
I have one question though, that I haven't found the answer to;
I only use DSLR's for my astrophotography. Shooting lights and darks under the same circumstances is understandable, but how about flats and biases? Can you have one set of these for all your lights and darks, or do you need different sets for different ambient temperatures, ISO settings etc?

Best regards,
Lage
 
I have one question though, that I haven't found the answer to;
I only use DSLR's for my astrophotography. Shooting lights and darks under the same circumstances is understandable, but how about flats and biases? Can you have one set of these for all your lights and darks, or do you need different sets for different ambient temperatures, ISO settings etc?

Many people create a master bias and use that (or a super bias as it's called) and it can be done at any time and any temperature. Typically, I do flats using nina flat wizard at the end of observing (using sky flats during morning hours). Nina will calibrate the flats to optimum exposure. Goes so quick I haven't tried to do master flats and save them. Plus, I move my telescope around a lot so dust won't necessarily always stay in same place long enough for masterflats sets to remove motes reliably.
 
What are people doing with their ASI533? There is no real ampglow to speak of. I've seen a lot of discussions of people not using darks and some even dropping bias if they have good dark flats, flats and lights. I'd love a similar capable monochrome to the 533... would simplify processing.
 
For the ASI533MC (and its big brother, ASI2600MC), I've have had great success with darks, dark flats, flats but no bias. I now use WBPP with minimal PP. I started out just testing it with no flats and darks and the images were truly amazing, due to low noise and no amp glow, especially considering nothing was done with the lights. Of course, if your optics are dirty, the image will look nasty. So, not a good practice.
 
I did some tests w/my 533, tried it with just darks, no dark optimization, no bias, then calibrated the same frames in the standard PI workflow, i.e. bias, and calibrated and optimized darks. Resulting frames were nearly indistinguishable. Pretty sure the sensor is so good it won't matter what route you take. :)
 
Let's just say, compared to when I used SX CCD cameras, which are very good, the 533 and 2600 CMOS are very forgiving cameras. They make the hobby much more enjoyable, in my opinion. Especially so, when you do not (like me) have a lot of clear nights to experiment and keep tweaking things.
 
Let's just say, compared to when I used SX CCD cameras, which are very good, the 533 and 2600 CMOS are very forgiving cameras. They make the hobby much more enjoyable, in my opinion. Especially so, when you do not (like me) have a lot of clear nights to experiment and keep tweaking things.
Glad to hear it--my ASI533MC-Pro is my first cooled camera, looks like I started with a good one. :)
 
Been happily just processing with Flats, Flat darks and Lights using WBPP. Super simple. Love this camera!
 
Thanks for this. As an ASI294MC Pro owner, this really helps pull a lot of things together. Hopefully I can improve my calibration results now - have already removed the amp glow which wasn't completely, and look forward to some clear nights to see how everything else goes!
 
This article is exactly what I was looking for. In clear language, it described the WBPP processes with supporting information. Often we hear-read of how complicated Pixinsight is to learn. I've struggled with Photoshop for years, obtaining inconsistent results and not understanding why. I simply do not "get" Photoshop. In comparison, Pixinsight (although challenging) makes sense to me. My challenge over the past few months (through much reading and video watching) has been to understand the basic, sequential steps for processing OSC CMOS pictures. Your article has put a lot of my mis-understanding in context and scope.

Thank you much for this! Now on to learning post processing!

- Jerry
 
Very useful guide - thanks!

But I tried using Local Normalization files in ImageIntegration and cannot. You say to load the .xnml files created by LN into ImageIntegration together with the aligned files via the Add Files button. I can load the aligned files just fine. But when I navigate to the folder where the .xnml files exist, PI sees no loadable files and does not recognize the .xnml extension. How can I load the .xnml files into ImageIntegration? I checked the forum and found nothing useful.
 
did you click the "Add L. Norm Files" button in ImageIntegration? you need to use that button so that II recognizes the extension.
 
Today I updated my guide "Preprocessing of Raw Image Data with PixInsight". The PDF file can be downloaded from post #1 of this thread.

The current version contains additional information about:
- binning,
- readout mode,
- FITS format,
- temperature,
- light leaks,
- resources for PixInsight,
- applicability of LDC,
- requirement of loading calibrated light frames into SFS,
- recommendation of applying Drizzle Integration and
- a list of abbreviations.

Bernd
 
The following changes were made today in my guide "Preprocessing of Raw Image Data with PixInsight":
  • Flat-darks are not (and never have been) a distinct type of calibration frames. There are only three types of calibration frames: bias, dark, and flat frames. The term "flat-dark" is replaced by "dark frame with exposure time matching the flat frames".
  • The recommended way to calibrate flat frames is modified.
  • The summary of prerequisites for recording calibration frames that are compatible with the light frames is moved to the appropriate section, B3 Settings and Conditions for Image Acquisition of Calibration Frames.
  • The recommendation to not apply dark frame optimization when a camera exhibiting "amplifier glow" is used is expressed more clearly.
  • New reference documentations about processes that are mentioned in this guide (FastIntegration and LocalNormalization) are listed.
Bernd
 
Back
Top