Integrating multiple sessions of images

[BobinBend]

Sorry if this is a simple question but I've taken several nights worth of M42 and would like to integrate them to produce a more refined image. These multiple sessions have different darks (change in temps) and therefore cannot be integrated using BPP in one step. What is the best way to integrate multiple nights of images? Should each nights images be separately integrated and then reintegrate all the integrated images?
Thanks for your help,
Bob

[STEVE333]

Hi Bob - I don't use BPP so I can't help with that.

However, I can tell you the general approach for combining multiple nights data (or even Lights from a single night that have widely varying temperatures). When I say the temperature of a Light I mean the temperature of the camera when the Light was captured. this temperature should be saved in the header information for each image.

First separate kthe Lights into groups where each group contains Lights that are all within 5F of each other. For example, say I had 26 Light frames from the two nights with the following temperatures:
2x46F, 3x48F, 4x50F, 4x52F, 5x54F, 5x55F, 3x57F for a total of 26 images.
I would group the 46F, 48F and 50F images into one group (9 images) and the 52F, 54F, 55F and 57F images into a second group (17 images).

Next create a Master Dark from all dark frames with temperatures of 46-50F for the first group of Lights. Similarly create a second Master Dark from all dark frames with temperatures of 52-57F for the second group of Lights.

Next, calibrate each group of Lights individually with the corresponding Master Dark frame. When this task is finished you have all of your Light frames calibrated.

Next the Calibrated Lights are DeBayered.

Next the DeBayered Lights are Registered (Aligned).

Finally the Registered Lights are Integrated.

Hopefully there is a way to do this within the BPP script.

Hope this makes sense.

Steve

[RickS]

It's not a good idea to use BPP for the integration anyway.  Do multiple BPP runs, one for each night (or temp range or whatever you prefer) but with the same reference image for all of them.  Then combine the calibrated and registered results of all the nights in one integration.  It will take some tweaking of the rejection parameters to get best SNR (reduce rejection to the minimum needed.)  This optimisation of the SNR is why we don't rely on BPP to do integration.

Cheers,
Rick.

[BobinBend]

Next create a Master Dark from all dark frames with temperatures of 46-50F for the first group of Lights. Similarly create a second Master Dark from all dark frames with temperatures of 52-57F for the second group of Lights.

Next, calibrate each group of Lights individually with the corresponding Master Dark frame. When this task is finished you have all of your Light frames calibrated.

Next the Calibrated Lights are DeBayered.

Next the DeBayered Lights are Registered (Aligned).

Finally the Registered Lights are Integrated.

Hopefully there is a way to do this within the BPP script.

Hope this makes sense.

Steve,
Thanks for your workflow. I understand your steps but have only used the BPP script so far in PI. Can you let me know what processes/scripts you recommend for each of the steps you mentioned above?
Thanks,
Bob

[STEVE333]

Next create a Master Dark from all dark frames with temperatures of 46-50F for the first group of Lights. Similarly create a second Master Dark from all dark frames with temperatures of 52-57F for the second group of Lights.

Next, calibrate each group of Lights individually with the corresponding Master Dark frame. When this task is finished you have all of your Light frames calibrated.

Next the Calibrated Lights are DeBayered.

Next the DeBayered Lights are Registered (Aligned).

Finally the Registered Lights are Integrated.

Hopefully there is a way to do this within the BPP script.

Hope this makes sense.

Steve,
Thanks for your workflow. I understand your steps but have only used the BPP script so far in PI. Can you let me know what processes/scripts you recommend for each of the steps you mentioned above?
Thanks,
Bob

Hi Bob - I just got up this morning and haven't even had my coffee!

I have a saved set of process icons that I use to do the PreCal. After I am awake I'll send them to you in an attachment along with some information on how to use them. Saved icons have really helped me to progress in PI.

Steve

[STEVE333]

OK - First coffee almost complete!

I'm not sure, but, it might have been RickS who helped me (a lot) to get the manual PreProcessing to work. He always has good answers I know that for sure.

I use a Canon DSLR, not sure which camera type you are using. I don't know if that will make a difference. Just a heads up.

• The link below is to is my file (Precal.xpsm) containing all the process icons I use for PreCal. Save this file to your computer (I save all my icon files in a folder named PI Icons).
  
  https://www.dropbox.com/s/4tlfuoqkgg9fmqh/Precal.xpsm?dl=0
  
  
• Open PixInsight, right click on the workspace, then left-click on Process Icons\Load Process Icons....
• Navigate to where you saved the Precal.xpsm icon file and double click on the file name. This will open up the Precal icons on your workspace. I love this feature! The first attachment shows how the group of icons will look on your workspace.
• On the lower right corner of each icon you will see the letter D. When you click on the D it opens up a text window where I have Directions on how to use that process.
• Double clicking on one of the icons will open up the process associated with that icon making it ready to use.
  
  OK - Take a deep breath.  Icons loaded and ready to use.
  
  
• The icons are arranged top to bottom in the order they should be used. You won't need all the processes for each precal.
• The first icon is used to combine multiple Bias frames into one MasterBias frame. Bias frames are good for a long time, so, you will only need this process about once a year. Load the Bias frames into the process and run it. Save the resulting image as your MasterBias frame.
• The second icon is used to combine multiple dark frames into a MasterDark frame. Load all the dark frames (I combine the dark frames in groups of 5F) then run the process. Save the resulting image as your MasterDark frame.
• The third icon opens up the process to calibrate Flats (see second attachment). Load all the flats into the process.
  a) In the Output Files section navigate to the directory where you will temporarily store the calibrated Flats.
  b) In the Master Bias section navigate to the directory where your MasterBias is stored.
  c) Run the process.
• The fourth icon is used to combine the calibrated flats into a MasterFlat. Save the resulting image as MasterFlat.
• With all the calibration frames ready, the fifth icon is used to Calibrate the Lights.
  a) Load the Lights (again I use groups of 5F).
  b) Navigate to the folder where the calibrated Lights will be temporarily stored.
  c) Navigate to and select the appropriate MasterBias, MasterFlat, and MasterDark frames.
  d) Run the process.
• The sixth icon is used to DeBayer the Calibrated Light images.
  a) Load all the Calibrated Light images.
  b) Make sure that the Bayer/mosaic pattern is appropriate for your camera.
  c) Select the output directory where the DeBayered files will be temporarily stored.
• The seventh icon is used to register (align) the Debayered images to one another.
  a) Load all of the Debayered images.
  b) Choose the "best" image as the reference image, open it on the workspace, select it as the Reference image.
  c) Select the output directory where the Registered files will be temporarily stored.
  d) Run the process.
• The eighth icon is used to combine the Registered images. Load the Registered images and run the process. The resulting image is the final result of Precalibration.

This completes the PreCal proceedure.

Please feel free to contact me if more explanation is needed. This took longer than I expected, but, I probably still forgot to explain something.

Hope this helps.

Steve

[BobinBend]

Steve,
Thanks for your detailed response. I've loaded up your Process Icons file (It was loaded into myt 5th Workspace for some reason) and will give it a try.

One question; In step 13b you mention to select the best subframe for the Reference frame. In the past I've used the SubframeSelector script to identify the frame with the lowest FWHM value for my Reference frame. Can I use the same method or do you use some other tool?

One more question; How are these steps (although more time consuming) better than using BPP? What are the advantages?

Thanks again for all your help. I'll master this program yet!!
Bob

[STEVE333]

Steve,
Thanks for your detailed response. I've loaded up your Process Icons file (It was loaded into myt 5th Workspace for some reason) and will give it a try.

One question; In step 13b you mention to select the best subframe for the Reference frame. In the past I've used the SubframeSelector script to identify the frame with the lowest FWHM value for my Reference frame. Can I use the same method or do you use some other tool?

One more question; How are these steps (although more time consuming) better than using BPP? What are the advantages?

Thanks again for all your help. I'll master this program yet!!
Bob

You are welcome Bob - Others helped me and it was very much appreciated. I'm just happy to be able to pass it along.

For step 13b) your approach to finding the best Reference frame is probably better than mine. Should work perfect.

I don't know if this approach is better/worse than BPP. It does go all the way through to the final step of stacking Integrating the Registered images. Doing this approach does allow you to adjust the noise rejection parameters in this step which may be an advantage. I have had very good success with this approach. It has always completely removed airplane trails (the approach to San Jose Airport runs right over our house) even when the lights of the airplane pass right through the target whether galaxy or nebula.

Hope it works for you.

One last suggestion. I created a folder named CALIBRATION Temp Files to hold all of the temporary files generated during the Precal process. The structure is:

CALIBRATION Temp Files
- CalibratedFlats
- CalibratedLights
- DebayeredLights
- RegisteredLights

The path to these folders remains constant, so, they can be saved in the "Destination" paths in the Precal Icons.
Just remember to empty them each time before running a new Precal, or, you will have to sort out the new and old target files (bother). Works for me. Maybe it will be useful for you too.

In case you aren't familiar with changing the setting inside an icon:
a) Open up the icon and make the changes to the process settings (including file locations).
b) Left click on the diamond at the lower left corner of the process and drag it onto the icon for that process. Answer Yes to any questions.
c) Save the changed icons.

The changed settings are now part of the stored icons.

Steve

[BobinBend]

Steve,

I've setup all my process settings and created a new instance and save them before. But I like your suggestion to create a calibration temp files folder that can be erased and reused for each precal session.
Thanks again for all your help,
Bob

[STEVE333]

You're most welcome Bob.

Happy processing!

Steve