Calibration and align batch operations

[Juan Conejero]

They must asking for bias overscan too?

Not so far, but they may ask for it eventually. That will depend essentially on whether they start using PI intensively for preprocessing tasks or not.

I'm not sure if overscan support is actually a necessity for the BatchPreprocessing script, but since this doesn't add too much complexity, I'll implement it in the next version.

[mmirot]

Thanks Juan

You rock  8)

[Harry page]

Right Have found this bug , if you uncheck the " export calibration files " then saving the settings with new instance does not work and forgets the selected files

This is precisely the purpose of the "Export calibration files" option. When enabled (default state), newly created instances will contain all file paths selected in all frame groups. When this option is disabled, new instances will contain no file paths but only calibration, integration, registration and deBayering parameters.

Maybe the tool tip information is not clear enough for this check box?

Hi

Ok I get it now , not what I thought it meant so A better description would be better

Regards
Harry

[NKV]

I'm not sure if overscan support is actually a necessity for the BatchPreprocessing script

Juan, my camera 160803 FLI ProLine required overscan for proper calibration. I can't use it without overscan.
Of course, I can apply overscan to all images in ImageCalibration module. And after that use BPP script.
So i will happy to see overscan in BPP. For me enough one overscan region.

[Juan Conejero]

Let's start the week with a smile Version 1.10 has been just released as an update; the source code is attached to this post, as usual.

Version 1.10 implements full support for calibration overscan with four independent overscan regions, just as the ImageCalibration tool. Please check that it is working well.

Enjoy!

[NKV]

Let's start the week with a smile

[atheelen]

Let's start the week with a smile Version 1.10 has been just released as an update; the source code is attached to this post, as usual.

Version 1.10 implements full support for calibration overscan with four independent overscan regions, just as the ImageCalibration tool. Please check that it is working well.

Enjoy!

Juan, looks good. The issues I was having with loading images via FITS header info and binning numbering problem looks to be fully resolved with v1.10 

[cs_pixinsight]

Is there a reason debayering and noise estimation is a sequential operation in the current version?  The total processing time would decrease quite a bit if it could occur in parallel like CosmeticCorrection is currently doing.  Only using 1 of 8 cores seems like a waste of resources 

Craig

Juan, any chance more multi-threading can be added to these areas?

Craig

Hi Craig,

All processes currently executed by the BatchPreprocessing script are parallelized. However, not all of them are parallelized in the same way, neither all of them are using multithreading with the same efficiency for large batch tasks.

The ImageCalibration and CosmeticCorrection tasks use high-level parallelization. Basically, to process N files with P worker threads, each worker thread runs independently for a sublist of N/P files. There is an additional coordination thread for worker thread supervision and file reading/writing tasks. For large file lists, this is usually the most efficient implementation. ImageIntegration also uses a very efficient multithreading scheme.

Other processes use low-level parallelization, which in some cases is not appropriate for the BPP script. For example, some parts of the StarAlignment task are parallelized, such as the star matching and RANSAC routines. This is very efficient to process a few images (e.g. for mosaic construction, or to align less images than the number of processors available), but rather inefficient to align large sets of disk files. A future version of the StarAlignment process will use a high-level parallelization scheme, which will improve the performance of the BPP script considerably. This is near the top of the to-do list.

The whole PixInsight platform will be improved significantly for multithreaded execution in future versions. There is still a lot of work to do in this regard.

Juan, thank you for the information regarding this request.  In watching the CPU resources while running the BPP script, it's apparent that just about everything is running in parallel.  It's really the debayering (which is fairly fast even using 1 core) and the noise estimation (very slow) which slows the process down significantly.  When you get around to improving the BPP script in this regard, please keep in mind the script will run several times faster if the noise estimation utilizes the high-level parallization similar to the CosmeticCorrection module.

Thanks again,
Craig

[mcbbcn]

Hi,

I'm quite new to Pixinsight, but I'm sure I'm doing something wrong when I try to run the script, I get this error:

C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing-helper.js

run --execute-mode=auto "C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing.js"

Processing script file: C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing.js
*** Error [000]: C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing-GUI.js, line 811: Error: ProcessInstance.iconsByProcessId(): unknown process class: 'CosmeticCorrection'

I don't have any images or anything open, I just tried to run it...do I need to have any images loaded or select anything?

Thank you,

Miquel Casas
Portland, OR

[georg.viehoever]

You Need to use the ladest Pi Version with all Updates, which should include an install of the cosmetic correction Module.

[mcbbcn]

Hi Georg,

Thanks for the quick reply.  I "think" I have all the updates, when I go to "Resources->Updates->Check for updates", I get a message saying that I have all the updates.

I'm running Windows 7 Ultimate 64 bit edition and my pixinsight version is 01.07.06.0793 Starbuck (eng x86_64).

Do you have any thoughts if I have do anything else other than the standard way to update PI?

Thanks for your help,

M.

[mcbbcn]

...also, I don't know if this has anything to do with it, but I noticed that I script called "CosmeticCorrection v1.6.1" and it seems to work just fine...

[mcbbcn]

 
I found a solution:

I edited the file updates.xri (Pixinsight folder) with PI Script editor, and I deleted the last two packages (i.e. batchreprocessing & cosmeticcorrection), save the file, and I checked for the updates which force PI to re-download & re-installed both packages.  Now it works!

Cheers,

M.

[mcbbcn]

...and yet, another question.  Using this script, is there a way to substract the master bias from the master dark (i.e. creating a thermal)?  I think you accomplish this during calibration by selecting the calibrate checkbox, but I did not see this option in the script...

Thanks for your help,

M.

[vicent_peris]

Hi,

I'm getting this error just after the bias integration.


Regards,
Vicent.