Author Topic: Calibration and align batch operations  (Read 135712 times)

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7089
    • View Profile
    • http://pixinsight.com/
Re: Calibration and align batch operations
« Reply #405 on: 2012 May 12 03:21:13 »
Quote
They must asking for bias overscan too?

Not so far, but they may ask for it eventually. That will depend essentially on whether they start using PI intensively for preprocessing tasks or not.

I'm not sure if overscan support is actually a necessity for the BatchPreprocessing script, but since this doesn't add too much complexity, I'll implement it in the next version.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
    • View Profile
Re: Calibration and align batch operations
« Reply #406 on: 2012 May 12 06:26:48 »

Thanks Juan

You rock  8)



Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1457
    • View Profile
    • http://www.harrysastroshed.com
Re: Calibration and align batch operations
« Reply #407 on: 2012 May 12 11:31:31 »
Right Have found this bug , if you uncheck the " export calibration files " then saving the settings with new instance does not work and forgets the selected files

This is precisely the purpose of the "Export calibration files" option. When enabled (default state), newly created instances will contain all file paths selected in all frame groups. When this option is disabled, new instances will contain no file paths but only calibration, integration, registration and deBayering parameters.

Maybe the tool tip information is not clear enough for this check box?

Hi

Ok I get it now , not what I thought it meant so A better description would be better

Regards
Harry
Harry Page

Offline NKV

  • PTeam Member
  • PixInsight Guru
  • ****
  • Posts: 677
    • View Profile
Re: Calibration and align batch operations
« Reply #408 on: 2012 May 12 18:53:43 »
I'm not sure if overscan support is actually a necessity for the BatchPreprocessing script
Juan, my camera 160803 FLI ProLine required overscan for proper calibration. I can't use it without overscan.
Of course, I can apply overscan to all images in ImageCalibration module. And after that use BPP script.
So i will happy to see overscan in BPP. For me enough one overscan region.

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7089
    • View Profile
    • http://pixinsight.com/
Re: Calibration and align batch operations
« Reply #409 on: 2012 May 13 16:44:31 »
Let's start the week with a smile :) Version 1.10 has been just released as an update; the source code is attached to this post, as usual.

Version 1.10 implements full support for calibration overscan with four independent overscan regions, just as the ImageCalibration tool. Please check that it is working well.

Enjoy!
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline NKV

  • PTeam Member
  • PixInsight Guru
  • ****
  • Posts: 677
    • View Profile
Re: Calibration and align batch operations
« Reply #410 on: 2012 May 13 17:45:37 »
Let's start the week with a smile :)
:)

Offline atheelen

  • Newcomer
  • Posts: 7
  • Astronomy is looking up
    • View Profile
Re: Calibration and align batch operations
« Reply #411 on: 2012 May 14 04:36:17 »
Let's start the week with a smile :) Version 1.10 has been just released as an update; the source code is attached to this post, as usual.

Version 1.10 implements full support for calibration overscan with four independent overscan regions, just as the ImageCalibration tool. Please check that it is working well.

Enjoy!

Juan, looks good. The issues I was having with loading images via FITS header info and binning numbering problem looks to be fully resolved with v1.10  ;D
regards,
Andrew

Offline cs_pixinsight

  • PixInsight Addict
  • ***
  • Posts: 156
    • View Profile
Re: Calibration and align batch operations
« Reply #412 on: 2012 May 14 11:16:23 »
Is there a reason debayering and noise estimation is a sequential operation in the current version?  The total processing time would decrease quite a bit if it could occur in parallel like CosmeticCorrection is currently doing.  Only using 1 of 8 cores seems like a waste of resources  ;)

Craig

Juan, any chance more multi-threading can be added to these areas?

Craig

Hi Craig,

All processes currently executed by the BatchPreprocessing script are parallelized. However, not all of them are parallelized in the same way, neither all of them are using multithreading with the same efficiency for large batch tasks.

The ImageCalibration and CosmeticCorrection tasks use high-level parallelization. Basically, to process N files with P worker threads, each worker thread runs independently for a sublist of N/P files. There is an additional coordination thread for worker thread supervision and file reading/writing tasks. For large file lists, this is usually the most efficient implementation. ImageIntegration also uses a very efficient multithreading scheme.

Other processes use low-level parallelization, which in some cases is not appropriate for the BPP script. For example, some parts of the StarAlignment task are parallelized, such as the star matching and RANSAC routines. This is very efficient to process a few images (e.g. for mosaic construction, or to align less images than the number of processors available), but rather inefficient to align large sets of disk files. A future version of the StarAlignment process will use a high-level parallelization scheme, which will improve the performance of the BPP script considerably. This is near the top of the to-do list.

The whole PixInsight platform will be improved significantly for multithreaded execution in future versions. There is still a lot of work to do in this regard.

Juan, thank you for the information regarding this request.  In watching the CPU resources while running the BPP script, it's apparent that just about everything is running in parallel.  It's really the debayering (which is fairly fast even using 1 core) and the noise estimation (very slow) which slows the process down significantly.  When you get around to improving the BPP script in this regard, please keep in mind the script will run several times faster if the noise estimation utilizes the high-level parallization similar to the CosmeticCorrection module.

Thanks again,
Craig

Offline mcbbcn

  • PixInsight Enthusiast
  • **
  • Posts: 78
    • View Profile
Re: Calibration and align batch operations
« Reply #413 on: 2012 May 25 15:44:05 »
Hi,

I'm quite new to Pixinsight, but I'm sure I'm doing something wrong when I try to run the script, I get this error:

C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing-helper.js

run --execute-mode=auto "C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing.js"

Processing script file: C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing.js
*** Error [000]: C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing-GUI.js, line 811: Error: ProcessInstance.iconsByProcessId(): unknown process class: 'CosmeticCorrection'

I don't have any images or anything open, I just tried to run it...do I need to have any images loaded or select anything?

Thank you,

Miquel Casas
Portland, OR

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
    • View Profile
Re: Calibration and align batch operations
« Reply #414 on: 2012 May 26 06:29:17 »
You Need to use the ladest Pi Version with all Updates, which should include an install of the cosmetic correction Module.
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline mcbbcn

  • PixInsight Enthusiast
  • **
  • Posts: 78
    • View Profile
Re: Calibration and align batch operations
« Reply #415 on: 2012 May 26 14:28:19 »
Hi Georg,

Thanks for the quick reply.  I "think" I have all the updates, when I go to "Resources->Updates->Check for updates", I get a message saying that I have all the updates.

I'm running Windows 7 Ultimate 64 bit edition and my pixinsight version is 01.07.06.0793 Starbuck (eng x86_64).

Do you have any thoughts if I have do anything else other than the standard way to update PI?

Thanks for your help,

M.

Offline mcbbcn

  • PixInsight Enthusiast
  • **
  • Posts: 78
    • View Profile
Re: Calibration and align batch operations
« Reply #416 on: 2012 May 26 14:31:09 »
...also, I don't know if this has anything to do with it, but I noticed that I script called "CosmeticCorrection v1.6.1" and it seems to work just fine...

Offline mcbbcn

  • PixInsight Enthusiast
  • **
  • Posts: 78
    • View Profile
Re: Calibration and align batch operations
« Reply #417 on: 2012 May 26 14:53:21 »
 :D
I found a solution:

I edited the file updates.xri (Pixinsight folder) with PI Script editor, and I deleted the last two packages (i.e. batchreprocessing & cosmeticcorrection), save the file, and I checked for the updates which force PI to re-download & re-installed both packages.  Now it works!

Cheers,

M.

Offline mcbbcn

  • PixInsight Enthusiast
  • **
  • Posts: 78
    • View Profile
Re: Calibration and align batch operations
« Reply #418 on: 2012 May 26 17:27:16 »
...and yet, another question.  Using this script, is there a way to substract the master bias from the master dark (i.e. creating a thermal)?  I think you accomplish this during calibration by selecting the calibrate checkbox, but I did not see this option in the script...

Thanks for your help,

M.

Offline vicent_peris

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 988
    • View Profile
    • http://www.astrofoto.es/
Re: Calibration and align batch operations
« Reply #419 on: 2012 June 09 01:42:47 »
Hi,

I'm getting this error just after the bias integration.


Regards,
Vicent.