Preprocess Pipeline v0.9.0Beta

[sjbrown]

Hi

When I use the reset button I can not reslect any Item without reloading the script 

Keep up the good work

Harry

Problem found and corrected. Will be in next release (Sunday?).

John

[DarrenS]

Excellent John,

just processed  16x 16meg images with no problems   

One thing I've found is I have to fine tune my pixel rejection settings depending on my data. Normally I integrate the calibrated and registered images then test different settings on a region of interest (much quicker on my old machine).
Once the script is instantiable  (had to look that one up) will it be possible to reload the script with the previous settings - disable the calibration and registration processes and integrate on a region of interest?

Apologies if this is not very clear   
keep up the good work

Darren

 

[mmirot]

Thanks John. This is great news. 

Max Mirot

[sjbrown]

PreProcessPipeline v 0.9.15 is now available in the initial post in this thread,  and also an updated copy of the documentation source materials.
This version repairs two bugs and adds two new features.

Bugs:

• Repaired a bug that allowed files with formats other than .fit to be fed into the integration process. All inputs to the II process are
  now automatically converted to the .fit file format. This does not affect your choice of ouput file format.
• Repaired a bug that prevented operation of the menu buttons following a reset of the treeview contents.

Features:

• Added the ability to control how many parameters of a given process are shown. Two modes are available and selectable for each
  process added to the treeview - 'commonly changed parameters' and 'all parameters'. The default behavior is to show only the most
  commonly changed parameters. This feature is controlled by a new node control button on the bottom of the treeview.
• Added a function to copy the value of a selected parameter to the same process parameters in all other image sets of the same type.
  This version only allows for the copying of one parameter at a time. This feature is also controlled by a new node control button.

Other changes:

• Moved all node control buttons to the bottom of the treeview. (I was running out of room!)
• Added a string to the value field of all process nodes indicating the current parameter show mode.

The updated document source files includes images with .png extensions instead of .PNG so all platforms should be able to load them. I have
included breif descriptions of the new features under the "Node Control buttons" section in the parameters area of the documentation.

I need some input from you as to what should really be included in the 'commonly changed parameters' list. I picked out what I might normally
want to change but I doubt it is an accurate list. 

Hope you enjoy the changes and keep the bug reports/feature ideas coming.
On to the instantiation code.

John

[sjbrown]

Excellent John,

just processed  16x 16meg images with no problems   

One thing I've found is I have to fine tune my pixel rejection settings depending on my data. Normally I integrate the calibrated and registered images then test different settings on a region of interest (much quicker on my old machine).
Once the script is instantiable  (had to look that one up) will it be possible to reload the script with the previous settings - disable the calibration and registration processes and integrate on a region of interest?

Apologies if this is not very clear   
keep up the good work

Darren

Hi Darren,

Glad it worked well for you, that's always nice to hear.

Making the script instantiable will allow ou to do exactly that, and much more.  8)
You can do something like this:

• Run your images through the entire process once. Inspect the integration output to determine what area to experiment with.
• Start the script again and create however many MasterImage sets you need  (depends on how many parameter variations you want to try).
• Add only the integration process to all of the image set nodes.
• Load the registration output files from the first run into the inputs of all your image set nodes' input files.
• Use the new copy feature to copy the ROI values from one image set integration parameters to the rest of them(saves time).
• Make any other parameter changes and run the script.

This should allow you to relatively quickly narrow in on the parameter values you need.

Thanks

John

[sjbrown]

Thanks John. This is great news. 

Max Mirot

Hi Max,

I know you have been waiting a long time for smething like this. It may not be all you want right now, but it is a start!

More is coming.

John

[j.w.]

This is the script I've been waiting for - but I'm not having much luck with it!  I've tried it in both the 32 bit Mac version (it never finished due to an out-of-memory error) and 64 bit Mac version (also didn't finish due to a "line 5536: TypeError: node.parent.child(28) is null).  Is this a Windows only script so far?

John

[sjbrown]

Bug fix release!

Version 0.9.16 is available for download on the initial post of this thread.

This version repairs a bug that was causing a null node error when changing any of the true/false nodes that appear in the
short list of the image calibration process.

Please replace version 0.9.15 with this version.

Sorry for the inconvenience.

Thanks,

John

[sjbrown]

This is the script I've been waiting for - but I'm not having much luck with it!  I've tried it in both the 32 bit Mac version (it never finished due to an out-of-memory error) and 64 bit Mac version (also didn't finish due to a "line 5536: TypeError: node.parent.child(28) is null).  Is this a Windows only script so far?

John

Hi John,

I have found and repaired the bug producing the 'node.parent.child(28) is null' error. That was a serious problem and I thank you for reporting it! 

The script should run on all platforms supported by PI. One problem noted by Juan is with displaying buttons on the Mac OS X Aqua style.

On a 32 bit machine it is easy to overwhelm the system's resources(RAM). Did the script fail during the integration process? If so, try reducing the
'Buffer size' parameter from 16MB to 12 or maybe even 8MB. I need to do this when trying to integrate 100 12mexapixel bias frames.

With the new, bug fixed version, your 64 bit machine should now work. Let me know of any other problems you encounter.

Thanks,

John

[j.w.]

Thanks John - I'll give it a try and let you know.  This is an outstanding addition to PixInsight by the way!

[mmirot]

Thanks John. This is great news. 

Max Mirot

Hi Max,

I know you have been waiting a long time for smething like this. It may not be all you want right now, but it is a start!

More is coming.

John

Actually, this just what I had in mind. Perhaps even better.

[j.w.]

Hi John,

I'm rerunning the new script (v0916), but I've hit a bump in that I can't remove the previously created path for the Master Bias Path in Master Dark 1/Calibrate.  I can redirect to a new path, but I can't remove the path and rely upon the Master Bias created as part of the script.  Any ideas?

Thanks!  John

[sjbrown]

Hi John,

I'm rerunning the new script (v0916), but I've hit a bump in that I can't remove the previously created path for the Master Bias Path in Master Dark 1/Calibrate.  I can redirect to a new path, but I can't remove the path and rely upon the Master Bias created as part of the script.  Any ideas?

Thanks!  John

Hi John,

From your second sentence it appears that you want to use the output of Master Bias 1 for the Master Bias Path in Master Dark 1/Calibrate.
If this is correct, ypu need to add Calibration Links to Master Dark 1.
The treeview should look similar to this:

    Master Bias 1
    Master Dark 1
        Input Files
        Calibration Links setup
            Inherit master bias from:             Master Bias 1
        Processes

You will need to doubleclick on 'Inherit master bias from:' node in order to select the master bias you want to link to.
Once the link is set up, the engine will programatically replace the value of the Master Bias Path parameter with the output of Master Bias 1.

Hope this helps,

John

[j.w.]

Thanks John - I did in fact have the Calibration Link set up for the script generated master bias.  So, I shouldn't worry about the the value in the Master Bias Path then?  That's good to know.

John

[sjbrown]

Thanks John - I did in fact have the Calibration Link set up for the script generated master bias.  So, I shouldn't worry about the the value in the Master Bias Path then?  That's good to know.

John

Correct.  The Master Bias Path parameter will be rewritten immediatlely after the Master Bias 1 integration process engine finishes.

John