Preprocess Pipeline v0.9.0Beta

[georg.viehoever]

Hi,

got this error when the script finished creating the master bias:

Code: [Select]

Writing file:
E:/Benutzer/Oeffentlich/Pictures/2011_03_05_orion_wide/PI/masterbias.fts.fit
Writing FITS image: 32-bit floating point, 1 channel(s), 3908x2602 pixels: 100%
*** Error [000]: C:/PCL_Georg/scripts/PreprocessPipeline_v0925Beta/PreProcessPipeline.js, line 3250: ImageWindow.saveAs(): null image window

Georg

[sjbrown]

Hi,

got this error when the script finished creating the master bias:

Code: [Select]

Writing file:
E:/Benutzer/Oeffentlich/Pictures/2011_03_05_orion_wide/PI/masterbias.fts.fit
Writing FITS image: 32-bit floating point, 1 channel(s), 3908x2602 pixels: 100%
*** Error [000]: C:/PCL_Georg/scripts/PreprocessPipeline_v0925Beta/PreProcessPipeline.js, line 3250: ImageWindow.saveAs(): null image window

Georg

Hi Georg,

This is strange. It says it wrote the file and then says the image window is empty??

Did it in fact save the file? I haven't been able to reproduce the error(using the same output name as you did).

John

[georg.viehoever]

Apparently, it error is related to the rejection map. When I defined a filename for this, the error went away.
Georg

[georg.viehoever]

John,

Some random observations when playing with the script
- with a newly started script, the new instance triangle does not do anything. Actually, during my experiments I never was able to produce a new instance (I tried with the CanonBanding script, this one works). This takes a while, put produces no result (not even an error message).
- It would be nice if the script came with a preconfigured pipeline for calibration as described by Vicent. Currently, there are too many settings that just can be configured the wrong way.
- The different processes should have defaults as recommended by Vicent.
- It would be good to be able to define a directory for temporary files (such as the files transformed from .cr2 to .fit), and make this the default directory for all intermediate files.
- I wonder if it would be better to make the .cr2->fits step explicit as one process step!?!
- Would it be possible to implement a makefile-like mechanism that automatically (re-)runs only the necessary steps?
- I think a Cleanup process would be useful, such that intermediate files can be removed with just one command.

I know I am asking for a lot. For the moment, I suggest to make bugfixing and usability your priority. There are still too many people horrified by the complexity (and the many error sources) of doing image calibration in PI (and this does include me  ).

Georg
 

[mmirot]

George,

 Drag and drop the script from the process explore list. I will create iconized process. You can then make child processes.
Have not tried to go beyond this so far.

Max

[mmirot]

I agree with Georg.
The script is a masterpiece. Nevertheless, it has a lot of moving parts so is naturally tricky to use.
 Preset and other refinements will go a long way in help others adopt the use of the script.  It always best to make it intuitive as possible.

I made a few suggestions earlier. Please read the old post.

One suggestion is being able to specify the reference frame for both the integration and registration at point.
Perhaps an additional button at the same point in the script.

Max

[Juan Conejero]

The script is a masterpiece.

Absolutely. As soon as I can have some free time, I'll pay the due attention to this task and will review the script in depth. I hope to be able to contribute to John's wonderful work, and it would be very nice if we can have a first stable version during March. I hope so.

Just be patient with me because I am involved in a new project that you'll know very soon. PI is growing!

[RBA]

PI is growing!

What, you having Little Pixies?

[sjbrown]

PreprocessPipeline v 0.9.30 Beta Bugfix release has been attatched to the first post in this thread.

This release fixes two critical bugs, one that was causing the 'cloning' of integration output images when being saved to disk and one that was not handling the occurance of a 'null image' causing the script to abort.

Also included is code to initialize the parameter storage variables to the values recommended by Vicent in an earlier post. I have included a 'Reset Parameters' button on the bottom of the script so this task may be done at the user's discretion (The button is guarded with a message box to prevent inadvertant resetting).

From the script header:

• Added look-ahead code to Image Integration engine to anticipate serialization of default output image IDs.
  
• Added error handling code for null image errors in Image Integration engine.
  
• Added current parameter defaulsts for ImageCalibration and ImageIntegration processes. Added a 'Reset Defaults' button to load the defaults into the saved parameters in the Settings file.
  
• Fixed a focus issue when closing out of OpenFile or GetDirectory common dialogs. Added three calls to the activateWindow() method of the Control object.
  

Hopefuly this version will work better for everyone.

John

[sjbrown]

One funny bug.

I have set up a pipeline in which I calibrate, register and integrate some images of 5 different objects. For each object, different skyflats are used, and each object has 10 individual frames.

All is going well... until I see the integrated images. The first three images (the script is running right now) are "int_007_G.fit", "int_008_G.fit" and "int_010_R.fit". I load the three images in another instance of PixInsight, and I see that the three images are identical! I mean that "int_008_G.fit" and "int_010_R.fit" are clones of "int_007_G.fit".


Regards,
Vicent.

Vicent,

I have corrected this problem in version 0.9.30. Please try it again and let me know if it is completely gone.

Thanks,

John

[sjbrown]

Hi,

got this error when the script finished creating the master bias:

Code: [Select]

Writing file:
E:/Benutzer/Oeffentlich/Pictures/2011_03_05_orion_wide/PI/masterbias.fts.fit
Writing FITS image: 32-bit floating point, 1 channel(s), 3908x2602 pixels: 100%
*** Error [000]: C:/PCL_Georg/scripts/PreprocessPipeline_v0925Beta/PreProcessPipeline.js, line 3250: ImageWindow.saveAs(): null image window

Georg

Hi Georg,

I included error handling for this in the latest release. The script won't prevent the error but it will handle it so it can at least exit cleanly.

Also:

- with a newly started script, the new instance triangle does not do anything. Actually, during my experiments I never was able to produce a new instance (I tried with the CanonBanding script, this one works). This takes a while, put produces no result (not even an error message).

If you could, try this:
Without moving the mouse, click the new instance button and hold. Wait for the button's appearance to change and then drag the button to the workspace.
Let me know 1)if the button's appearance changes and 2) if it does change, are you then able to create an instance.

Thanks,

John

[RobF2]

Thanks again John for the fantastic amount of work put into a fantastic script.
Some thoughts after my first hour of playing with this:

1.  Is it possible to get "tool tips" to display when you wave your mouse over the buttons?  For those people that already have Master frames ready to go for example the "Master" buttons could perhaps tell you you only have to point to those files from your Master Image node settings.

2.  Agree with Max that it would be nice to not have to go in and explicitly set the Reference for Registration.  If I load up a series of input files, I would hope and expect that the first file will default as the reference for registration and integration.  If that was the case I could have a process icon for unbinned data and another for binned and be off to an easy start (assuming I wasn't going to flat correct for every filter which could obviously be easily set up too) by just loading up the Input Files for each new dataset.

3.  Perhaps one for Juan and a bit into the future - is it possible to set up an Image Container that points to a set of images?  It might then be a simple matter to drag that container over to Nikolay's fantastic Blink script, then after I cull my images in the container, drag it on over to the PreProcess Pipeline.  

4.  I'm getting picky, but I suspect many newbie users might easily overlook what the heck a "Preprocess pipeline" is.  I saw this thread come up in the new posts a number of times before I had my "duh!" moment and excitedly started to decipher how to get it going.  Perhaps mention of something like "Automated Image calibration and processing script" might sort out more dodos like myself.  



Thanks again for all the hard work,
Rob

[pfile]

2.  Agree with Max that it would be nice to not have to go in and explicitly set the Reference for Registration.  If I load up a series of input files, I would hope and expect that the first file will default as the reference for registration and integration.  If that was the case I could have a process icon for unbinned data and another for binned and be off to an easy start (assuming I wasn't going to flat correct for every filter which could obviously be easily set up too) by just loading up the Input Files for each new dataset.

well, the thing about this is that undoubtedly you have one image that has the best FHWM, the least trailing, etc. that's the image you want to use as a registration reference, and it's not necessarily the first image in your sequence. of course right now PI can't really automagically evaluate an image for you... i usually use DSS to score my files and then pick the best one as the registration reference in PI.

[sjbrown]

Max, Rob & pfile,

I have a very simple solution that will satisfy all concerns with the registration reference file.
The integration reference already defaults to the first image in the list. The 'Set Reference' button just allows the user to select a different image than the first in the list and does this by moving the selected file to the top of the list. By inserting a short section of conditional code I can set the reg ref image to point to the first file in the list by checking for an empty string in the ref image parameter. If the string is not empty, that indicates the user has selected a prefered ref image and this choice will be used.
So, if you want the code to "select" the reference images for you, leave the reg ref parameter blank and do nothing for the integration ref image.
Selecting an integration ref image and leaving the reg ref image blank will cause both processes to use the same image.

This will be included in the next release. Please note that leaving the reg ref parameter blank in the current version will give you an error message.

Hope this makes sense, not getting much sleep lately.

John

[sjbrown]

.
.
- It would be nice if the script came with a preconfigured pipeline for calibration as described by Vicent. Currently, there are too many settings that just can be configured the wrong way.

I agree. But rather than have this hardcoded I would like to have a way of loading a preconfigured xpsm file from disk, perhaps a 'Load' button that opens up the OpenFile dialog allowing the user to select one of multiple preconfigured calibration setups. The script would then parse through the file, reconstruct the saved pipeline and all you would have to do is add input files. The preconfiled xpsm's could be part of the official PI release and therfore stringently controlled(no bugs, malware, etc...). What do you think?

John