Author Topic: Iterating Parameters (a.k.a. Cracking the PI Enigma Machine)  (Read 3563 times)

Offline marekc

  • PixInsight Addict
  • ***
  • Posts: 177
Okay everybody, I've got a bit of a `thinker' for you. I suppose this is mostly directed at the PI authors and gurus, like Juan, Vicent, etc..., but I'd be curious to hear anyone's opinion.

Many of us have commented on the difficulty we commonly have when choosing parameters for PI's processes, such as Deconvolution, MMT denoising, StarMasking, etc... We are often somewhat overwhelmed by the number of things to adjust. I'm always very grateful to read, say, one of Juan's tutorials on noise reduction or deconvolution, in part because he gives us useful settings to try as a starting point.

It's finally starting to dawn on me that the VaryParams script could be pretty useful here. I've known about it, and I've had it saved on my computer for a while, but I've finally started trying to use it to help me figure out some Deconvolution settings for an image. Here's what hit me the other day: It's like starting an imaging run! Setting VaryParams to explore a parameter is kind of like setting my telescope/mount/camera on a run of light frames. I go off and do other things, such as observing with binoculars (in the case of an imaging run) or doing something else on the computer (in the case of VaryParams). Okay, so far so good.

Here's what I'm wondering... for a given process, such as Deconvolution, how much does the ORDER of our explorations matter? For example, in Deconv, the `Global Dark' setting matters a lot. It wasn't until I read Juan's tutorial on Deconvolution (http://pixinsight.com/forum/index.php?topic=2727.msg18512#msg18512) that I found out it should be set to a very low value, such as 0.005. If I'm trying to use VaryParams to explore the parameter space within Deconvolution, I might start my exploration by doing several Deconv runs with different values of Global Dark. Once I find the best value, I might go on to some other parameter.

But here's what nags at me: What if the parameters affect each other so much that we can't take this `sequential exploration' approach? What if, having found a Parameter X that looks best, the optimal Parameter Y (or whatever) would actually necessitate going back to Parameter X and re-tweaking it? In such a case, it seems to me that our exploration of `Pixinsight parameter space' is fairly close to hopeless, without some massively automated AI system doing it for us.

So, my question boils down to this: In most of PIs major modules (Deconv, MMT denoising, etc...), will we get good results if we iterate parameter-by-parameter, sequentially? And more to the point, *what order of parameters should we use*?

I think that if the PI Team could give us a simple list of the `parameter order' for each processing module, these lists would be some very powerful information for PI users. We'd know what order to use VaryParams, and then processing in PI becomes like shooting a celestial object with a CCD camera and autoguider... just set it up and let it go. Forget that manual guiding with the illuminated cross-hairs, go take a nap or look through a visual scope or do something that you actually enjoy. In the case of PI, forget staring at PI all day, just check on it from time to time to see if you need to set VaryParams on some new task, and go live your life. True, processing wouldn't *really* become a brainless, automated task. A reasonably deep understanding of the methodology of imaging and the workings of PI will still be necessary, as well as an aesthetic sense. But this just seems to be a way to harness the power of a computer to do our iteration for us, instead of staring dumbly at the PI screen, as I do.

So, to make this work... what is the preferred order of `parameter exploration' for each of PIs modules? Hmm? If we have this, we can (sort of) all become PI wizards like the gurus we admire so much! (There's some tongue-in-cheek exaggeration there, of course.)

Wait a second! Now I know what image processing (particularly with PI) is like! This is a problem in cryptanalysis! I can finally put my finger on what processing feels like. The correct set of parameters to apply in PI, which would yield the best possible image (according to my aesthetic sense), is the plaintext. The image data are the ciphertext. PI is the Engima Machine. It has `rotors', `reflectors', plugboard settings, a day key, and the like. VaryParams is like one of Turing's `bombes', or his Colossus. It has the power to help me explore parameter space and make progress on the decryption, but it isn't strong enough to do the whole process by brute force. Ideally, different parts of the decryption process could be isolated from each other, so it could be done in a piecewise way. To the extent that the different parts of the decryption *can't* be isolated from each other, the task becomes more hopeless.

My dream of the `parameter exploration order lists' would be like telling Marian Rejewski or Alan Turing what parts of the decryption process could be isolated from which other parts, thus allowing their machines to do as much of the work as possible.

- Marek

- Marek Cichanski

Offline Philip de Louraille

  • PixInsight Addict
  • ***
  • Posts: 289
Re: Iterating Parameters (a.k.a. Cracking the PI Enigma Machine)
« Reply #1 on: 2012 September 28 05:52:04 »
Where did you get the VaryParam script from? Sounds like something I'd like to play with. Thanks
Philip de Louraille

Offline marekc

  • PixInsight Addict
  • ***
  • Posts: 177
Re: Iterating Parameters (a.k.a. Cracking the PI Enigma Machine)
« Reply #2 on: 2012 September 28 06:55:08 »
Hi Philip,

As far as I can recall, I got it from this thread:

http://pixinsight.com/forum/index.php?topic=4108.msg28562#msg28562

- Marek

Offline dayers

  • PixInsight Addict
  • ***
  • Posts: 201
    • The Orlop
Re: Iterating Parameters (a.k.a. Cracking the PI Enigma Machine)
« Reply #3 on: 2012 September 28 07:14:20 »
Marek, I can readily identify with your healthy frustration with finding the "right" combination of parameters for a given tool.  I  like your "Enigma Machine" analogy!

If I may put on my philosophical hat for a moment, I wonder if most of the fun is in the challenge of seeking a "logical" path to a satisfactory set of parameters for a given image and tool? In my engineering past life, I can remember no problem solved that resulted in such satisfaction. We are chasing the Holy Grail, here. The chase is important, but the goal may not be possible, at least for me, given my particular set of mental and psychic limitations. But that doesn't keep me from having a lot of fun with the chase.

I like your thinking on this common dilemma. Maybe it will smoke out some new light on the subject.

Dave
Dave Ayers
  Stellarvue 80 mm refractor on CG-5 mount, Orion 50mm guide scope. Imaging camera SBIG STF-8300M, guide camera ASI120mm. PHD Guiding. Sequence Generator Pro, PixInsight.

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Re: Iterating Parameters (a.k.a. Cracking the PI Enigma Machine)
« Reply #4 on: 2012 September 28 07:22:01 »
Hi Marek,

your idea to compare image processing with PI to cryptoanalysis is interesting! I loved your post a lot!

I think we are actually not in such a bad shape as cryptoanalysts. Usually, the effect of changing parameters (keys) is not as (seemingly) chaotic in the case of cryptographic encoding/decoding: Small changes (usually) result in small changes in the final image. This does not mean that finding good parameter sets is easy: parameters interact with each other (which can result in a parameter being important in one context, but totally unimportant in another, so it is not so easy to say which parameter is really "important"), effects by no means are linear or monotone (e.g. higher values always improves quality), or continuous (i.e. no jumps in quality). Also, how to measure the quality of a result image is difficult to say (and also has aestethic components).

To me, the problem is an optimization problem, and could be approached by common optimization procedures used in engineering: optimization algorithms, sensitivity analysis, predefined blocks of processes with known good properties,... Having some experience in this area, I doubt that this could be made automatic for most image processing algorithms (even defining a quality function would be a very hard problem), but certainly it can be supported by tools.

The VaryParameters script is a first step in this direction, and is actually the first tool that I would have recommended to build. I dont know if anyone has the time (and drive) to work on additional tools. If there is someone, I can provide some pointers to literature.

Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline bitli

  • PTeam Member
  • PixInsight Guru
  • ****
  • Posts: 513
Re: Iterating Parameters (a.k.a. Cracking the PI Enigma Machine)
« Reply #5 on: 2012 September 29 01:59:02 »
Hi,
In fact when building VaryParam I first though that I could do something like iterate a parameters until some result is good enough. But as highlighted by Georg, it is pretty difficult to find an evaluation function, except in very simple case.  You also have the problem of combinatorial explosion - you may need a sequence of operations to have an interesting result. Iif you multiple the number of parameters and values and operations possible, you reach astronomical numbers :-)

In general I use VaryParam in two modes: either varying wildly a parameter to understand its effect (you often need to look at extreme values so that the effect is obvious), or fine tuning a parameters when it is almost right.  It is true that some parameters interact, for example you can often increase the intensity of a processing if you also increase the corresponding denoising.  Is the result better or worse? So far I have not found another way than looking at the result and making a decision, also based on how much time I want to spend fine tuning a parameter rather than proceeding to the next step.  Fortunately for most parameters the effect is quite understandable and does not impact too much the other parameters.

There are still cases where it would be very useful to automate either the search of the optimum, or may be the use of alternate processing depending on the result. For example some parameters of star alignement could be changed until we have 'enough' algined stars.  For that, we would need to get some information back from the process.  Unfortunately I did no find a way to get the interesting information from most processes - they are just logged on the console.  This is a pity, as PI is largely object oriented but stops short of returing results in a way useable by other components, at least in general. 

It would be very useful if the processes would emit their (non image) results in a systematic way, for example as a list of properties. They would be logged to the console if there is no other consummer.  Windows PowerShell is a good example on how this can be used to have both the benefit of simple console logging, but also object oriented composition of process if desired.

May be a better way to analyze the results of a process by the calling PJSR will be available in the future. Then we could start driving more intelligent search of parameters.
-- bitli

Offline marekc

  • PixInsight Addict
  • ***
  • Posts: 177
Re: Iterating Parameters (a.k.a. Cracking the PI Enigma Machine)
« Reply #6 on: 2012 September 29 11:31:50 »
Hi bitli,

Thanks very much for writing the VaryParam script! Even if we can't use it to auto-magically do all of our processing, it's very helpful. I'm going to spend some time experimenting with workflows that incorporate it to a significant extent. I certainly agree that the main way to evaluate the results is by visual inspection, especially with things like deconvolution or noise reduction. Fortunately, the script makes it easy to click through the results of a `parameter run', and see which value of the parameter looks best.

- Marek