Non Linear to Linear

[Radar]

Hi,

Can a non-linear monochrome master frame be placed back into a linear state after stretching, closing and reopening please?

Thanks

Ray

[Niall Saunders]

Hi Ray,

I would be extremely doubtful. Conversion to a non-linear state (using the Histogram, or Curves porcesses for example) is very much a 'one-way' process. In all but the very simplest of cases, you can think of the process as a 'many-to-one' conversion.

For example, take an image whose darkest pixel ADU values are non-zero. You might then choose to 'clip' the image by setting the Shadows Clip value to the faintest pixel value in the image - and that would be a perfectly acceptable Histogram Transform. You now have an image whose darkest pixel values are 'zero'.

However - now you propose to 'invert' the Histogram Transform - however, you have no access to the Process used to create the second image. Obviously, all that is needed is to add a fixed value to every pixel value, throughout the entire image. Get that added value (which has to be a 'constant') right, and you will recreate the original image perfectly.

But, this is where the 'many-to-one' analogy comes in - now, in the inverse process, you are dealing with a 'one-to-many' problem. You will not be able to determine, exactly, what the 'constant' has to be to reduce the solution to the pwoblem to a 'one-to-one' scenario.

And, the above description seriously trivialises the whole concept. There is far more than just the one constant, and the mathematical processes become alarmingly complex horrifically quickly.

As always, this analogy is the one that I use for myself, and I would like anyone out there to show me where I have failed to understand the issue myself 

[NGC7789]

For me the best analogy is asking if I can unbake a cake and get back the eggs, milk, flour and sugar.

[Radar]

Thanks guys. So in a nutshell, if the data has been clipped, then obviously it cannot be brought back, but if unclipped, then possibly?

Ray

[aworonow]

Sorry to repeat, but if you do not know the exact steps and values that went into the linear-nonlinear transformation, than it cannot be undone--regardless of saturation issues. Even knowing the steps and values, coding the inverse problem could be exceptionally difficult and subject to instabilities. That's just the nature of the problem. For some images, one might be able to set conditions that must be met for an acceptable inversion, but, again, that could be a programming nightmare, and I am not aware of anyone who has broached that daunting task.

Alex

[Niall Saunders]

Hi again Ray,

That is why you need to get used to saving copies of applied processes - as Process Icons (with an 'xpsm' extension). You can aven save a series of applied processes as a Process Container.

Now, if you always have the steps saved, and you always make sure that your 'Master' image is protected from any changes, then you will always be able to repeat your processing efforts - either from beginning to end, or from beginning to some 'mid-way' point, where you feel that you could try a different approach.

This is always my 'bare minimum' - I keep all of the Master Calibration frames (and the Processes used during the calibration phase), and all of the Master Lights. Then I keep all of the Processes used throughout the post-processing phase. And, finally, I keep the output JPG (if the data made such an image worthwhile).

You will eventually develop a folder structure that suits your methods of working, and will use it to organise the images and processes that you encounter and create during your efforts. Sure, I could show you how I structure my working directory - but it may not be anything like the one that you would rather use - there are no rules here 

[msmythers]

I think the easiest answer is if the stretch was done with a math equation or series of equations and no data was lost due to those equations actions then you could recover the linear data. Any other method of stretching where the actual math is unknown then no you cannot get back to the absolute original data. It's all about tractability of the data flow.


Mike

[Niall Saunders]

Hi Mike,

But, sticking to a Math analogy, if you were to calculate the first derivative of a simple function such as :

y = m.x + c     (the equation of a straight line, with gradient 'm', which passes through point (0, 'c')

you would end up with this equation :

y' = m

However, you cannot 'reverse' the process, because you can only 'integrate' the second equation to give :

y = m.x + d

This is because there are an infinite number of lines whose first derivative would be of the form y' = m

This type of behaviour is at the heart of why some processes cannot be 'mirrored' or 'reversed'.

[Stu]

Perhaps I'm not understanding, but if you save it as a project, all your swap files will be saved.  So all you have to do to revert back to linear is to hit "undo" a bunch of times. 

What I do sometimes is save my history explorer as a process container, clone my final image to save it, and then hit "undo" all the way back to where I want to restart.  Then, I can open my process container and look at the settings of whatever process I want to fiddle with (say MLT), and then try a different setting. 

[Niall Saunders]

Hi Stu,

The way I understood the problem was, given two images, the second one of which was created my a single application of the Histogram Process, and then given that the first image is "no longer available", then "is it possible to recreate that 'original' image with only access to the second image and, possibly, the MTF information?".

There is no provision for relying on the Undo facility, the History Explorer, any saved Processes, etc. All you have is the final image (and the MTF, not thatthis will help in any way whatsoever).

Personally, I do not believe that the MTF (Histogram) Process can be considered as 'reversible'.

[Stu]

I guess I misunderstood what you meant by "reversible".  You can't recover the lost data, I agree.  I was just talking about the internal swap files.  Sorry about that. 

[Niall Saunders]

Don't worry about apologies, Stu. The original issue was/is a complex one, and it is one that deserves to be discussed.

Apart from anything else, it highlights the need for the very action that you (and I) take to avoid the issue in the first place (always saving Process data and interim images). And it is one of the strongest arguments in favour of using PixInsigt in the first place - it's inate ability to completely, and precisely, replicate a whole series of procedural steps such that it is always possible to recreate a specific  final image given the original data set and the process.

It doesn;t take too many 'mistakes' before a user realises how important that 'saving' exercise actually is 

[Stu]

The disadvantage of doing it our way, of course is that each project has about a million different masks, process icons, masters, and false starts.  I guess this is why we have external hard drives...

[Niall Saunders]

Yes Stu - you are right.

Fortunately, nowadays storage is 'cheap'.

And, 'our' process does requireconstant attention to detail - to ensure that every single support image is also stored, along with the method we used to create it as well.

I still believe that there is room for improvement in the way PixInsight stores individual PRocess icons (including ProcessContainer icons). I find tthat I am having to constantly [Ctrl-A], [Ctrl-C] and [Ctrl-V] the name of a process icon on the workspace into the Save Icon dialogue box. And I also wish there was a better way to control where a reloaded icon, or group of icons, ends up on the workspace (just plastering them all over any existing icons is 'messy' at best, and fraught with 'danger' at worst).

I tend to number all of my processes in the order that I create and use them - so that I can re-run things in the same order if required. This isn't made easy because of the strict naming conventions required, so I end up prefixing every Process with the letter 'p', followed by a 2-digit number (to indicate the point in the sequence), followed by the ubiquitous '_' and a whole bunch of 'camel-case' to try and explain the actual process. Interim support images get the same treatment (when they are 'iconised'), except that they are prefixed with an 'i' character.

And, somehow, I have had to create a wrap-around directory structure that keeps everything collated in some meaningful fashion (leaving me to just hope that the horrendously expensive backup software package that I have invested in is actually going to run for more than three days before disappearing up its own back-door.

Sorry - what was the original question . . . .