Opinion and Suggestion: What PI Needs . . .

[oldwexi]

Hi Jim!
Thanks for expressing your thoughts which i fully subscribe!

I never had a workflow in astro image processing which was usable for any ther astro image.
Each image needs ist own specific workflow depending on the specific  different characeristics each image has,  and you have to learn and understand what is needed at each stage.
By the way that has nothing to do with PI it is valid for any Astro Image processing SW.
But the big advantage of PI is that for every new situation there is a tool and you dont have to do
software jumping.


Gerald

[mmirot]

My suggestions for STF would be very useful.

I am for changes wound make certain processing steps more fluid. They increase flexibility not limit it.  They add function and logic to some areas but never subtract.
I am not suggesting dummy buttons or wizards.

No reason to defend PI, it is wonderful as is.  However, there is always room for new ideas.  I hope user suggestions are not dismissed too casually.

Also, thanks Rick. I forgot about the Pixel Math Editor.   I was really thinking a long a new lines for a different module that could do some math tasks in real time.

[mmirot]

"A lot of the tools are unnecessarily complex and non interactive. It is like folows: set  a parameter - try - set parameter - try again - repeat 10 times. I would much prefer more live previews and sliders. Compare the noise reduction algorithm in LR with a few sliders where you see immediately what happens to the various noise reduction algorithms in PI. Although the PI ones are probably much "better" than the LR ones, it takes forever to get it right. More sliders and interactive previews please !

- I feel that PI, right now, is a tool for those who want to spend hours and hours optimizing each picture. I would prefer (but perhaps I'm the only one ?) to spend  (much ?) less time and get the image 80% or 90% right. Instead of spending a LOT of time, and getting it 100% right. So I'd like to have more tools that work "almost" with the default settings, and need much less tweaking. I am sure this is possible, but it needs a big investment from the software team, in testing various cases on different images, to come up with good default values, or automatize some parameter optimizations."

Miska.  Some tasks are processor intensive so the only way to make them functional is via iteration on a preview. I think Juan and the PI team like RT previews when they can do it.

I agree with you a few things could be tweaked so they are more intuitive and perhaps a little less confusing to beginners. That's not going make everything a simple task.
You can see that most of users here reject presets and wizards in favor of knowledge.
Whether you use PS or PI these are advance tools. There are easy buttons in many lesser programs.

Max

[ruediger]

I never had a workflow in astro image processing which was usable for any ther astro image.
Each image needs ist own specific workflow depending on the specific  different characeristics each image has,  and you have to learn and understand what is needed at each stage.

This is a description of the current situation. Is it inherent to astrophotography that it has to be like this? I want to have workflows in an algorithmic sense, i.e. that they can be applied to a broad range of input data. On the extreme side, this is fully accomplished e.g. with Noel Carboni's action set for a wellknown program. All these actions perform high-level tasks like "reduce star size" or "make stars more colorful" etc., yet they are all fully parameter-less!

Similar high-level tasks in PI must be assembled from low-level processes which often need a lot of trial&error parameter tweeking. And this is mostly necessary because a typical PI process has no interest to take advantage of the processed data itself and adjust it's parameter accordingly.

Look e.g. at all the different noise reduction processes out there. They all have in common, that a lot of parameters need to be tweaked, but they all don't have interest in important information like my camera's gain, its dark noise, my background sky flux etc. The first "real" noise reduction in my understanding was Mike Schuster's MureDenoise. No parameter tweaking at all!

I gave more examples in my last post.

So I would like to see something between the current situation and a toolset like Noel Carboni's. Robust processes which automatically preset their parameters based on picture intrinsics and also reduction of repetitive tasks.

Rüdiger

[Juan Conejero]

this is fully accomplished e.g. with Noel Carboni's action set for a wellknown program. All these actions perform high-level tasks like "reduce star size" or "make stars more colorful" etc., yet they are all fully parameter-less!

a toolset like Noel Carboni's. Robust processes which automatically preset their parameters based on picture intrinsics and also reduction of repetitive tasks.

If you really want to have this, and if you really think that Noel Carboni's actions for Photoshop are of any value, then PixInsight might not be the right tool for your purposes. Our philosophy of image processing is exactly opposite to the "reduce star size" and "make stars more colorful" kind of things.

[ruediger]

If you really want to have this, and if you really think that Noel Carboni's actions for Photoshop are of any value, then PixInsight might not be the right tool for your purposes. Our philosophy of image processing is exactly opposite to the "reduce star size" and "make stars more colorful" kind of things.

Maybe we don't understand each other at all. Vicent Peris himself developed a "reduce star size" strategy for PI, later made public by RBA. Let's assume someone encapsulates this strategy in a script which only exposes very few parameters to the user. It would be the PI equivalent of a Carboni-action. There are already scripts like this officially integrated in PI. Narrowband integration, dark structure enhance and so on. So they are obviously not against your philophy of image processing. These Action-style scripts could help address typical tasks users are interested in without reinventing the wheel over and over again.

It's not easy to be critical here in the forum, even if meant to be constructive.

Rüdiger

[jkmorse]

Rudiger,

I take your point that you are trying to be constructive, but as an example lets take star size reduction.  Back when I was using PS I had Noel Carboni's  tools loaded up and at first thought they were a godsend.  But soon I stopped using them because his cookie cutter approach rarely worked the way I wanted it to and there was no way to tweak the result without rewriting his code.  But when I want to do star reduction in PI, I open up Morphological Transform, tweak a couple of settings, and get exactly what I want, though I may take a couple of trials to get those settings just right.  Or I may use masked stretch instead of histogram transform for stretching my linear data.  Or I can try a get a unique solution with PixelMath.  The key is I have a number of tools available with the ability to adjust each one as necessary, to get what I want from my data. 

And the same thing goes for sharpening, noise reduction, background neutralization, hue and saturation adjustments and any other possible tweak you want to do to your data.

I agree none of that comes without butting your head against a steep learning curve.  But you spend nights in the cold with expensive equipment to get your data.  For me I want to treat my data after acquisition with the same respect I treat my equipment in acquiring it. 

I hope that helps in understanding where I am coming from in the debate.

Best,

Jim

[msmythers]

Well I might as well throw in my 2 cents on the subject of tools for other software. If I'm not mistaken Carboni's Actions are not from Adobe. They were written and sold by a user of PS. The same thing could be done with PixInsight through it's scripting. As far as the selling point I don't know about that. Any user could write the same type of 'Actions' as Carboni's in a PI script. Javascript is very powerful. If you look at the code in Carboni's you will see how they use set parameters with some using input from the user. Juan is not stopping anyone from writing a script from doing those types of tasks. Whether Juan or others in the PI team wants to write a tool module to do certain types of function is up to them but the team has not restricted the users from writing there own tool modules or scripts. Matter of fact it seems to be encouraged.

Myself I don't program or do scripting, I wish I could. I do use process containers for some repetitive tasks. Mostly I use these 'quick' process containers for inspection to better understand where I need to adjust a parameter or do I need a completely different tool. We as users can share what we create if we want to but more importantly to me is we can openly see how others are writing there scripts or process containers. This allows us the user to tailor someone else work to suit our particular needs. This is encouraged. We learn.

One more thing, I do my processing for pretty pictures. I don't do science anymore, too many years of science for a living. I don't have great equipment. I shoot from a very light polluted location where the night time temps are mostly warm to hot and the seeing is always terrible. Processing is the only thing that saves my images and what I do in PI I could never figure out how to in other software. That's just me.

Regards and I hope every ones images surpass there expectations.


Mike

[ruediger]

I shouldn't have mentioned this "Lord Voldemort like 'you must not say it's name' program from a multinational company" in this forum. Now the discussion is only focussing on this.

My suggestions were:
- make more interconnections between processes. Examples: Drag/Drop of STF to TGVDenoise. Use "Detected Stars" from StarAlignment as Input for DynamicPSF
- more robust default settings of processes by utilizing all information that is possible. Example: TGVDenoise Edge Protection from background's avgdev
- "glueing" of robust single processes to "actions" with a much reduced parameter set. Examples are already available in PixInsight's script section.
- better user experience by adding simple things. Examples: line drawing (for satellite removal), reducing strength of last operation comfortably with a slider.
- reduction of dull repetitive tasks by automisation. Example: Anyone remember the time before Kai Wiechen introduced the BatchPreprocessingScript? I do.

Rüdiger

[mmirot]

I agree that some enhancements would be helpful. These should not be short cuts in the sense of pre set processes or scripts.

Rather it would be nice additions to the tools we have that currently.

For example, I can't see any reason that a STF should be retained after converting to non linear. 
The histogram module should be aware of the STF and automatically deactivate it after application. Little changes like this actually enhance the user experience.


Your idea applying multiple iteration parameters at one go to a process is not a bad one.
This actually would be helpful if you could add a range of settings to a set of previews and chose the best one.
These are productivity ideas that are worth thinking about.

 I hope no one rejects good suggestions because it merely sounds like something that happens in another software package. That's a knee jerk reaction that is counter productive and should be avoided.

I bet I could come up ten recommendations the make processing more efficient without reducing flexibility. Call it PI ergonomics for lack of better term.

Max

[vicent_peris]

Hi,

I think the real subject of this thread is not about PixInsight or any other software package. The real subject is about how we approach astrophotography and about our goal in our photos.

There is a huge misconception in some messages. Asking in PixInsight about fully automated processing techniques that are built like black boxes where the user don't know what's happening inside is just the opposite philosophy of PixInsight itself. Asking here to build such tools is not criticism at all, it's just like going to a vegan restaurant and asking for a steak. Maybe you're going to the wrong restaurant, and there are a lot of BBQ ones all around.

PixInsight users are happy because they have the opportunity to reach new knowledge when they process images. If you get used to use black boxes, at some time you'll be completely stuck with your work because you're really not learning anything. All the people coming to my workshops are just happy because they now know what are they doing. They know why they should my that specific slider. And, more important, they start to learn what to look into the image to determine if the process is doing right. In my intensive workshops, and all the people coming know this, the main subject of the full week is not about using PixInsight, it's not about using the tools. It's about learning to look at the image. That's my approach. And I think that's where astrophotography is.

On the other hand, that concept of "moving the slider to increase the color on the stars" started to be outdated about 15 years ago, when I introduced the concept of the (wrongly named) "star masks". Currently, we don't work separately on the stars to make them up. We work separately on the stars because they have a different meaning in the image, and because they are different from a structurally  view of point. And we don't process separately just stars, in some processes we need to work separately with the small-scale, high-contrast structures because they need a different processing in order to preserve their contained information. I think a good article to read is this one. It just explains my current methodology to delinearize an image.

On the other hand, some workflows are actually much easier to apply in PixInsight than in any of the competing software packages. Two good examples are the LRGB composition and the Hubble's palette.


Best regards,
Vicent.

[ruediger]

Asking in PixInsight about fully automated processing techniques that are built like black boxes where the user don't know what's happening inside is just the opposite philosophy of PixInsight itself.

Vicent, thanks for answering. But... If this thread and especially my contribution to this thread is summarized this way, it really feels like where are speaking two different languages.

When concentrating everything said so far now to this "automated processing" and "black box approach", I may cite you from one of your own processing examples (narrowband integration https://pixinsight.com/tutorials/narrowband/):

"These are the key features of this new algorithm:
[...]
Fully automatable workflow with consistent results"

This is the automation I'm speaking of. I feel I'm an quite experienced PixInsight user and know many tricks, yet I don't want to spend hours in front of my computer for processing data which has been shot unter sub-optimal condition from my garden just to get a pretty picture with no need of scientific interpretation.

Let me try to approach this "what PI needs..." from a total different point of view:

There is a subforum here titled "Wish list ... post them here and discuss them with us". Yesterday I quickly walked through all the pages of this forum and tried to count the number of threads with "0 replies". I found so many, many of them. People (like me) are asking for simple enhancements (some of them feel more like bug reports), but yet there is no dicussion going on and even worse: simple things are left unsolved for years. Example: "why are all FITS headers lost after image registration/integration" or "Please add a rotation parameter to starfield generator". No answers, still no solution (for years now), but the second example only coincidentally obsoleted by Andres' new script implementation.

Imagine how frustrating it must be for a new PixInsight user, who tries to contribute, but doesn't even get a short answer like "scheduled for release x.y" or "wait for new script by ..." or something alike.

Also: I opened two bug reports a week or so ago. No answers so far. But once I place these forbidden words in a post, for sure I get an immediate answer!

I know, that once we paid our license, we don't generate any more income for the PTeam for years, yet we receive all updates for free (and complain in this forum about bugs and want to be heard and steal your time). But this can be solved easily by yourself: Just sell your major new features. I certainly would have paid for DrizzleIntegration (but not for a file explorer, new file format or the windi module BTW).

Rüdiger

[vicent_peris]

Asking in PixInsight about fully automated processing techniques that are built like black boxes where the user don't know what's happening inside is just the opposite philosophy of PixInsight itself.

Vicent, thanks for answering. But... If this thread and especially my contribution to this thread is summarized this way, it really feels like where are speaking two different languages.

When concentrating everything said so far now to this "automated processing" and "black box approach", I may cite you from one of your own processing examples (narrowband integration https://pixinsight.com/tutorials/narrowband/):

"These are the key features of this new algorithm:
[...]
Fully automatable workflow with consistent results"

This is the automation I'm speaking of. I feel I'm an quite experienced PixInsight user and know many tricks, yet I don't want to spend hours in front of my computer for processing data which has been shot under sub-optimal condition from my garden just to get a pretty picture with no need of scientific interpretation.

Well, that's why PixInsight has a developer community. I think you should ask in that subforum.

Right now, many of the tools are already automations that let the user control the key features of the algorithms. Two examples are HDRMT or SCNR. You can apply HDRMT through the multiscale tools and PixelMath (that's the way I usually work with my images, as I don't use HDRMT), but it will take way longer.

On the other hand, once you have the master lights you can have a LRGB composite fully processed in 15 minutes because we designed good tools that let you apply that workflow with very quick steps. Again, if you need any further automation of this workflow to save 5 minutes, then you should ask in the developer forums.

Let me try to approach this "what PI needs..." from a total different point of view:

There is a subforum here titled "Wish list ... post them here and discuss them with us". Yesterday I quickly walked through all the pages of this forum and tried to count the number of threads with "0 replies". I found so many, many of them. People (like me) are asking for simple enhancements (some of them feel more like bug reports), but yet there is no dicussion going on and even worse: simple things are left unsolved for years. Example: "why are all FITS headers lost after image registration/integration" or "Please add a rotation parameter to starfield generator". No answers, still no solution (for years now), but the second example only coincidentally obsoleted by Andres' new script implementation.

Imagine how frustrating it must be for a new PixInsight user, who tries to contribute, but doesn't even get a short answer like "scheduled for release x.y" or "wait for new script by ..." or something alike.

Also: I opened two bug reports a week or so ago. No answers so far. But once I place these forbidden words in a post, for sure I get an immediate answer!

I know, that once we paid our license, we don't generate any more income for the PTeam for years, yet we receive all updates for free (and complain in this forum about bugs and want to be heard and steal your time). But this can be solved easily by yourself: Just sell your major new features. I certainly would have paid for DrizzleIntegration (but not for a file explorer, new file format or the windi module BTW).

Rüdiger

Well, PixInsight itself is not my business, so I cannot answer you here. But I think the keyword here is "wish".


Best regards,
Vicent.

[miska]

This thread is very interesting....
Clearly there is a wish for more automation from some people. I think BPP is a wonderful example of what automation can be. I makes assumptions which may not always be optimal, but allows to get a decent result without much tinkering. You want better ? You can tweak. You want even better, don't use BPP, and use the "bricks" that make BPP.
I bet you, that BPP is one of the most used routines in PI. I think that fact tells a lot about how people want to use PI.

Right now, to me, PI has a lot of those "bricks", but few "meta-routines" like BPP, which make processing much faster, with "pretty good" results (without tweaking). Is it a black box ? If you want, you can know what goes inside. So no, it's not a black box in my opinion. It is a meta-routine. Nothing opaque in there. IF there was no documentation and the routines could not be read to know how they work, then it would be a black box.

Since some of the routines take quite a while to run, it could be nice to have the system automatically generate several "variants". Yes, it would take even longer, but you could go for a coffee, come back, and have (for example) 5 different images calculated, with 5 levels of noise reduction, and choose the one you like best.
But an even better solution would be a "meta-routine": first evaluate the noise level (there are routines for this in PI), evaluate (for example) the noisiest wavelet layers, do some other analysis, and then apply the noise reduction that has found to be "best" for this noise level. Yes, this takes time to test and calibrate, but once it is done, it could be quite efficient. Of course, some people like more noise reduction, some people less - but at least it would be a starting point. The routines are there, but effort should be made to automate - using the existing tools.

It is also clear now that the PI team has no interest in automation, and that this burden will fall on the users. I feel this is disappointing.

[tommy_nawratil]

hello,

discussing in extremes like "blackbox" and "full knowledge about every process" rarely leads to fruits.

Rüdiger gave good examples what would need improvement, like drawing a line from A to B
needs the respective X, Y values written down on a piece of paper and entered into some dialogue.
Hey, this is a basic task a computer program should make easy. Live previews that enable scrolling through the full pic
at full zoom level. Mixing pictures with brightness fade in and fade out regions, while applying a mask, and live preview for it.
There are so many things that would ease the use, and help saving time and give faster feedback on the parameters usefulness.

This has nothing to do with philosophy. I totally agree with Vicen, the most important point is developing an eye for the picture
and train the sense what is the next step and how could it be done. It could be more effective if we could concentrate more on it,
and less on how to get the tools working to the point we want them.

best regards, Tommy