PixInsight Forum (historical)
PixInsight => General => Off-topic => Topic started by: cfranks on 2010 October 26 19:20:24
-
I didn't want to hijack the thread praising RBA for his excellent contribution to AIC2010, I only wish I could have attended, but I have a little confusion with the DSA philosophy, specifically the last paragraph:
>The following procedures are expressly rejected, due to the fact that they don't respect the data and the information therein:
?The incorrect application of algorithms that result in a knowing generation of false information, or in a knowing suppression of some existing information.
?The arbitrary selection of data without a criterion applied homogeneously to the whole image.
<
This suggests that images presented in the Hubble pallette et. al. are to considered False Information and should be rejected. The colours are obviously 'wrong' since we cannot see the actual narrow-band frequencies but the presentation surely gives us a much better (layman's) understanding of the chemical composition of the object.
Charles
-
Hi Charles,
A color palette doesn't implies that you are introducing new information or supressing it from the actual data set. OTOH, when you select a color palette, you are applying the same color rendition to the whole image (in Hubble's, simply SII=R, Ha=G and OIII=B). So to me there isn't any problem using a color palette. Furthermore, for me real color doesn't exist in AP, so you are always working with some sort of color palette. :)
Regards,
Vicent.
-
I don't mean to stir the fire but the fact is that a number of outreach Hubble images (if not many, or most) do not meet the DSA criteria of adequate astroimage processing. Not because they use such or such color palette but because they use what the DSA calls arbitrary selective processing - or something like that.
Zolt Levay and Lisa Frattare (both of whom were at this last AIC and with whom I had a chance to talk for a bit) have said many times they have to walk a fine line when processing their images so that they are both aesthetically pleasing and scientifically accurate. This involves "arbitrary" localized contrast enhancements and many other "arbitrary" techniques. It's called outreach, it works, and it contributes to science a lot more than what even the most "radicals" (none around here AFAIK) would think.
It doesn't matter whether I do or do not with my images whatever they do with theirs - what they do has all my support. Hopefully, most people would agree.
:angel:
-
The end doesn't justify the means. Of course I support science outreach —and the DSA statement clearly says that scientific spreading must be one of the main goals of astrophotography—, but it cannot justify the use of poor image processing techniques.
Arbitrary selective manipulations without a documentary basis are never necessary in astrophotography. They damage astrophotography because they are both an obstacle to its development and a quick path to lose its documentary nature. Even if they may seem admissible to achieve certain outreach goals in the short term, in the middle-long term they can destroy —are destroying IMO— astrophotography, at least as we understand it.
Obviously there are many gray zones, there are exceptions or special cases, and I agree that sometimes this can be very difficult to implement in practice. That's precisely one of the main reasons why we always must self-question regarding our image processing techniques and procedures.
Astrophotography is a wonderful mixture of art, science and technology. I think we have demonstrated that it is possible to achieve aesthetical goals without arbitrary selective manipulations and other poor practices, and at the same time, keep pushing the limits of the discipline. Here are a few recent examples:
http://astrofoto.es/M51.jpg
http://pixinsight.com/examples/NGC7331-CAHA/en.html
http://pixinsight.com/examples/M57-CAHA/en.html
-
Okay I've read the DSA Founding Statement and I'm having a problem understanding the following phrase
The arbitrary selection of data without a criterion applied homogeneously to the whole image
Jack or Juan would you please expand on what the meaning of this phrase is intended to be
Thanks
Nigel
-
Hi Nigel,
A homogeneous criterion means that all the pixels in a mask have been modified with the same criteria. I. e., you can modify a luminance mask applying a curve so that the objects get better selected and the background sky is darker. This is a homogeneous selection.
If you decide to make brighter the upper spiral arm of a galaxy in the luminance mask, then you are not doing a homogeneous selection.
V.
-
Hi Nigel,
Along with the excellent example that Vicent has just put, I think this is better described with some practical examples. This is a sample of things that are wrong or inadmissible in our opinion:
- Paint a mask by hand with a brush and apply a sharpening filter through that mask.
- Enhance or diminish some particular features arbitrarily. For example, tame down a bright star by direct intervention with a brush or a similar tool.
- Define an arbitrary selection, with a lasso tool for example, with the purpose of isolating one or more selected objects from the background.
- Replace the existing sky background with a synthetically generated background.
- Repair a prominent blooming artifact with a clone stamp tool, when some significant structures (other than the background or a smooth and constant region) must be "invented" on the regions occupied by the blooming.
- Apply image processing tools and techniques incorrectly. Incorrect application of image processing tools leads to generation of artifacts (I mean questionable artifacts such as ringing, excessive noise, or patterns), to generation of false structures (or the impression of them), or to the removal of existing structures.
Things that are OK in our opinion:
- Define a mask based on the brightness (luminance or lightness) of the whole image, activate it, and apply a sharpening filter in a reasonable way —without generating artifacts such as ringing or false structures— through that mask.
- Define a star mask based on morphological properties of the stars on the whole image, or using appropriate multiscale analysis techniques, and apply an erosion filter with caution —no star must be removed or diminished to the point it becomes practically invisible— through that mask.
- Separate all the objects on the image from the background with a mask based on the brightness of the whole image, then apply appropriate processes to the background or to the objects, separately. The kind of processes applied must be well adapted to the characteristics and physical properties of the selected regions. For example, it makes no sense applying deconvolution to the background, due to its inherent lack of signal.
- Use the clone stamp tool to repair relatively small artifacts or cosmetic defects. Bloomings can also be repaired manually, as long as one doesn't feel guilty of guessing too much. When in doubt, it is always preferable leaving the blooming as it is than falsifying what the blooming is supposed to be blocking.
An example of things that can be considered borderline (note: this is my personal opinion; other DSA signers may think differently):
- Increasing color saturation for a particular color. This can be admissible, in my opinion, as long as it is clear that a particular color is selecting a specific type of objects, and as long as the saturation increase is being applied with good doses of common sense. For example, I see no problem in raising saturation of the reds to enhance Ha structures. Another example would be raising saturation of the blues to enhance reflection nebulosity. In both cases, the applied transformation should only improve the visibility of some objects without creating false relations or false proportions between them. The point where these techniques begin falsifying an image is always a matter of opinion; this is definitely a gray area.
I hope this clarifies our opinions. I want to stress the fact that this is not intended to be, nor should it be understood as, a threat of any kind. This is only our particular vision of astrophotography, and we have formalized it publicly as a contribution to the development of this discipline.
-
Thanks for all the interesting explanations and examples. I can sleep easier now. 8)
Charles
-
This is very interesting. Like Rogelio I spoke with Lisa Frattare of the Hubble Heritage team at AIC last weekend and saw her presentation about the work that her group did for the Hubble IMAX 3-D film. She had the designation of "star cleaner" in her group because she goes around with a paint brush tool and cleans up stars. I could not tell if this was only for the work in the IMAX film where they were rendering 3-D images or for the standard outreach images that her group publishes on the internet. For the IMAX film they were constructing 3-D fly-throughs of space, substituting galaxies and stars willy-nilly. Since the point of view of these was extra-terrestrial I assume there could be no objection on DSA grounds for this work.
Mike
-
i think even if the hubble heritage team were card-carrying members of the DSA they'd still have had to resort to hand-methods for everything they were doing for Hubble 3D. of course, if they were DSA members perhaps they would have had to refuse to work on this movie out of principle!
i was a little disappointed that they were kind of winging it on the 3d modeling of M42. to be honest, when you get up close to a nebula i'll bet its so diffuse that you can't even really see it, so the whole thing is probably unrealistic.
but i did see the movie in 3d and it is fantastic. it's a shame there are so many commercial 3d IMAX movies now. i think hardly anyone has seen this movie because all the 3D IMAX theaters were showing other movies.
apparently it will appear at some point as a 3D blu-ray, so hopefully more people will get to see it.
anyway as i mentioned in the other thread i did encourage Lisa to check out pixinsight. i hope she and Zolt do so.
-
I want to stress the fact that this is not intended to be, nor should it be understood as, a threat of any kind.
Why you keep saying that? Just because you see those who do not agree with you as a threat - they're destroying astrophotography, you say (I won't even get into that) - doesn't mean they see you as a threat as well.
-
I want to stress the fact that this is not intended to be, nor should it be understood as, a threat of any kind.
Why you keep saying that?
I picked up on this as well. Let me stress I'm not a threat to anyone - honest ;) In fact I think the DSA is a very commendable concept. I am simply trying to understand the statements. On this point I still think the English is not what you intended it to mean but maybe that's just my literal interpretation and, as a Chemist homogeneous means something else entirely .......
-
Just because you see those who do not agree with you as a threat
No I don't see anybody as a threat; I am certainly too old for that :) It is bad practices what is threatening astrophotography, not the persons. So the "those who" part of your sentence is inaccurate to describe what I think and what I have said.
Why you keep saying that?
Because some attitudes that I have heard and read (not here) indicate that the DSA initiative is being seen as a threat, or as something that requires a defensive reaction. We are not trying to go against anybody, but against bad image processing practices and its current proliferation in astrophotography (also in other documentary photography branches). Our intention is exclusively contributing to the development of image processing culture in astrophotography. Being seen correctly —that is, as we really are— is important and requires additional care on our part. I probably should have written this exactly as I've just written it here instead of the way I did it before, to avoid confusion.
-
I don't share the a lot of the DSA concept.
However, I do say it best think local but act global.
That is opposed to acting locally only.
Use global processing to bring out details on the local level.
This includes attacking portions of the images based on variety of parameters.
Acting on the images elements by intestity, hue, structure, noise level, shape is whole lot more fun too.
When we act locally only (paint brush etc ) it is then often the methoid of last restort.
Personally I think global approaches are a lot easier too.
I most techinques that require heavy use of painting are rather a pain IMO.
I really don't want to spend hours with a WACO pen tablet
I admit the brush can be a mighty tool in a few circumstances where is really hard to select a local feature based on global parameters. Then I say go for it.
Max
-
I ran across the DSA website a little over a year ago. I had to read it several times because I couldn't understand what the philosophy was saying. I came away with mixed feelings, I was delighted, appalled, and amused. I do know that at the time it made me not want to buy PixInsight. After reading this thread, I have many more questions. Perhaps a list of my observations can be presented so someone can address them one by one?
1. The philosophy is black and white, at least until a proponent decides an exception can be made, then it is "ok".
2. The philosophy purports to be constructive, but the use of words like "false", "fraud", "honesty", "unethically", etc. can be highly emotionally charged.
3. The philosophy is very difficult to read and understand, a specific list of proscribed processing methods would have been helpful.
4. The philosophy appears to be very gray now to me after reading this thread.
So I would like to present some situations, if you could help me understand each dilemma in the context of the DSA philosophy. It is not my objective to be negative, but to develop my understand of the philosophy. (I did purchase PI last night, I look forward to learning it)
Situation 1:
Although I work with CCDs, my primary means for my astroimaging endeavors has been a Canon DSLR. These are known for severe horizontal banding when the image data is highly stretched. Now, the philosophy states that is is ok to correct for instrumental defects, but in a post above it is indicated that the use of an artificial background is prohibited. The only way I can see to recover the image is by using an artificial background to correct for the instrument defect. My only other choice is to throw away the data, throw away the camera, and quit astroimaging altogether. Background is not signal anyway, and I recovered every single little faint fuzzy. Here is the image in question: http://www.darkfireastro.com/images/M33.html.
My goal is not to document, but to make a "pretty picture". If I state this as a disclaimer on my images, would this satisfy the DSA? Please comment in context of the DSA philosophy. Thank you.
-
This is my first post on a PixInsight forum. As a new imager maybe I shouldn't even comment, but I feel compelled to chime in. I read the DSA Founding Statement yesterday and I found it very well-reasoned and persuasive. It very much matches my own philosophy - maybe that's why I like it. :)
In my opinion, the signers of the DSA and the supporters of the DSA are exhibiting a fair amount of courage. It's never easy to challenge the practices of others. Inevitably, someone is going to get defensive. Nonetheless, I think the issues raised by the DSA Founding Statement and the discussions in forums like this are very healthy. If nothing else, these discussions force people to face this issue head-on and really consider how they want to contribute to the "development of the image processing culture in astrophotography" (quote from Juan elsewhere in this thread).
Finally, I was at the presentation by Lisa Frattare at the AIC (Adding Depth to Hubble Images for Hubble 3D IMAX). I must admit that I was very uncomfortable with the amount of image manipulation that was described in her presentation. I am not a film maker and I would freely concede that maybe the manipulation was necessary in order to create a 3D movie. However, if that is the case it should be so stated in the movie credits at the end. In my opinion, Hubble is a scientific instrument not an entertainment industry tool. Yes, outreach is very important but I think there's a dividing line somewhere and I'm suspicious they crossed over the line. If I'm wrong please feel free to educate me.
Just my two cents worth...
Thanks, Ken
-
Just because you see those who do not agree with you as a threat
No I don't see anybody as a threat; I am certainly too old for that :) It is bad practices what is threatening astrophotography, not the persons. So the "those who" part of your sentence is inaccurate to describe what I think and what I have said.
Why you keep saying that?
Because some attitudes that I have heard and read (not here) indicate that the DSA initiative is being seen as a threat, or as something that requires a defensive reaction. We are not trying to go against anybody, but against bad image processing practices and its current proliferation in astrophotography (also in other documentary photography branches). Our intention is exclusively contributing to the development of image processing culture in astrophotography. Being seen correctly —that is, as we really are— is important and requires additional care on our part. I probably should have written this exactly as I've just written it here instead of the way I did it before, to avoid confusion.
You say you are addressing bad imaging practices, and I can understand that position. But when you use words like fraud, unethical, and honesty, then those terms are an attack against people, or are at least viewed that way. I myself don't feel threatened, actually I was more amused. I'm not laughing at you, but the website presents itself to me as proselytizing, kind of like a fire and brimstone evangelist at a tent revival. I notice there do not seem to be any more members to have signed on in the past year. Again, I am not trying to be negative, just trying to understand.
-
Hi Michael,
Situation 1:
Although I work with CCDs, my primary means for my astroimaging endeavors has been a Canon DSLR. These are known for severe horizontal banding when the image data is highly stretched. Now, the philosophy states that is is ok to correct for instrumental defects, but in a post above it is indicated that the use of an artificial background is prohibited. The only way I can see to recover the image is by using an artificial background to correct for the instrument defect. My only other choice is to throw away the data, throw away the camera, and quit astroimaging altogether. Background is not signal anyway, and I recovered every single little faint fuzzy. Here is the image in question: http://www.darkfireastro.com/images/M33.html.
Very nice image indeed. No, there's no problem at all with repairing banding. In fact, we have a nice and efficient banding reduction script written by Georg Viehoever. You'll find it on the Process Explorer window, under the Scripts > Utilities category (or under the corresponding options of the Script menu). Its name is CanonBandingReduction.
When I wrote "replace the existing sky background with a synthetically generated background" I was referring to a completely different thing; I now realize that I have been quite inaccurate; sorry for that. I refer to techniques that select the whole background (e.g. with a magic wand tool) and replace it with a (usually noise-free) synthetically generated background. Some of these procedures then add a certain amount of synthetic noise to achieve a more "natural" look. Although I can understand why these procedures are implemented, the background is part of the acquired data and due to its uncertain nature (its signal-to-noise ratio is very low) it cannot be selected with the necessary accuracy as to affirm that by replacing it in this way no object is lost. There are appropriate noise reduction techniques and tools to address these problems, which are much more respectful with the data.
-
when you use words like fraud, unethical, and honesty, then those terms are an attack against people, or are at least viewed that way.
We don't try or pretend to attack anybody. The paragraph of the DSA statement where these words are included is this one:
In the time of chemical photography the processing resources were much more limited than they are today, although it was much easier to distinguish between fair and false works. Today, unfortunately, there are frauds in astrophotography, despite the fact that a majority of authors work with absolute honesty and don't manipulate their images unethically.
The fact is that there are frauds and unethical practices related to astronomical imaging. For example, there are people that steal the works of others. Some users on this forum could put several examples. There are also people that purposely falsify images. These things happen. In the above paragraph, we are not talking about wrong image processing techniques, or techniques that we don't like, but about other conducts that are possible "thanks" to the power and versatility that digital imaging provides, in contrast to the time of chemical photography, when these things were much more difficult. That's all that the above paragraph wants to say, but perhaps we weren't clear enough, or we expressed it poorly.
the website presents itself to me as proselytizing, kind of like a fire and brimstone evangelist at a tent revival.
I'm sorry to read that. I don't agree but that's your opinion and I respect it. This is mine: we are not trying to proselytize; we have just formalized publicly our own vision of astrophotography.
I notice there do not seem to be any more members to have signed on in the past year.
That's true. In part that's because we haven't had the time to dedicate more efforts to the DSA project (you'll see that most of the website is under construction for a long time). And I suppose not a lot of people are interested. But if the amount of people interested were something to decide whether or not to start and continue a project, then not only the DSA, but PixInsight wouldn't exist since some years ago :)
-
Hi Michael,
Situation 1:
Although I work with CCDs, my primary means for my astroimaging endeavors has been a Canon DSLR. These are known for severe horizontal banding when the image data is highly stretched. Now, the philosophy states that is is ok to correct for instrumental defects, but in a post above it is indicated that the use of an artificial background is prohibited. The only way I can see to recover the image is by using an artificial background to correct for the instrument defect. My only other choice is to throw away the data, throw away the camera, and quit astroimaging altogether. Background is not signal anyway, and I recovered every single little faint fuzzy. Here is the image in question: http://www.darkfireastro.com/images/M33.html.
Very nice image indeed. No, there's no problem at all with repairing banding. In fact, we have a nice and efficient banding reduction script written by Georg Viehoever. You'll find it on the Process Explorer window, under the Scripts > Utilities category (or under the corresponding options of the Script menu). Its name is CanonBandingReduction.
When I wrote "replace the existing sky background with a synthetically generated background" I was referring to a completely different thing; I now realize that I have been quite inaccurate; sorry for that. I refer to techniques that select the whole background (e.g. with a magic wand tool) and replace it with a (usually noise-free) synthetically generated background. Some of these procedures then add a certain amount of synthetic noise to achieve a more "natural" look. Although I can understand why these procedures are implemented, the background is part of the acquired data and due to its uncertain nature (its signal-to-noise ratio is very low) it cannot be selected with the necessary accuracy as to affirm that by replacing it in this way no object is lost. There are appropriate noise reduction techniques and tools to address these problems, which are much more respectful with the data.
I will be MOST interested to try to CanonBandingReduction tool, I have many, many data sets that were unusable due to banding. Once I discovered the cause of the banding, I was able to continue imaging without fear of bad data. I tried several banding tools in the past, but they've only been partially effective.
Unfortunately, in this image, I did as you described. I even fine-tuned the process further by creating a screened R, G, and B background, with random noise inserted, to simulate a natural background as would be given by a OSC. Except I almost never use magic wand, I used the select color tool, it is more accurate. Believe me, I pored over the image background selection to make sure nothing else was included. But that doesn't resolve the dilemma.
Can the CanonBandingReduction Tool be used last in the image processing, or is it necessary to do it early in the process?
-
This is very interesting. Like Rogelio I spoke with Lisa Frattare of the Hubble Heritage team at AIC last weekend and saw her presentation about the work that her group did for the Hubble IMAX 3-D film. She had the designation of "star cleaner" in her group because she goes around with a paint brush tool and cleans up stars. I could not tell if this was only for the work in the IMAX film where they were rendering 3-D images or for the standard outreach images that her group publishes on the internet. For the IMAX film they were constructing 3-D fly-throughs of space, substituting galaxies and stars willy-nilly. Since the point of view of these was extra-terrestrial I assume there could be no objection on DSA grounds for this work.
Mike
I have been having a verbal discussion with Lisa since her presentation, which has since carried on into an email correspondence. I have directed her to the DSA site, as well as the PI forum. I invited her to come on the forum, she and Zolt are very interested, but fear there might be "tomatoes thrown". I assured her that we are having a healthy discussion, I am finding all this fascinating (Mr. Spock twirling his imaginary mustache <g>)
-
Hi Michael,
You may be interested in the original forum thread where this script was first developed:
http://pixinsight.com/forum/index.php?topic=1159.0
In particular, take a look at the screenshots on the second page of the thread.
As all image calibration and preprocessing routines, the CanonBandingReduction script will work better at early stages, when the images are still linear (before the initial nonlinear histogram stretch). However I think it should also work with nonlinear images. I hope Georg will chime in because he is the father of this creature.
-
I invited her to come on the forum, she and Zolt are very interested, but fear there might be "tomatoes thrown"
She (or they) would be very welcome. Nah, we don't use to throw the tomatoes very hard (joking ;D)
We are always open to all opinions and enjoy constructive discussions, especially about very important topics like this one. Their contributions would surely be extremely interesting here.
-
Hi Ken,
I read the DSA Founding Statement yesterday and I found it very well-reasoned and persuasive. It very much matches my own philosophy - maybe that's why I like it.
Welcome to PixInsight Forum!
Thank you for your support and nice words.
-
Hi Michael,
You may be interested in the original forum thread where this script was first developed:
http://pixinsight.com/forum/index.php?topic=1159.0
In particular, take a look at the screenshots on the second page of the thread.
As all image calibration and preprocessing routines, the CanonBandingReduction script will work better at early stages, when the images are still linear (before the initial nonlinear histogram stretch). However I think it should also work with nonlinear images. I hope Georg will chime in because he is the father of this creature.
Wow, that's quite an extended thread. I've downloaded the latest version, and will be trying it as soon as possible. That may not be too soon, we are building two new obs. What will be interesting is if this also fixes the old "dark river" issue from the Canon 20D. Thanks for the reference.
-
...As all image calibration and preprocessing routines, the CanonBandingReduction script will work better at early stages, when the images are still linear (before the initial nonlinear histogram stretch). However I think it should also work with nonlinear images. I hope Georg will chime in because he is the father of this creature. ....
I am honored to be called the "father" of this script, but in reality it is based on ideas that I found in Jens Dierks's FitsWork http://www.fitswork.de/software/softw_en.php . I have been using it both in early stages of processing (i.e. immediately after calibrate+align+stack+debayer) and late stages just before the final noise reduction/stretch with nice results. I have the feeling that it might work even better if applied to the RAW images (fixing the slightly distorted statistics of the Canon images), but that would probably require implementing Bayer patterns in the script...which I have not implemented yet.
Georg
-
Hello,
I wish to add a further reflection on the DSA topic. As you know, I started about 13 years ago doing amateur astrophotography and became professional about four years ago.
One of my main goals from my actual position is to open a small door to professional astrophotography.
The DSA statement is not anything to believe on. It's very simple: it is the particular concept about astrophotography by a group of astrophotographers. Instead of believing, to me it's more important to use the DSA statement as a basis for education.
Today in our discipline, there isn't any defined thought basement, and this reflects in the astrophotography community (in my particular teaching experience) because 99% of the astrophotographers need a better basic education. This is even more needed in the digital imaging area. In some aspects, a large amount of astrophotographers continue having a completely analogic conception of the digital image.
Having said this, to me is quite evident that a professional future for astrophotography can be a reality only if we advance in education. But this education cannot be possible if we don't know how to educate. The DSA is the first initiative in the world.
This is why, when I teach, I cannot separate DSA, education and PI.
For the moment, I consider the DSA a successful project. Just after founding it we started working at Calar Alto Observatory with a 1.23 meter scope. We have again 55 nights for next year. And we have published (and will continue publishing) some innovative works.
I think that an observatory must have a small portion of observing time for astrophotography and outreach. This is extremely hard to accomplish today, but we're working on...
Best regards,
Vicent.