Processing Challenge: M33 Triangulum Galaxy

[georg.viehoever]

Hi all,

after seing the fantastic work that real experts can do with professional data (e.g http://pixinsight.com/forum/index.php?topic=1507.0 M57 with Calar Alto Observatory 1.23 m Telescope), I wonder what PI can do for those of us that have to live with more modest equipment and skies. In my case that is a 6 inch f/5 Newton on a mount with rather large periodic errors, no guiding, fairly bright Munich/Germany skies, Canon EOS 40D with LPS filter. I usally take 20-40 light frames of 120 secs, plus assorted dark, flat and bias frames. I stack with DeepSkyStacker.

Here are some files showing M33 that might be used to demonstrate how real expert would handle this rather modest quality data:
- 50 Mbytes FITS file as it comes from DSS http://cid-17334332edde1430.skydrive.live.com/self.aspx/Public/Pixinsight/ProcessingChallengeM33/stacked_80percent_center.FTS
- 50 MBytes FITS file from lights, darks, flats, bias preprocessed with the Canon Banding Reduction Tool, then stacked with DSS. I was hoping that this gives me more signal. I am not so sure after some tries to process it: http://cid-17334332edde1430.skydrive.live.com/self.aspx/Public/Pixinsight/ProcessingChallengeM33/stacked_80percent_debanded.FTS
- My best result so far, 500 KBytes JPG (small version attached below): http://cid-17334332edde1430.skydrive.live.com/self.aspx/Public/Pixinsight/ProcessingChallengeM33/stacked_80percent_center_processed.jpg

I think the specific problems of my data are relatively high noise on the red channel, the notorious Canon Banding Problem, and the bright sky background (low S/N ratio). I wonder if you can do better than I did so far. Feel free to give it a try, and please publish your best processing approach.

Georg

[David Serrano]

Hmm, I'm getting this:

Code: [Select]

XML Parsing Error: not well-formed
Location: http://cid-17334332edde1430.skydrive.live.com/self.aspx/Public/Pixinsight/ProcessingChallengeM33/stacked_80percent_debanded.FTS
Line Number 120, Column 20:

for (var i = 0; i < selfPageData.items.length; i++)
-------------------^
That "live.com" smells of m$...   it seems it's interpreting the less than symbol as the start of an XML tag xD

[georg.viehoever]

David,

currently this is the only public storage that I have, and yes, it is MS. But it works fine for me with IE8 and Firefox on Windows Vista. Let me know where to upload the data for you, and I will make it available to you.

Georg

[David Serrano]

Don't upload it elsewhere just for me. I tried to download your data just to place it in the queue of "raw data from other people to be processed when I have some time", and I don't expect to have time for processing all that data in at least 30 years..

Thank you in any case!

[dhalliday]

Hey,...its cloudy tonite...I love a challenge...I will have a look at your DSS stack...
Here is my best result...its hard to get well !!

http://www.flickr.com/photos/daveh56/4043168228/sizes/l/

Dave

[dhalliday]

Well...here is a quick stab at it...maybe not much different than what you got...
The funny thing is,that when I applied ABE to it,all the data is to the far left,and it is BLUE +++

Anyways..Here is what I got...
An awful lot of color stretching had to be done,as when I corrected the blue,I was left with mostly just white...
Having said all this,I am mostly a "novice" at processing...
"GreyCstoration messed up the star centers a bit...
Is the camera modified ??...very little red signal...
Here it is;

http://www.flickr.com/photos/daveh56/4078239523/sizes/l/

I am also curious to see what experienced folks might be able to do...

[georg.viehoever]

Hi Dave,

no, the camera is not modded.

What I like about your interpretation of the data is that it appears to be much less noisy. Which processes did you use?  I think I got the colors better-but also got more noise

Kind regards,
Georg

PS: Forget the old message - my mistake...

[dhalliday]

I just used ABE (20 pix seperation),then GreyCstoration (regular settings,but if you apply a weak L mask first the stars are protected...)
Then 1.5 of L masked L NR...
I then (often) work the contrast up,and darken the background, through the use of Lum masks,in Histo...i use the mask,and then invert it,depending on whether I am brightening the nebula,or darkening the background. This of course costs me more noise...
The problem was color,or the lack of red,mostly.
With the background masked,in Curves,I cranked up the red,and did the same in "Saturation"...
I applied the default HDR Wavelet routine,under the L mask setting,then a bit more NR,masked,to the Lum,and Chrom channels...It is still quite white,but then your camera is not modded.
Lets hope a real pro takes a look...

cheers
Dave  

[avastro]

Hi everybody,
Here is my attempt to process Georg M33 data. This is a very complicated target to process and need a lot of data to reach a correct S/N. the kind of target who can easily dishearten even seasoned astrophotographer.
Georg your data are a bit noisy with a color bias in the right side of the image.A good master flat will help.
I've done a deconvolution at the start targeted to the galaxy with a star mask, I utilized several mask during the process Stars mask and two large structures.
I made a a DBE to remove the bias color background.
I processed by small steps taking care to avoid the noise to develop, I applied a noise reduction with ATWT two time during the process and finished the noise reduction with ACDR.
I applied also two times HDRWT  at different step of the process.
I adjusted several time the colors with the histogram transformation tool as curve transformation tool targeted to the galaxy.
The whole process take around one hour. Everything has been done with PI 1.509.

http://astrosurf.com/avastro/Georg-M33.jpg

Antoine
http://astrosurf.com/avastro

[georg.viehoever]

Hi Antoine,

I am glad to hear that this is a difficult target, it was driving me crazy during preprocessing. I like the level of structure that you were able to highlight in your processing. Good work. I guess the HDRWavelets were the key to sucess? I don't like the blue tint of the nebula in your picture, I think I got it better in my attempt that even shows some of the red regions.

About the "sky gradient" you noticed: I am not sure that bad flats are the cause. I think the sky in Munich just has this kind of gradient because of all the illumination in the streets. But I am not sure.

Thanks for your work, I am certainly learning about some critical steps (never tried a deconvolution before).

Georg

[georg.viehoever]

Hi all,

just for the record, here is the procedure that I used for the processed image in the original post (see attached process screenshot):

0: I started with the stacked_80percent_center.FTS file. Experiments with the "pre-debanded" one were even worse.
1: I cropped a small piece of the borders, since this frequently seems to have different characteristics than the main part of the window. I think this is caused by some DSS characteristic.
2: Format conversion from int to float. Otherwise I get steps in brightness values after strong stretches.
3: DBE to get rid of the remaining sky gradient
4: Canon Debanding Reduction Script. Removes much of those annoying dark/bright bands that severely limit the amount of Histogram Transformation that I can do. I am not sure if I should better do this before DBE, probably needs some experimentation.
5: Background Neutralization
6: Color Calibration. These two are soooo useful the arrive at a decent color balance. Glad they are available...
7: Histogram transform, RGB only, no special handling for red. Previous steps have been done with STF, which is switched off after this step.
8: masked ACDNR of dark areas to reduce the noise in the low signal portions of the image. The mask was created by copying the image, transforming it to greyscale, using Binarize to identify bright areas, and AtrousWaveletTransform to soften the borders between bright and dark regions
9: ColorSaturation, mainly pushing the red part of the color spectrum, such that the red nebulas become clearly visible
10: Curves transform, clipping the dark red portions to get rid of red channel noise
11ff: mirror and flip image into right position
15: Experiments with HDRWavelets produced dark "blots" in some parts of the galaxy, which I did not like Same is true for the Dark Structure Enhancement script. So I did not apply them in the published version. However, I still would like to find a way for enhancing the spiral structure of the galaxy.

Quite a lot of experimentation was required to find this procedure and its parameters. But still, I am not sure if I am missing important steps or do things it the wrong order. I am hoping for additional input from the real experts.

Have a nice evening (and clear skies),
Georg

[Harry page]

Hi

Have had a quick look at your image and offer the following comments

1) There is not enough signal  for decon

2) I must admit there is not a lot of red signal

3) To much noise reduction is not a good thing

4) Your image is very green have you used the SCNR tool !!!!!!!

I will have a play but there is not a lot of signal


Regards Harry

[avastro]

Hi Georg,
Few comments about your result compared with my attempt, I considered that this image have enough background to show an histogram with the 3 components at the same level in relation to the black point, but I agree my color isn't right. looking at your histogram the red is clipped, a bit too "pushed" to show the red color in the stars formation region probably. Some Ha exposure will help to improve  that.
I extracted the RG and B channel for your image to look at the signal, obviously the red channel is really weak.
The main difficulty is the lack of signal to get a nice picture. The  total time exposition with a telescope of 5 to 6" diameter should be at least around 6 to 8 hours for RGB panes to expect enough signal, of course dark skies help a lot, as a sensitive chip.
Depending of the chip utilized extra time for one color could be needed.
As said previously this is a very difficult target and you made a great job.

Antoine