Author Topic: Newbie Processing Help  (Read 2565 times)

Offline manojkoushik

  • Newcomer
  • Posts: 20
    • View Profile
Newbie Processing Help
« on: 2016 June 18 15:59:33 »
Hello all, very new to the forums and also just starting to use Pixinsight. Recently captured M81/82 using a QSI683
H-alpha-20nm: 12 subs of 900s
Stromgren Y: 12 subs of 900s
Stromgren V: 12 subs of 900s

(I use this combo on continuum objects to get around light pollution since these still measure the planck curve but are narrower than regular RGB)

Attached are the calibrated, registered and integrated masters and then saves from checkpoints in my workflow: https://www.dropbox.com/sh/x2hn90ixf3s0s7e/AADTpi5EwwLg7EXfkq4zXFgOa?dl=0

The moon was waxing and there are a lot of street lights, which can be seen as gradients in the image.

I followed the following sequence:
- Color Combine (ChannelCombine.xisf)
- ABE (ABE.xisf)
- Another ABE to remove residual gradients (ABE_ABE.xisf)
- Color Calibration using structure detection on M81 (ABE_ABE_ColorCal.xisf)
- Background Neutralization
- Dyamic crop to get rid of edges
- SCNR to remove a little bit of green haze left
- Histogram stretch
- ACDNR to remove noise
- Histogram to redo black point (ABE_ABE_ColCal_BN_Crop_SCNR_His_ACDNR_His.xisf)
- HDR to bring out detail
- Histogram to redo black point (ABE_ABE_ColCal_BN_Crop_SCNR_His_ACDNR_His_HDR_His.xisf)

Would love to hear comments on the steps above and if there is a better way to get to the final image. I am not sure I am doing the best to tease out what data I have. Someone else processing the same set with their flow and showing me the results would be much appreciated as well.

Offline manojkoushik

  • Newcomer
  • Posts: 20
    • View Profile
Re: Newbie Processing Help
« Reply #1 on: 2016 June 19 17:53:25 »
No takers?

Offline biosif

  • Newcomer
  • Posts: 40
    • View Profile
    • http://www.astrobin.com/users/iosifbodnariu/
Re: Newbie Processing Help
« Reply #2 on: 2016 June 20 05:55:57 »
Hi.
Your file on dropbox is. There was an error downloading your file.
Please fix it and then i'll show you some steps to proces image.

Thankyou.
Astrotech AT8IN, CGEM, Atik 314 L+

Offline manojkoushik

  • Newcomer
  • Posts: 20
    • View Profile
Re: Newbie Processing Help
« Reply #3 on: 2016 June 20 18:18:45 »
Thanks for the response and the offer to help.

So sorry about the permissions. Apparently dropbox wants me to upgrade to do that.

Find the files here instead: https://drive.google.com/folderview?id=0B6waQZAzxHToWWdrZmNpc2l6THM&usp=drive_web#list

Clicking on each allows you to download them.

LMK if you hit any snags.

Offline jkmorse

  • PixInsight Padawan
  • ****
  • Posts: 884
  • Two questions, Mitch . .
    • View Profile
    • Jim Morse Astronomy
Re: Newbie Processing Help
« Reply #4 on: 2016 June 21 18:43:29 »
On the noise reduction, there are newer tools than ADCNR.  In the linear stage, you can use Morphological Linear Transform, TGVD or the Mure denoise script.  After stretching, try using TGVD.  Other thing is I typically try to clean up my stacks with DBE or ABE before I combine them, then, if necessary, you can apply either to the combined stack to clean up any residual.  Before doing the channel combination consider using Linear Fit to equalize the color channels.  I would also run background neutralization before doing color calibration. 

I have set all this out in great detail on these and other steps for PI in a workbook I share with others.  If interested, drop me a line at jkmorse57@gmail.com and I will shoot you a copy.

Oh yeah, one more thing, try not to shoot with both street lights AND a full moon :)

Best,

Jim
Really, are clear skies, low wind and no moon that much to ask for? 

New Mexico Skies Observatory
SBIG STXL 6303E w/AOX
Planewave CDK17 - Paramount MEII
Planewave IFR90 - Astrodon LRGB & NB filters
SkyX - MaximDL - ACP

http://www.jimmorse-astronomy.com
http://www.astrobin.com/users/JimMors

Offline jkmorse

  • PixInsight Padawan
  • ****
  • Posts: 884
  • Two questions, Mitch . .
    • View Profile
    • Jim Morse Astronomy
Re: Newbie Processing Help
« Reply #5 on: 2016 June 21 18:47:42 »
One other thing to think about and its worth doing some research on this.  Typically, when faced with light polluted skies, it is better to shoot shorter subs, and a lot of them.  My longest subs are 900s (for NB only, 600 for RGBs) and I shoot in some of the darkest skies in the country.  something to test.

Jim
Really, are clear skies, low wind and no moon that much to ask for? 

New Mexico Skies Observatory
SBIG STXL 6303E w/AOX
Planewave CDK17 - Paramount MEII
Planewave IFR90 - Astrodon LRGB & NB filters
SkyX - MaximDL - ACP

http://www.jimmorse-astronomy.com
http://www.astrobin.com/users/JimMors

Offline manojkoushik

  • Newcomer
  • Posts: 20
    • View Profile
Re: Newbie Processing Help
« Reply #6 on: 2016 June 22 02:40:51 »
You are absolutely correct in that shooting with both LP and moon was a bad idea and so was the 900s exposures which builds up gradients significantly. I would get better SNR with more of the shorter subs. Lesson learnt.

On the plus side, that data I have is really making me learn all the tools which is great :)

I looked up some threads to see if doing DBE was better before or after channel combine and the threads seemed to indicate that after would be fine since it does it on each channel anyways.

Any good resources on Morphological Linear Transform, TGVD or the Mure denoise script?

How do I do Linear Fit before channel combine?

I will PM on that workbook.

Offline schwim

  • PixInsight Addict
  • ***
  • Posts: 117
    • View Profile
Re: Newbie Processing Help
« Reply #7 on: 2016 June 22 05:55:56 »
I found David Ault's presentation on noise reduction very helpful:

https://www.youtube.com/watch?v=em7AXdvNHRA

TGVDenoise is great to use because it is pretty simple to use. To play around, keep all settings as default and simply adjust the edge protection parameter until you don't see any artifacts. Use masking to control the application. This works well for a lot of data.

For Linear Fit info, I think Harry's Astro Shed has some useful info.

- Greg
Scottsdale, Arizona, USA

Offline jkmorse

  • PixInsight Padawan
  • ****
  • Posts: 884
  • Two questions, Mitch . .
    • View Profile
    • Jim Morse Astronomy
Re: Newbie Processing Help
« Reply #8 on: 2016 June 22 15:08:26 »
Linear Fit is really easy to use.  If you are working on three image stacks that you are going to combine, simply open all three, then open Linear Fit.  Select your best image as the one to input into the tool, then simply drag the little triangle at the bottom left onto the other two images.  You may need to reset STF to get the stretching to look right, but now all three images are using the same base.  I use this for both color combine and when creating synthetic luminance frames (I no longer shoot luminance frames but that is a whole different discussion).

Best,

Jim
Really, are clear skies, low wind and no moon that much to ask for? 

New Mexico Skies Observatory
SBIG STXL 6303E w/AOX
Planewave CDK17 - Paramount MEII
Planewave IFR90 - Astrodon LRGB & NB filters
SkyX - MaximDL - ACP

http://www.jimmorse-astronomy.com
http://www.astrobin.com/users/JimMors

Offline manojkoushik

  • Newcomer
  • Posts: 20
    • View Profile
Re: Newbie Processing Help
« Reply #9 on: 2016 June 24 20:17:21 »
Thanks everyone for the pointers and also to Jim for the handbook. Helped a lot. So I redid the pre-processing and processing and here's what I ended up with:
https://drive.google.com/drive/folders/0B6waQZAzxHToWWdrZmNpc2l6THM

- Redid calib to make sure I was hitting good SNR in the stack with the right pixel rejection
- Used subframe weighting and rejection in the stack for good SNR

I think these two gave me good stack frames. Then I did the following

- Linear fit using the best looking stack
- DBE on each channel to remove background gradients
- Synthetic luminance
- Channel combine using LRGB tool (no luminance selected)
- Another DBE to remove residual gradients
- Background neutralization
- Color calib
- Histogram stretch on RGB
- Histogram Stretch on L
- LRGB combine
- TGVD
- Histogram
- HDR
- Histogram

Seems like the end result is pretty good for the data I had.

What I would love your feedback on, is there are other steps or tools I can use in the chain above to make it even better.
« Last Edit: 2016 June 24 21:16:04 by manojkoushik »

Offline schwim

  • PixInsight Addict
  • ***
  • Posts: 117
    • View Profile
Re: Newbie Processing Help
« Reply #10 on: 2016 June 25 07:23:05 »
You have some odd artifacts in that image. Open the final, stretched version and run a screen stretch again on it. You'll see the vertical effect.

Also, you look to be very blue biased due to your filter selection. Not sure how to handle that. Maybe some experimentation with combining using pixel math is in order, or accept the purple look as it might be expected.

Interestingly, I did not get vertical effect. Here is my very quick and dirty result:



Very different data than I'm used to. I just switched to mono CCD from a DSLR so I guess I get to say that for a few more months. :)

I did no noise reduction beyond SCNR on this. Here's my process to get this - certainly not complete. :

 - ChannelCombination of RGB
 - DBE of the result
 - BackgroundNeutralization
 - ColorCalibration
 - SCNR
 - Deconvolution - not sure i really needed this
 - Modified STF-based stretch via HistogramTransformation
 - Curves - Contrast
 - Masked Curves - color - de-color background
 - Masked Curves - color - saturation of objects, stars

Curious - what OTA and mount are you using for this image?
- Greg
Scottsdale, Arizona, USA

Offline schwim

  • PixInsight Addict
  • ***
  • Posts: 117
    • View Profile
Re: Newbie Processing Help
« Reply #11 on: 2016 June 25 20:50:17 »
I gave the PixelMath combination a try. For the green channel I used a 50/50 blend of what I presume is your Y (your "green" sub) and B (blue) filters. I didn't do a deconv on this one, but the rest is more or less the same.

- Greg
Scottsdale, Arizona, USA

Offline manojkoushik

  • Newcomer
  • Posts: 20
    • View Profile
Re: Newbie Processing Help
« Reply #12 on: 2016 June 26 05:43:13 »
Hi there schim, thanks for taking a swing.

That vertical banding is from the QSI683WSG. Apparently some units had this banding issue. I am awaiting a firmware upgrade from them to fix it.

The mount is a Mesu Mount 200 and the scope is Stellarvue SVQ100. Mesu mount is simply awesome. No backlash and I consistently get <.5" RMS on guiding. Load capacity is 200 lbs. My polar alignment is less <3' in each. The sitech controller takes a bit getting used to, but after that is fairly powerful with a full mount model etc. and platesolve built right into it.

On the image, looking at what you did, seems like I am pretty much on the right track? Looks like I teased out everything that could be teased out?

On the blue bias, I am not using conventional RGB filters. Instead the set I am using is Ha 20nm passband for red, Stromgren Y (~16nm passband) for green and Stromgren V for blue (~16nm passband). Since they are used to measure metallicity, they give a good sampling of the planck curve and so can be easily calibrated to a G2V type calibration (or in the case of the images above, to the spiral galaxy). But looks like the optics and calibration is weighting blue a bit heavier. That said though, star colors still seem true both in yours and mine?

The pixel math case is very interesting and something I need to look into closer. It is probably approaching closed to what the true colors should be. What was your process here?

For R you used Ha 20nm as is.
For Green you combined Y and V 50/50
For blue you used V as is.

And then the process follows the regular RGB process?

Offline schwim

  • PixInsight Addict
  • ***
  • Posts: 117
    • View Profile
Re: Newbie Processing Help
« Reply #13 on: 2016 June 26 07:40:13 »

That vertical banding is from the QSI683WSG. Apparently some units had this banding issue. I am awaiting a firmware upgrade from them to fix it.

Hmm... interesting I didn't see it in the data when I processed it. I assumed something else brought it out.

Quote
On the image, looking at what you did, seems like I am pretty much on the right track? Looks like I teased out everything that could be teased out?

That's up to you, my friend. At the end of the day its art that you create.

Quote
On the blue bias, I am not using conventional RGB filters. Instead the set I am using is Ha 20nm passband for red, Stromgren Y (~16nm passband) for green and Stromgren V for blue (~16nm passband). Since they are used to measure metallicity, they give a good sampling of the planck curve and so can be easily calibrated to a G2V type calibration (or in the case of the images above, to the spiral galaxy). But looks like the optics and calibration is weighting blue a bit heavier. That said though, star colors still seem true both in yours and mine?

I felt they had a bit of a bias towards blue, so I tried the pixelmath combination method.


Quote
The pixel math case is very interesting and something I need to look into closer. It is probably approaching closed to what the true colors should be. What was your process here?

For R you used Ha 20nm as is.
For Green you combined Y and V 50/50
For blue you used V as is.

And then the process follows the regular RGB process?

Yes - that is how I mixed the channels. I was trying to make green more green, rather than using yellow by itself. To do this, I added blue to the yellow. I didn't experiment with the percentages, I just guessed 50/50. I think the color balance in that blend looks a little better to my eye, but it could still be a bit more balanced in the stars. For a final image I'd experiment with the blending a bit more.

From there you might add some noise reduction. I'd start with TGVDenoise, try to deal with chrominance noise first, then go after lum noise.You might try this at various points in the process to see how it affects, say, saturation adjustments.

- Greg
Scottsdale, Arizona, USA

Offline manojkoushik

  • Newcomer
  • Posts: 20
    • View Profile
Re: Newbie Processing Help
« Reply #14 on: 2016 June 26 16:29:32 »
The banding is there in all frames. Becomes more apparent after stacking and stretching.

Thanks for the help. I think I have a good starting point. The thing needs most modification right now is how I am capturing data. 15 minute exposures led to not so good SNR. I am going to try lower subs but higher number of them and see how that changes things. The color combination is still something I have to play around with. When I measured a G2V star for that filter combination, I got
Red-H-Alpha-20nm   0.53
Green-sY                   0.35
Blue-sV                   1.00

Which seems counterintuitive to what we have just seen. But worth a manual color cal. Will check that out next.