Author Topic: Pixinsight Workflow  (Read 11131 times)

Offline DaveS

  • PixInsight Addict
  • ***
  • Posts: 115
    • Dave's Astronomy Pages
Pixinsight Workflow
« on: 2010 October 02 07:52:21 »
Over the years I have adopted a pretty standard workflow for my image processing, starting at calibration through to the final image.

This of course has not been with Pixinsight (PI).

PI introduces some new processes to this worflow, such a DBE, and I would be interested to know what you find is your typical workflow when using PI.

For example, at what stage do you apply DBE, and unsharp masking etc?

Dave
8" LX200ACF
William Optics FLT110
NEQ6 Mount
SXVF-H9
SXVR-16
SX Lodestar
DMK21AU04
Baader LRGB and NB filters
DiY Observatory
http://www.progressiveastroimaging.com/davesastronomy/

Offline Nigel Ball

  • PixInsight Addict
  • ***
  • Posts: 277
    • Astrophotography by Nigel
Re: Pixinsight Workflow
« Reply #1 on: 2010 October 02 08:36:37 »
Dave

DBE is the first task I do after combining the FITS files into a RGB, Luminance or Gray for Narrowband

UM is the last task I do

Nigel
Nigel Ball
Nantwich, Cheshire, United Kingdom

Takahashi FSQ-106 at f/8, f/5 and f/3.6 on AP900, Nikon 28 mm and 180mm f/2.8
SBIG STL-11000M, Astrodon LRGB, 5nm Ha
ST-10XME, Astrodon HaLRGB
www.nigelaball.com

Offline DaveS

  • PixInsight Addict
  • ***
  • Posts: 115
    • Dave's Astronomy Pages
Re: Pixinsight Workflow
« Reply #2 on: 2010 October 02 08:45:58 »
Hi Nigel

It seemed to me that following calibration, alignment and stacking, DBE would be the next process in-line, but its good hear how other go about it.

USM is always pretty near the 'tail-end' for me, but always with the star out of the way on a layer (PS speak  ;)).

Dave
8" LX200ACF
William Optics FLT110
NEQ6 Mount
SXVF-H9
SXVR-16
SX Lodestar
DMK21AU04
Baader LRGB and NB filters
DiY Observatory
http://www.progressiveastroimaging.com/davesastronomy/

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Re: Pixinsight Workflow
« Reply #3 on: 2010 October 02 08:57:25 »
Hi Dave,

OK - although this has been discussed many a time elsewhere on the Forum, there is no reason why it can't be revisited here, on a 'dedicated' thread.

This is 'my take' for RGB (or three-channel) image processing, assuming that all the calibration steps have already been completed:-

Dynamic Crop - you need to eliminate 'rough edges' around the periphery of the image
STF - to let you see what you are dealing with (noting that this doesn't actually 'change' the image in any way)
DBE - to eliminate any sky gradients (and take the time to inspect the 'subrtacted' image, to ensure to 'real data' was eliminated by mistake)
STF#2 - renew the STF to cater for the fact that DBE may have significantly altered the image
DBE#2 - consider whether a second, 'coarser' DBE is needed for serious gradient issues (not often the case)
STF#3 - assuming that DBE#2 was implemented
BN - BackgroundNeutralisation to get the background looking right - preview(s) may be needed, and I will also use the PreviewAggregator PJSR script
CC - still with whatever image view was used to define the Background, and with more previews defined and aggregated as necessary, deal withthe foreground colour, by assuming that defined areas of the image SHOULD be 'nominally white')
ACDNR - using a carefully engineered Luminance mask to protect the bright areas, reduce noise in the dim areas (often applied 'stronger' for Chrominance noise and 'weaker' for Luminance noise)
STF/HISTO - using a combination of STF and Histo (transferring settings back and forth), minimally clip the Dark end, and set a reasonably strong MTF, without being too severe

From now on the image is no longer LINEAR

Lum MASK - I will usually extract the Lu channel, and apply a Histo to it to give me a strong Lu Mask, to be used next to protect the DIM areas (I don't like the 'internal' HDRWT mask, as I can't 'see' it)
Masked HDRWT - now that the image is no longer linear, HDRWT has 'more range' to work with, and can thus be more effective - applied to Layer 3 (my experience, for my images)
Masked HDRWT#2 - as before, default values, except that it is applied to Layer 4
Masked HDRWT#3 - as before, default values, except that it is applied to Layer 5 (sometimes this is 'too harsh', and I need to decide which two layers to apply HDRWT to - experimentation usually required)
New Lum MASK - extract a mask to protect the brightest areas, using Lu channel, ATWT, Histo, etc - this is not an easy step to define, or explain, and it may even have to be applied earlier in the overall process as well
Masked HISTO/Lum-CURVE - (this may also have to be done earlier as well) here I would be trying to increase the brightness of the faintest regions, without blowing out brighter regions - this HAS to be 'subtle'
Masked Sat-CURVE - trying to bring out the colours of the dimmest areas - again, subtelty is important
Inv Masked Sat-CURVE - this time looking at colour in the brighter areas, remembering that over-saturation will look bad
SCNR - looking at the whole image, several small iterations may be needed to balance out coulour casts
ACDNR#2 - a recheck of ACDNR to make sure that no low-level noise has crept back in
HISTOclip - looking at the Black level, can the image be clipped without it looking too harsh? This is worked on in cooperation with the previous ACDNR step
RESAMPLE - because Meade DSI-II images have non-square pixels, at this stage I will stretch the X-axis by (8.6/8.3) to compensate for this phenomenon. Unfortunately this usually works AGAINST the fact that I may well have 'tracking smear' along the X-axis in any case, as a result of less than optimal auto-guiding (i.e. differential flexure), so this is always a step that I am happy to miss out!!

My DSI images never seem to benefit from either Deconv or MT processes, so I have given up trying to use these. I will revisit these processes once I upgrade my imager.

Hopefully others will add their thoughts.
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline Luigi

  • PixInsight Addict
  • ***
  • Posts: 140
Re: Pixinsight Workflow
« Reply #4 on: 2010 October 02 09:26:01 »
I will chime in here with my workflow, evolving though it may be.

<--- Also, it says 'newbie' over there for a reason :)

In my last processing 'run' I worked on an RGB image, so from the point where the 'channels' have been integrated, I did the following:

Linear:
Background Neutralization
Dynamic Crop if needed
DBE
Color Calibration
Histogram Stretch

Non-linear:
ATrous Wavelet (with a Star Mask)
ACDNR (with luminance mask)
Saturation Boost

I did notice in the 'non-linear' steps that without the masks my image was markedly inferior (looking at them side-by-side)

Anyway, this flow is a work-in-progress and I'm looking forward to seeing how others are processing.
Regards,
Luigi Marchesi

Offline DaveS

  • PixInsight Addict
  • ***
  • Posts: 115
    • Dave's Astronomy Pages
Re: Pixinsight Workflow
« Reply #5 on: 2010 October 02 09:28:25 »
Hi Niall

A very comprehensive reply indeed, so thanks for taking the time to document that.

I'm sure it will also be very useful to others new to the forum and PI, as well as me  :)

Dave
8" LX200ACF
William Optics FLT110
NEQ6 Mount
SXVF-H9
SXVR-16
SX Lodestar
DMK21AU04
Baader LRGB and NB filters
DiY Observatory
http://www.progressiveastroimaging.com/davesastronomy/

Offline DaveS

  • PixInsight Addict
  • ***
  • Posts: 115
    • Dave's Astronomy Pages
Re: Pixinsight Workflow
« Reply #6 on: 2010 October 02 09:30:40 »
Hi Loius

Well 'newbies' to PI are we both  :D, but your input is as valid as any, and all info is useful.

Thanks for 'posting'.

Dave
8" LX200ACF
William Optics FLT110
NEQ6 Mount
SXVF-H9
SXVR-16
SX Lodestar
DMK21AU04
Baader LRGB and NB filters
DiY Observatory
http://www.progressiveastroimaging.com/davesastronomy/

Offline Luigi

  • PixInsight Addict
  • ***
  • Posts: 140
Re: Pixinsight Workflow
« Reply #7 on: 2010 October 02 12:24:20 »
I'm wondering about why I'm using the DBE tool so 'late' and also why I'm doing ACDNR after I stretch. Hmmm.
Regards,
Luigi Marchesi

Offline Jack Harvey

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 975
    • PegasusAstronomy.com & Starshadows.com
Re: Pixinsight Workflow
« Reply #8 on: 2010 October 03 11:21:14 »
IMO ACDNR should always be near the end for  a couple of reasons I can think of  1.  Some of your processing will introduce some noise most likely.  If you do ACDNR early you have to do it again perhaps.  Using minimal evolutions of noise reduction is desirable.  Also near the end you can decide what is noise and what may be data in the final image and you most likely do not what to remove real data?
Jack Harvey, PTeam Member
Team Leader, SSRO/PROMPT Imaging Team, CTIO

Offline DaveS

  • PixInsight Addict
  • ***
  • Posts: 115
    • Dave's Astronomy Pages
Re: Pixinsight Workflow
« Reply #9 on: 2010 October 03 11:41:31 »
Valid points Jack, thanks.

Dave
8" LX200ACF
William Optics FLT110
NEQ6 Mount
SXVF-H9
SXVR-16
SX Lodestar
DMK21AU04
Baader LRGB and NB filters
DiY Observatory
http://www.progressiveastroimaging.com/davesastronomy/

Offline RBA

  • PixInsight Guru
  • ****
  • Posts: 511
    • DeepSkyColors
Re: Pixinsight Workflow
« Reply #10 on: 2010 October 04 03:02:43 »
Dynamic Crop - you need to eliminate 'rough edges' around the periphery of the image
STF - to let you see what you are dealing with (noting that this doesn't actually 'change' the image in any way)
Since you define when you do STF, shouldn't you do it also before the dynamic crop?  ;)

Definitely crop should be the very first thing, then DBE.

A couple of times I have seen images from "superstars" that in the final JPEG had one of those rough edges due to dithering, misalignment or whatever.
And no, I won't name names. All I'd say is that the images were praised in the public forums despite the edge was clearly visible.

And I wouldn't go too quickly on the second DBE. I know Juan says: "you can apply DBE twice, that's ok". And yeah, it's ok, but I'd rather spend some time perfecting the 1st background model than going for a second pass. Sometimes I generate well over 5-7 background models before I nail it. And by perfecting I mean trying really hard to nail it, not just sampling more here or there. Plus the more you do it, the better you become at it.

Anyway, why do I try to aim for just one DBE if possible? Because, to me at least, the best way to know a DBE has been successful is by looking at  the background model, even more than looking at the resulting image (both are important of course). If you're removing, say, a gradient, and the background model  has slight variations that you simply know that can't be the shape of your gradient (even considering that the gradient may have been building up during the capture as your target moves, etc), your model is wrong, and you can try to correct it adjusting the DBE parameters and adding/removing samples. Once you apply the first DBE, your only guide is the final image, not the model - because you no longer will be correcting a gradient but the remains of having partially corrected a gradient, and so your 2nd model can look like anything. If you're imaging a pretty galaxy or nebula and don't care about the background, this is probably ok, but if you're after very faint background stuff, you can no longer trust you're removing crap and preserving the right signal.

I usually do 1 or 2 DBEs on RGB data (when I do LRGB, which is most of the time), but with L I try to stay with just 1 DBE. The only exception would be dealing with a gradient so cabrón that I can "read" clearly what hasn't corrected, so in the 2nd DBE I aim only for that uncorrected area (something on an edge or stuff like that)...  I don't mind so much doing two DBE passes (if needed) with RGB for two reasons:

1) A RGB background model is not as easy to read, having technically three models in one (R, G and B). I could split it, but I don't go through that trouble in part because of reason #2:

2) It's "just" color, for my own stuff I'm often content with the fact that things get "colored" and my main goal is to see color gradients dissapear - faint signal isn't that important, for me, in the RGB data. If it was a "work for hire" I'd probably spend more time.

Anyway, this is how I approach this stuff and so far it seems to work well. Your mileage may vary.
Cheers,
Rogelio


« Last Edit: 2010 October 04 03:08:38 by RBA »

Offline RobF2

  • PixInsight Addict
  • ***
  • Posts: 189
  • Rob
    • Rob's Astropics
Re: Pixinsight Workflow
« Reply #11 on: 2010 October 04 04:21:12 »
Great thread.  Plenty of ideas here to try out during my next cloudy weekend.... ;D
FSQ106/8" Newt on NEQ6/HEQ5Pro via EQMOD | QHY9 | Guiding:  ZS80II/QHY5IIL | Canon 450D | DBK21 and other "stuff"
Rob's Astropics

Offline h0ughy

  • PixInsight Addict
  • ***
  • Posts: 226
Re: Pixinsight Workflow
« Reply #12 on: 2010 October 04 06:41:36 »
some great idea here and i get to learn more about the linear and non linear process
Doghouse Observatory

Offline Luigi

  • PixInsight Addict
  • ***
  • Posts: 140
Re: Pixinsight Workflow
« Reply #13 on: 2010 October 04 09:16:25 »
I agree, this is a great thread. If workflow has been discussed before it was before I joined.

<-- Now that I've been promoted, anything I write automatically carries more weight  :P

I did a quick combine of some M33 color data and I found the results to be better if I did DC, then BN, then DBE.

Regarding USM, has that been 'deprecated' in favor of wavelets?

The most difficult part of processing for me is the initial stretch. Is it OK to just import the parameters from the STF? Is it purely judgment?

Is it possible for DBE to 'clip' an image? I did a DBE on M33 and the histogram seems to be lopped off. Granted I did not do due diligence in my background modeling ... most of the frame is galaxy and points that are arguably not background were likely included in the model.
« Last Edit: 2010 October 04 09:29:44 by Star Dog »
Regards,
Luigi Marchesi

Offline DaveS

  • PixInsight Addict
  • ***
  • Posts: 115
    • Dave's Astronomy Pages
Re: Pixinsight Workflow
« Reply #14 on: 2010 October 04 12:35:47 »
Thanks guys, I couldn't have asked for more out of this 'thread'  8), but all the same, keep it coming, its all part of the 'learning curve' for us that are new to the product  :)

Dave
8" LX200ACF
William Optics FLT110
NEQ6 Mount
SXVF-H9
SXVR-16
SX Lodestar
DMK21AU04
Baader LRGB and NB filters
DiY Observatory
http://www.progressiveastroimaging.com/davesastronomy/