Author Topic: Linear Data Processes  (Read 8571 times)

Offline sreilly

  • PixInsight Padawan
  • ****
  • Posts: 791
    • Imaging at Dogwood Ridge Observatory
Linear Data Processes
« on: 2011 January 04 10:34:10 »
Over the course of reading and viewing tutorials on using PI, as well as other processing software, I repeatedly see the term linear data. What I don't see or have is a firm grip on the definition. I've fallen into a fairly routine imaging process using PI where the data is calibrated and saved separately from the raw data. Then I star align all the RGB images, save again as aligned, combine the data using image integration creating masters. Save the master frames and then combine the RGB data. The process is similar with the luminance data. After the masters are created I'll use the screen transfer function to see if it needs cropping using the dynamic crop tool, use the DBE tool to correct any background gradients the image may have and save again. Now I'll use the histogram tool to get the hidden data in the image. At this point if I'm working on a galaxy image I'll usually make a star mask using the star mask tool, apply the mask, invert the mask, and then use HDRWavelets tool to enhance the galaxy details. When I'm happy with that I'll use the deconvolution tool and usually do 5 iteration of regularized Lucy Richardson deconvolution. Remove the mask and save the resulting image. Actually I usually save the image after each tool is used in case I later decide I don't like the effect.

All this leads up to this basic question. When did I no longer have linear data? What processes should be used on linear data? As an aside, I noticed that masks that Harry created in his tutorials don't have the red color mine do. Is there a reason for this?

As Always, Thanks
Steve
www.astral-imaging.com
AP1200
OGS 12.5" RC
Tak FSQ-106ED
ST10XME/CFW8/AO8
STL-11000M/FW8/AO-L
Pyxis 3" Rotator
Baader LRGBHa Filters
PixInsight/MaxIm/ACP/Registar/Mira AP/PS CS5

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: Linear Data Processes
« Reply #1 on: 2011 January 04 10:51:13 »
Steve,

the definition of a linear image is simply that the brightness of a pixel is proportional to the amount of light that was gathered by it. So if twice the light had fallen on that pixel, it would have had twice the value. As soon as you do a non-linear transform such as a histogram transform with a midpoint that's not in the middle the image is no longer linear.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Simon Hicks

  • PixInsight Old Hand
  • ****
  • Posts: 333
Re: Linear Data Processes
« Reply #2 on: 2011 January 04 10:52:41 »
The mask colour bit is the easiest to answer....go to Mask / Rendering Mode and select Mulitply....and the mask will be in greyscale. This is how I like it.

Data is generally linear up until the Histogram Stretch. So all the calibration, stacking, cropping, etc is keeping the data linear. As soon as you change the luminance relationship between the pixels then you've essentially gone non-linear.

However, the definition is not to rigorous. You are usually advised to do Deconvolution on linear data....but this changes the linearity slightly...but its usually still regarded as linear after a deconvolution...its a case of extent. And the same is true of Background subtraction.

The STF keeps the data linear but just changes the look of the screen. But a Histogram or curves stretch is definitely a transition to non-linear.

Hope that helps.

Cheers
         Simon

Offline sreilly

  • PixInsight Padawan
  • ****
  • Posts: 791
    • Imaging at Dogwood Ridge Observatory
Re: Linear Data Processes
« Reply #3 on: 2011 January 04 11:52:19 »
So with my work flow process as stated, when and how should the deconvolution be done? I ask how in so much that if the histogram stretch isn't done the only other way I see of seeing the effects would be using STF. Should HDR Wavelets also be done on linear data?

Thanks,
Steve
www.astral-imaging.com
AP1200
OGS 12.5" RC
Tak FSQ-106ED
ST10XME/CFW8/AO8
STL-11000M/FW8/AO-L
Pyxis 3" Rotator
Baader LRGBHa Filters
PixInsight/MaxIm/ACP/Registar/Mira AP/PS CS5

Offline Philip de Louraille

  • PixInsight Addict
  • ***
  • Posts: 289
Re: Linear Data Processes
« Reply #4 on: 2011 January 04 14:16:03 »
Deconvolution ought to be done before your histogram stretch ... but ... there are rare cases (I'm told) where you can do a decon later in the flow.
Some folks, actually, recommend that you do a wavelet transform rather than a decon. Again, I'm sure there are cases where a decon is better but those would be rare.
Philip de Louraille

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1458
    • http://www.harrysastroshed.com
Re: Linear Data Processes
« Reply #5 on: 2011 January 04 14:16:16 »
Hi

Decon is done mostley at the linear stage so you need the STF to see the results  ;D  But HDR wavlets will only work after the image has been stretched ( ie not linear )

clear as mud  ;D


Harry
Harry Page

Offline Philip de Louraille

  • PixInsight Addict
  • ***
  • Posts: 289
Re: Linear Data Processes
« Reply #6 on: 2011 January 04 14:30:32 »
Reviewing my notes, the À Trous wavelet transform is used right before you stretch the histogram (it can be used as a denoising/decon tool)

After the histogram stretch, you can use the HDRW transform.
Philip de Louraille

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: Linear Data Processes
« Reply #7 on: 2011 January 04 14:36:49 »
Yep.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline sreilly

  • PixInsight Padawan
  • ****
  • Posts: 791
    • Imaging at Dogwood Ridge Observatory
Re: Linear Data Processes
« Reply #8 on: 2011 January 04 14:43:47 »
I haven't seen anything on using the À Trous wavelet transform tool. A quick look and trying several things leaves me with nothing useful to use that I can see so some guideline on using the tool would be dandy. I also tried deconvolution using the combined luminance image with only DBE and the image cropped. Again, really didn't see any resulting image I could use. Obviously I'm missing some important steps or settings here. Anyone know of a video or tutorial on these tools?

Thanks,
Steve
www.astral-imaging.com
AP1200
OGS 12.5" RC
Tak FSQ-106ED
ST10XME/CFW8/AO8
STL-11000M/FW8/AO-L
Pyxis 3" Rotator
Baader LRGBHa Filters
PixInsight/MaxIm/ACP/Registar/Mira AP/PS CS5

Offline sreilly

  • PixInsight Padawan
  • ****
  • Posts: 791
    • Imaging at Dogwood Ridge Observatory
Re: Linear Data Processes
« Reply #9 on: 2011 January 04 14:46:05 »
I'd be happy to make available a luminance image on my website in FITS format if you would care to try these methods on the image and post the steps taken. ;D
Steve
www.astral-imaging.com
AP1200
OGS 12.5" RC
Tak FSQ-106ED
ST10XME/CFW8/AO8
STL-11000M/FW8/AO-L
Pyxis 3" Rotator
Baader LRGBHa Filters
PixInsight/MaxIm/ACP/Registar/Mira AP/PS CS5

Offline Ginge

  • PixInsight Addict
  • ***
  • Posts: 215
Re: Linear Data Processes
« Reply #10 on: 2011 January 04 14:50:48 »
Hi there!

Why exactly is the use of deconvolution on non-linear material not recommended?

Ginge

Offline DaveS

  • PixInsight Addict
  • ***
  • Posts: 115
    • Dave's Astronomy Pages
Re: Linear Data Processes
« Reply #11 on: 2011 January 04 14:53:35 »
Being fairly new to PI, and finding my way around it by experimentation, due to the absence of documenation, I have to say that I find the decon 'tool' quite effective.

The AT Wavelet 'tool' however, leaves me wondering what it actually does, as I've yet to see an effect from it.  ???

I accept that is maybe because there's nothing to tell me what it is supposed to do, and I have perhaps been using it incorrectly.

Dave
8" LX200ACF
William Optics FLT110
NEQ6 Mount
SXVF-H9
SXVR-16
SX Lodestar
DMK21AU04
Baader LRGB and NB filters
DiY Observatory
http://www.progressiveastroimaging.com/davesastronomy/

Offline Simon Hicks

  • PixInsight Old Hand
  • ****
  • Posts: 333
Re: Linear Data Processes
« Reply #12 on: 2011 January 04 15:09:46 »
The ATW tool is probably one of the most useful tools in PI...to me anyway. I use it to sharpen, to smooth, to create star masks, to modify star masks, to create nebula/galaxy masks, to emphasise features. So it gets used with just about all the other tools. If people are not using it then maybe PI needs a video tutorial on it  ;)

Offline sreilly

  • PixInsight Padawan
  • ****
  • Posts: 791
    • Imaging at Dogwood Ridge Observatory
Re: Linear Data Processes
« Reply #13 on: 2011 January 04 15:28:24 »
Well if your willing to download a 23.7 MB file, here's the M82 luminance 13 hour combined image. It's been combined and cropped and is 64 bit depth. http://www.astral-imaging.com/M%2082-46-L_Cropweb.fit
If anyone does try this please let the rest of know what steps and settings were used to get the resulting image. I would have to say that an in depth video/tutorial of this tool would be quite useful.

Thanks,
Steve
www.astral-imaging.com
AP1200
OGS 12.5" RC
Tak FSQ-106ED
ST10XME/CFW8/AO8
STL-11000M/FW8/AO-L
Pyxis 3" Rotator
Baader LRGBHa Filters
PixInsight/MaxIm/ACP/Registar/Mira AP/PS CS5

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: Linear Data Processes
« Reply #14 on: 2011 January 04 16:20:17 »
Well I thought I understood ATWT but I can't make it do anything useful on this image :) When I was preparing for my MWAIC talk I had a section about ATWT prepared but had a hard time building a coherent story around it so I skipped it. The only thing I've been able to do with it is some noise reduction and layer extraction. No sharpening of a linear image.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity