Author Topic: Correct values for DBE  (Read 14312 times)

Offline LorenzoM

  • Newcomer
  • Posts: 19
    • My pictures
Correct values for DBE
« on: 2012 February 05 14:15:30 »
Hi, i am using for the first time DBE tool. I have just corrected the image with flat but as you can see in the image some vignetting, or gradient, is remained at left and right of the image, there is also a darkest zone around M97
I apply DBE but the result is on the right, vignetting is not removed but just inverted, this process seems to be easy but why i am not able to reach a good result?
If you want to try here you can download the fit http://dl.dropbox.com/u/59547074/Public%20Fits/Raw.fit
Matter suggests to space as it needs to bend;
space returns the favor suggesting to matter as should moves
'John A.Wheeler 'Spacetime Physics'
http://www.flickr.com/photos/lorenzomassimi/

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Re: Correct values for DBE
« Reply #1 on: 2012 February 05 14:35:02 »
Here is what I usually do:
1. get your calibration right. You should not have this kind of vignetting if you do it correctly
2. if that fails: Use DBE iteratively.
2. a set tolerance to 3, set sample per row to 20, press generate
2. b set Correction to "Division" (since you are trying to compensate for problems in the flatness of your optical system=
2. c activate "Replace Target image"
3. Apply (green check mark)
4. Auto-STF
5. If you see "black holes", remove or relocate the sample point (in the screenshot, I would do this for the point below the "G" of "Gray")
6. Drag the blue triangle to the desctop (so you save your settings)
7. Close DBE
8. Undo last DBE
9. Open DBE by clicking on the saved settings
10. Continue with 3 until you are happy.

See screenshot.
Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline Warhen

  • PTeam Member
  • PixInsight Old Hand
  • ****
  • Posts: 490
    • Billions and Billions
Re: Correct values for DBE
« Reply #2 on: 2012 February 08 18:16:54 »
Georg,
           This is brilliant. I've been taking the subtraction route, but even w/ my sky flats I have gradients. Your method seems to work very well for a first pass, even better than subtraction- congrats!  Now, will you follow up w/ more traditional subtraction iterations after this main repair? If so, do you switch to the more common larger samples? Really Juan there seems to be so much variation in recommended sample sizes. Can you 'weigh in' on this please? Thanks guys!
« Last Edit: 2012 February 08 18:35:36 by Warhen »
Best always, Warren

Warren A. Keller
www.ip4ap.com

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: Correct values for DBE
« Reply #3 on: 2012 February 09 02:33:30 »
Nice strategy, Georg.

Another possibility is a two-step procedure with the ABE (AutomaticBackgroundExtractor) and DBE tools. ABE was designed and first implemented by Carlos Milovic in 2005, as a command-line standalone application. Then we ported it to PI as a standard module in 2007.

Contrarily to DBE, ABE generates background samples automatically, that is, you don't interact with the image to define samples. The other important difference between both tools is that ABE uses polynomial interpolation to generate the background model (computed with a least squares method), while DBE uses 2D surface splines (thin plates). Polynomials are much more rigid than thin plates, and this has advantages and disadvantages. The main ABE's advantage is that it's more robust and hence less prone to yield too incorrect results for wildly varying backgrounds. The main DBE's advantage is that it's much more flexible and hence able to model strong variations accurately. You know, you cannot have your cake and eat it too :)

For images with large free sky regions like this one, ABE usually works very well. This is a rather difficult case however, where we have irregular illumination caused by mixed multiplicative and additive effects. This is ABE working in just try samples mode:



In this mode, ABE only generates a 16-bit control image with background samples drawn as an overlay on the image. This is very useful to control the process and fine tune parameters. This tool has complete tooltip information (unfortunately no reference documentation yet).

This is ABE's model and the image corrected by division:



Note the relatively high polynomial degree (5), necessary in this case to model the strong variations in the background. ABE has overcorrected the image slightly in this case (bear in mind that the screenshot shows a heavily stretched image on the screen, thanks to STF). This is most likely due to the fact that the image combines multiplicative and additive effects. We could improve the correction by applying the model with PixelMath, multiplied by a correction factor < 1. However, instead of doing that we'll fix the residual irregular illumination with DBE:



This time the correction has been applied by subtraction. Given the severity of the initial problems, the result is rather good, in my opinion. However, as Georg says this should not happen if you improve your flat fielding technique. No software tool can replace the necessary precondition of accurate flat fielding.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Warhen

  • PTeam Member
  • PixInsight Old Hand
  • ****
  • Posts: 490
    • Billions and Billions
Re: Correct values for DBE
« Reply #4 on: 2012 February 09 06:43:25 »
Thanks JC,
                 ABE seems to do a nice job as well at subtraction, rather than DBE division as in Georg's nice method.

Now in DBE Juan, I see your example with hundreds of small smaples, but I see others using less/larger samples. What's best when and why? And surely when you place over 400 as you did, you aren't cycling through samples to be sure they're not on stars. So I'm assuming you trust the algorithm not to do much damage if a star is represented somewhat? Thanks!
Best always, Warren

Warren A. Keller
www.ip4ap.com

Offline Warhen

  • PTeam Member
  • PixInsight Old Hand
  • ****
  • Posts: 490
    • Billions and Billions
Re: Correct values for DBE
« Reply #5 on: 2012 February 22 08:53:19 »
Could someone answer that last one. So many varying number and size of samples used by many- what really is best for what application? I see sample sizes of 4-14 an 7-20 rows?!

How critical is it to move a sample accepted as green/good off of a star?

I've carefully avoided large stars' bright halos- why am I left w/ 'balck holes' and what can be doen about that?

Thanks guys!
Best always, Warren

Warren A. Keller
www.ip4ap.com

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Re: Correct values for DBE
« Reply #6 on: 2012 February 22 11:01:17 »
The numbers of samples depend on your image. If you have small gradients, few samples are sufficient. If they are strong (as in your case), you need more. Just experiment.
Regarding the black holes: remove the samples that cause the black holes, or decrease the tolerance value.
Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline Warhen

  • PTeam Member
  • PixInsight Old Hand
  • ****
  • Posts: 490
    • Billions and Billions
Re: Correct values for DBE
« Reply #7 on: 2012 February 22 15:17:42 »
Thank you Georg. I diskslike 'experimenting' as much as Harry loves it. ;>) Seems so darn arbitrary.  How about stars Georg- if you have many, many samples- how critical is making sure none contain stars, when they register green? It's soo much fun toggling through them all. :>)
Best always, Warren

Warren A. Keller
www.ip4ap.com

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Re: Correct values for DBE
« Reply #8 on: 2012 February 22 23:27:07 »
The problem just is that no image is like the other. So there is no single set of settings that works for all.

Regarding resilience to samples on/near stars: if you don't have large gradients in your image, the usual rejection mechanism usually works quite well. However, when I shoot my pictures near my home  town, the sky illumination gradients are so strong that I need to increase the tolerance value to get enough samples, but this also means that I need to avoid stars manually.
Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: Correct values for DBE
« Reply #9 on: 2012 February 23 02:29:25 »
Hi Warren,

Georg has explained everything important here. DBE is mostly about image statistics. It implements a robust rejection algorithm that knows how to gather background pixels avoiding outlier structures such as stars, hot and dead pixels, cosmic rays, etc.

However, as always happens when we are working with real world data, we have to deal with uncertainty. In this case, uncertainty arises when deciding if a given pixel belongs to the background or not. One of the most important lessons learned after some time involved in signal processing is that 'yes' and 'no' are both bad answers, in general, to this kind of questions. For this reason I implemented something more sophisticated in DBE: fuzzy logic. So we don't ask whether a pixel is part of the background or not; instead, we try to figure out how is it likely for a given pixel to be a background pixel. For each DBE sample, you can find an estimate of its probability of being representative of the true background of the image (whatever this happens to be) as the Wr, Wg and Wb values, respectively for the red, green and blue channels, where applicable.

Of course, this isn't a perfect system: in difficult cases (really wild gradients, complex-shaped gradients, dense star fields, etc.) we cannot remove uncertainty so easily, and then some manual verification work is required. Each particular gradient removal case tends to be a unique challenge. Feel free to upload one of the images where you're having trouble with DBE and we'll be glad to give it a try here, time permitting.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: Correct values for DBE
« Reply #10 on: 2012 February 23 02:47:49 »
Quote
I diskslike 'experimenting' as much as Harry loves it. ;>) Seems so darn arbitrary.

As you use PixInsight more and more, hopefully you'll change your mind on this. Experimentation is the mother of all image processing. Simplified solutions of the 'recipe' kind are its worst enemies because they are actually an obstacle to understand your images. Knowing what happens and understanding why things happen is the only way to find good solutions to actual problems, and this is only possible through experimentation.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Warhen

  • PTeam Member
  • PixInsight Old Hand
  • ****
  • Posts: 490
    • Billions and Billions
Re: Correct values for DBE
« Reply #11 on: 2012 February 23 11:59:41 »
"As you use PixInsight more and more, hopefully you'll change your mind on this. Experimentation is the mother of all image processing..."

Uh oh, now da boss man is quoting me! ;>) Juan you have to know a lot of things I say are 'tongue in cheek', spiced w/ a little sarcasm. I am loving PI.
Best always, Warren

Warren A. Keller
www.ip4ap.com

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1458
    • http://www.harrysastroshed.com
Re: Correct values for DBE
« Reply #12 on: 2012 February 23 12:09:25 »
Thank you Georg. I diskslike 'experimenting' as much as Harry loves it. ;>) Seems so darn arbitrary.  How about stars Georg- if you have many, many samples- how critical is making sure none contain stars, when they register green? It's soo much fun toggling through them all. :>)

Hi

DBE is very good at ignoring small stars  :-*  Its when Large stars with halos become involved there is a problem DBE might see the halo ( however faint ) as a background gradient  :surprised:  so keep clear of  these to avoid the black holes  8)

If you are lucky enough just have a smooth gradient you can use a few as say 20 samples , but difficult gradients might require hundreds

So my friend embrace experimentation you will come to love it  8)

Regards Harry
Harry Page

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
Re: Correct values for DBE
« Reply #13 on: 2012 February 23 13:01:08 »
Really like the ABE followed DBE example.  Never thought of that.

Thanks

Max