Author Topic: Managment of "synthetic luminance" and RGB questions (NOW MOSTLY DBE !!)  (Read 20074 times)

Offline dhalliday

  • PixInsight Old Hand
  • ****
  • Posts: 307
    • on Flickr
Here are the file links;
http://www.sendspace.com/file/vimiyq (blue)
http://www.sendspace.com/file/sxkzw7 (green)
http://www.sendspace.com/file/5a8edq (red)

About a 5% chance I got that right...
They are the "registered files"...
Hope that will not screw other non Pix users up too muchh...will it ??
They are 32 bit FITs files...

Hope to see what anyone else can do,and maybe learn some...
Harry,Niall ...out there ??
 >:D
Dave Halliday
8" Newtonian/Vixen VC200L/ TV 101,etc etc
SSAG/EQ6
CGE Pro
SBIG ST2K,ST10XME

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
I really think that you should update your PI version. There are a lot of bug fixes, and you are missing the new tools! :)
Use the wizard to unistall your current version, and then install the new one. Reset all configurations the first time you run it, and you are done (it should ask to do that, the first time is launched).

Indeed, normalization should avoid any clipping at the left of the histogram. After that, you should set the black point as you like it (either using HistogramTransform or AutoHistogram).

I think that the best way to keep everyone happy would be to upload the masters as 32bits floating point FITS (if you generated 32bits masters after integration of lights), or 16bits unsigned TIFF. Then compress the file with zip format (or rar format). There are dozens of free file hosting servers, like rapidshare, megaupload, etc.

You are right. Now the galaxy looks deeper :)

PS: I'll need that survival pack... the chances are that I'll end in Lost's island :P
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Oh, you already uploaded the files :P
I'll take a look at them next weekend. I want to install a new OS here, so I'll need a few days to get everything working again :P
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline dhalliday

  • PixInsight Old Hand
  • ****
  • Posts: 307
    • on Flickr
Carlos
Upgrade complete...!
Boy..am I ever getting savvy...
A little tricky...previews seem a bit hard to draw...
I will practice...look forward to a pro's asses of M101...

Normalize...its a good thing >:D

Dave

PS.."OS"...?
Operating system...?
That would take me a month,and 6 phone calls to my nephew...
Dave Halliday
8" Newtonian/Vixen VC200L/ TV 101,etc etc
SSAG/EQ6
CGE Pro
SBIG ST2K,ST10XME

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Hi Dave - sorry about some 'tardy' replies. I have actually been trying to calibrate some M82 data of my own (full PI 'manual' calibrate, i.e. 'my way') of R+G+B+Lu+Ha subs taken over the last few weeks, working around some scabby skies, a blown AutoStar, a failed On/Off switch on the LX90, and the swap of my TEC cooler from the DSI-IIC to my DSI-IIPro - oh, and the occasional volcano.

I really want to get to the bottom of my own data, so that I can then turn to the new ImageCalibration process and see how well it compares - in terms of convenience and outcome.

I will keep an eye on your poasts, and if there is something 'obvious' that I can post, I'll get back to you.

Cheers,
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline dhalliday

  • PixInsight Old Hand
  • ****
  • Posts: 307
    • on Flickr
Niall
No problem man...

I have gotten quite far here with this disjointed discussion...
Even have the latest version of this "hip" program...
I am a "Pixinsight cool cat"... O0

Boy I keep "running into the wall" of people getting all "chuffed" (? antichuffed ?) when we can't "talk photoshop" about an image...

Feel a bit like R Crusoe... :'(

Anyways...I am (?) hours away from (attempting) to align the new mount... >:D

I have tried the "Image integration" many times...
I don't know...DSS,with my "special (secret) settings" STILL seems its equal...sorry to say..
{As you know its straight (blather) only from me...}

cheers,good luck with the volcano,and POST M 82 whenever..!!

Dave
Dave Halliday
8" Newtonian/Vixen VC200L/ TV 101,etc etc
SSAG/EQ6
CGE Pro
SBIG ST2K,ST10XME

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
Re: Managment of "synthetic luminance" and RGB questions
« Reply #36 on: 2010 April 20 06:38:30 »
1) I usually increase a lot the Threshold. I rather like to include some "bad" pixel than having poor statistical samples. Also, what he thinks to be the expected background value may not match my opinion :)
In 99% of cases I add samples manually. If it were so easy to put automatic samples, I would use ABE :D At least for a first iteration.
2) I don't touch that parameter, but in very rare situations:
a) If you are not sure if the samples are placed over true background sky, or it shows small variations from noise/big stars/etc, it is better to increase a bit the smoothness of the model.
b) If you have few background samples, or there are strong variations, decrease the smoothness to create a more "accurate" model. Also take a look to the weight of the samples.
3) You are able to create a sample and manually set the pixel value. This may be very tricky, but in very difficult cases may be the only way to incorporate a meaningful sample. Before digging into this, try any symmetries that may cover that area.
Trick: Let's assume that you can place samples on a very narrow space, in one quarter of the image (for example, you are following a dust line in the milky way's core). If there is an obvious radial pattern in the gradient, create radial symmetries with different amount of "sides", specially along contiguous samples. For example, use 7 for one of them, and 8 with the other. This will create a richer sample field, that will avoid strange radial patterns. Also you may increase the smoothness.

Another trick: use a VERY aggressive STF to inspect the results. Make sure you are adding the pedestals (and not rescaling) so the mean flux of the sky is preserved. Use previews with PixelMath to quickly inspect the image, "blinking" between the result and the original image.


Yes, there is a lot to learn :) You have to give it time, and experiment with the parameters. This is the only way to gain experience and get the "feeling" where to go next. That intuition is based more on trial/error than theoretical knowledge (that helps too, of course), since nothing is better than "seeing" what actually happens. The same is true with almost every process in PI :)

I alway have to raise from the default threshold even on fairly flat images otherwise I can't sample the darker areas.
I am not how this setting helps define the background.

The only trick I have learned is to make sure no stars or object data is in the sample otherwise it results in a dark area in the DBE image.
 The sample box area often shows the stars darked. I thought this means the stars is rejected but it does not result in a clean background sample. I just manually move points off objects.

On colored images I find the sample box harder to interpret. I am not sure what the colors mean.

I try to add more samples to transitional areas this helps too.

What does the weighting box do?


I have some image where DBE just does not help much.

Max
Max

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Hi Max
Try increasing more the shadows relaxation parameter. It should include more dark pixels, whithout including bright features. This way you avoid contributions from stars.

Small stars are handled fine by the algorithm, but it is better to increase the size of the samples, so it has more background pixels to average, and thus the statistic is more representative. Object data, of course, destroys the model. In case of doubt, don't put a sample! :D
Yes, darker pixels in the sample "preview" means that they have less weight for the statistical calculation. Black means no inclusion at all. Other colors are just a mix, showing the weights at each channel. It is quite difficult to evaluate anyway... so just imagine the luminance of them, to get an idea.
The samples are likelly not to be "clean", pure white. This happens 'cause there are two rejection algorithms at work. One is local, witch rejects stars, noise, etc., and the other is global. The global parameter supposes a very simple model (I'm not sure if it is a constant number, like the median of the image, or it makes a quick interpolation from a few samples) and rejects sample boxes that are too far away from it, creating a weight factor. The closer it is to the model, more it weights. After all, real background models are very smooth, so wild values usually means a bad sample.

Look, from the unoficial guide:
Quote
Wr, Wg, Wb: Statistical sample weight for the red, green and blue channel respectively. A weight value of one means that the current sample is fully representative of the image background at the sample's location. A value of zero indicates that the sample will be ignored to model the background, since it has no pixels pertaining to the background. Intermediate weights correspond to the probability of a sample to represent the background of the image at its current location.
And the unweighted checkbox in the model parameters:
Quote
Unweighted: By selecting this option, all statistical sample weights will be ignored (actually, all of them will be considered as having a value of one, regardless of their actual values). This can be useful in difficult or unusual cases, where DBE's automatic pixel rejection algorithms may fail due to too wild gradients. In such cases, you can manually define a (usually quite reduced) set of samples on strategic locations and tell the background modeling routines that you know what you're doing – if you select this option, they will trust you.

Hope this helps. ;) If you need assistance with those rebel images, let us know ;)
(resistance is futile) VBG


Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline dhalliday

  • PixInsight Old Hand
  • ****
  • Posts: 307
    • on Flickr
Group
I would like to CONTINUE this discussion...
Could one of the "rebel/difficult" images be posted and maybe discussed further,...
When I am MANUALLY applying samples..(and THANKS..larger work better than the "default" size..)...

When I apply them I see a "picture" of them...all those wee pixels "pop up"...sort of a screen shot...
I always look at that and go "WOW"...hope that is being "dealt with" by Pixinsight... >:D
But I guess I should be using that image to position/abandon a given sample..?
So I read what is posted here today...and maybe I am just tired,but still confused about it,some....


What SHOULD I be "looking for" to tell me what is a good/bad sample..?
And presumably in grey scale its different.This is wher having an image to focus on would help me...

PS
Here is the group laugh for the day...
I just spent two ulcer inducing nites trying (unsuccessfully) to calibrate a new mount...
and just realized this AM I have the DEC/RA cables swapped... >:D

I hope to hear more/see more on sampling.

Dave
« Last Edit: 2010 April 20 09:16:00 by dhalliday »
Dave Halliday
8" Newtonian/Vixen VC200L/ TV 101,etc etc
SSAG/EQ6
CGE Pro
SBIG ST2K,ST10XME

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Quote
I just spent two ulcer inducing nites trying (unsuccessfully) to calibrate a new mount...
and just realized this AM I have the DEC/RA cables swapped...

Dave, your mount is faulty - trust me. And you will also find that I have arranged for it to now be 'out of warranty', because you have obviously been 'tampering with it'.

However, all is not lost. I have arranged for it to be 'recycled' - as soon as transatlantic flights schedules recommence. My mount is NOT faulty, so, you can have it as a straight swap - providing you are willing to pay the postage.

In the meantime, I will call the local (Clinterty) coke-can factory and arrange for them to take delivery. Honest.

Beleive me - you will be FAR happier, and your ulcer will trouble you no more :angel:

Cheers,
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline dhalliday

  • PixInsight Old Hand
  • ****
  • Posts: 307
    • on Flickr
Niall
Not sure about where the "coke can factory" ties into that offer...
(OK...now I got it...sleep deprived lately...)
Sly dog you...

Are you suggesting to recycle my mount into Coke cans... >:D
250 pounds worth...?
(rubs it in...)

Hopefully it is going to be "up to speed" soon... ;D
Really quite a story what this volcano has done,specifically to UK,European economy etc.

Listen,where do I go to see what you are up to/processing etc..?
I "wear my processing heart on my sleeve" so to speak ......in Flickr...for 30$/year.
Where is everyone "hanging out"...?
Another forum in Pixinsight..??
I'me lonely :'(

Dave[
Dave Halliday
8" Newtonian/Vixen VC200L/ TV 101,etc etc
SSAG/EQ6
CGE Pro
SBIG ST2K,ST10XME