Author Topic: photometric colour calibration error  (Read 3684 times)

Offline ian straton

  • Newcomer
  • Posts: 17
photometric colour calibration error
« on: 2018 May 30 10:46:55 »
I am trying to colour calibrate an image, I have calibrated with bias, dark and flat images then stacked lights with Drizzle.  I have done a dynamic crop to get rid of the uneven edges.

Astrometry.net can plate solve the resulting image and I have entered these coordinates for the centre of the image, PI appears to plate solve ok and works away for some time cooking my CPU before spitting out the result:
Photometry process finished:
   3 of 3 images processed successfully.
   0 images with errors.

*** Error: Insufficient photometric data: Got 4 sample(s); at least 5 are required.

I used the scale correction described here:http://www.lightvortexastronomy.com/tutorial-colour-calibrating-images.html to take account of the different image scale produced by drizzling, (dividing the number of pixels on the x axis of the image (7843) by the physical resolution of my camera (5184) and multiplying by the focal length of my scope (1950)) which gave me a scaled fl of 2950mm but got the same error.   

I have placed a copy of the .xisf in my google drive if anyone wants to have a look at it: https://drive.google.com/open?id=1AYmvc_9K8DlCshFMDEzTZ-ROmVC8-oz0

As usual I would like to understand what I have done wrong to provoke the error so I can avoid it in the future!

Offline UlteriorModem

  • PixInsight Enthusiast
  • **
  • Posts: 78
Re: photometric colour calibration error
« Reply #1 on: 2018 May 30 15:16:02 »
Usually when I do Photometric calibration I will use a single unprocessed frame of the target and use the 'get data from image' button. Then apply that to the image I am processing.

Offline ian straton

  • Newcomer
  • Posts: 17
Re: photometric colour calibration error
« Reply #2 on: 2018 May 30 15:49:47 »
Usually when I do Photometric calibration I will use a single unprocessed frame of the target and use the 'get data from image' button. Then apply that to the image I am processing.

Thanks for the suggestion, the calibration worked on the single frame (I guess something in the drizzle & crop upset it it?) however I don't know which is the "get data from image" button, I have hovered over every clickable element of PCC and the workspace toolbars and can't find anything like this... Could you give a bit more detail on how I can reuse the data from this frame?

Offline RickS

  • PTeam Member
  • PixInsight Jedi
  • *****
  • Posts: 1298
Re: photometric colour calibration error
« Reply #3 on: 2018 May 30 16:54:28 »
I don't know which is the "get data from image" button, I have hovered over every clickable element of PCC and the workspace toolbars and can't find anything like this... Could you give a bit more detail on how I can reuse the data from this frame?

It's in the "Image Parameters" section.  The button is called "Acquire from Image."  Just select the image that is plate solved and then click the button.

Offline John_Gill

  • PixInsight Old Hand
  • ****
  • Posts: 368
Re: photometric colour calibration error
« Reply #4 on: 2018 May 31 00:03:30 »
Hi,

When using a focal reducer or Drizzle, then in PCC ---> Image Parameters ---> you need to either double the focal length or halve the pixel size.

Look up
John

APM 107/700 apo on CGX mount
ZWO Optics - Autoguiding
ZWO1600mm and filters
... when there are no clouds ...

Offline ian straton

  • Newcomer
  • Posts: 17
Re: photometric colour calibration error
« Reply #5 on: 2018 May 31 01:22:38 »
So having tried both the suggestions above I am still getting the same error.

The plate solving seems to be working fine, the issue is in the photometric data.  There are no errors reported during the entire process until it terminates with the error: Insufficient photometric data: Got 4 sample(s); at least 5 are required.

In contrast when the pcc succeeded on the single sub frame it terminated with:
Photometry process finished:
   3 of 3 images processed successfully.
   0 images with errors.

* Color transformation functions:
R-G = -1.951032e-01 + 9.199642e-01*(Sr-JV) ± 1.822317e-02
B-G = +2.069878e-01 + 3.784510e-01*(JB-JV) ± 2.971791e-02
* White balance factors:
W_R : 4.4447e-01
W_G : 6.4520e-01
W_B : 1.0000e+00
Applying white balance: done
Evaluating background reference: done
* Background reference:
B_R : 1.64275e-03
B_G : 1.64372e-03
B_B : 1.42321e-03
Applying background neutralization: done
02:10.82

So I guess the question is how/why is the stacked frame not producing the 5 photometric samples required? How do I control or respond to this error given that the plate solving appears to be working fine.

Offline msmythers

  • PTeam Member
  • PixInsight Jedi
  • *****
  • Posts: 1178
    • astrobin
Re: photometric colour calibration error
« Reply #6 on: 2018 May 31 03:41:53 »
I don't have a clue why I had to go through the steps I did for PCC to 'calibrate' this image but this is the only way I could get it to work. One more thing before I go through the steps, I don't like the results just to make that clear.

I did nothing else to the image like DBE, just the image as downloaded. Oh yeah, the labeling of the image is wrong. It is M82 not M81.

I could not get this to work without first plate solving with the ImageSolver script. Even if I matched the settings from ImageSolver with PCC. PCC would plate solve just not color calibrate. I don't understand that. I also had to change a number of settings like the Sensitivity, Noise Reduction and Distortion Correction in ImageSolver. These were not needed to plate solve but for PCC to calibrate without errors.

In PCC I had to Acquire the solving from the image. I also had to turn on the distortion correction even though I was not plate solving with PCC. And PCC did not show any plate solving happening according to the console.

I also had to turn off the Automatic limit magnitude setting and set the limit to 30. I also had to set the Saturation to 0.30

Attached are the settings and the results. I'll let others speculate on whats going on. Maybe using DBE before might help the result and or a run with BackgroundNeutralization afterwards. I just wanted to find a way for PCC to finish with no errors.



Mike

   

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: photometric colour calibration error
« Reply #7 on: 2018 May 31 10:47:37 »
imo, overall, the stars in this image are not of high quality - focus could be off and it does seem to suffer from coma (is this a newt?)

to start with a narrow FOV image of this target is an area of the sky with a dearth of stars, and then i think PCC is not detecting most of the fainter stars in the image because they are really soft and not super star-like. with so few detections the quality of the PCC result is going to be poor... perhaps if you did a pass of deconvolution first it could help.

another issue is that i think the brighter stars are probably saturated - in the red channel for sure, but maybe all channels. but because of the way DSLR images are handled in PI, the stars don't end up saturated at 1.0 but at some other (lower) level, in the vicinity of 0.25. if that happens to too many stars the PCC result will be bad as the true color of the star is lost - they don't "look" saturated to the tool as the values are not high enough. then again i'm not sure if PCC tries to exclude saturated stars but even if it did i don't think it would detect these stars as being saturated. anyway you'd have to turn on the "show detected stars" checkbox in PCC to see which stars it's actually using and try to figure out whether they are overexposed or not.

rob

Offline ian straton

  • Newcomer
  • Posts: 17
Re: photometric colour calibration error
« Reply #8 on: 2018 May 31 11:07:23 »
Rob, again to be clear:

The problem is not the plate solving.

The image plate solves just fine.

The problem is with the colour correction.

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: photometric colour calibration error
« Reply #9 on: 2018 May 31 11:10:43 »
it is a difficult image to solve due to the quality of the stars. it may or may not solve but it does not follow that this is all PCC needs to be successful. i would tick the "detected stars" in PCC and see what is happening. from mike's screenshot there are very few stars in the graph, so you can't expect a high quality regression.

if the stars PCC detected are overexposed, the color calibration will be wrong. GIGO.

rob

Offline ian straton

  • Newcomer
  • Posts: 17
Re: photometric colour calibration error
« Reply #10 on: 2018 June 01 01:38:13 »
Rob, Since the individual sub frames don't seem to suffer from the same problem as the stack is there any value in applying PCC to the sub frames before stacking?  Or taking the colour transformation data I got from the single sub and applying it to the stack manually?

Also you comments about the potential problems plate solving aren't really helping me to understand what the problem is.  You seem to be focussing on the part of the process that is working (when configured appropriately) while the failure is in the colour calibration phase.  I don't understand what the "photometric data samples" are and why it can't find one, perhaps I am missing the link?

Offline msmythers

  • PTeam Member
  • PixInsight Jedi
  • *****
  • Posts: 1178
    • astrobin
Re: photometric colour calibration error
« Reply #11 on: 2018 June 01 06:55:28 »
rob is not talking about the plate solving section of PCC. He is talking about the star detection process in the Photometry section. The first section does plate solving to define the area boundaries of the image. The photometry section has a completely separate star detection routine and uses the APASS catalog for star information specifically needed to color calibrate. The 'quality' of the detected stars in the image that are defined during this process and compared to APASS data. If the stars in the image are not detected due to an image quality problem for whatever reason then they cannot be used by the photometry section. That's my very basic understanding of the PCC photometry section.


Mike

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: photometric colour calibration error
« Reply #12 on: 2018 June 01 09:51:14 »
yes i agree with mike. PCC uses 'aperture photometry' as a default method, meaning it has to detect stars and then compute an aperture for each star over which to sum the flux. i'm saying that PCC might be having problems 1) detecting stars in the image (as evidenced by the low star count and how hard it is to solve this image) and 2) computing correct apertures for the detected stars due to their profiles.

the 'detected stars' map is going to show you what stars PCC identified for photometry. i think it's an important debug tool here - you can evaluate just how many stars it can detect, and which, to see if those stars have meaningful color data.

one possibility would be to perform deconvolution on the image in order to tighten up the stars, but you have to be careful there as the decon could saturate the centers of otherwise unsaturated stars, again perhaps throwing the PCC calculation off.

on the theory that you have saturated star data which has shown up as unsaturated pixels, it might make sense to run Rescale in individual channels mode on the image to get those saturated stars into the ~1.0 range. this is a longshot though because i am not sure if that really will cause PCC to exclude them. the heart of this problem is that DSLRs have 14-bit sensors and when PI reads this data in, it stuffs the data into the lower 14 bits of 16 bit integers. so saturated pixels in your original CR2 end up with a value of 0.25 or thereabouts in the 16-bit data. so the saturated pixels suddenly don't look saturated, but you can see that the stars are "flat" with your eye - the whole center of the star has a value of 0.25. what i am not sure about is if PCC can understand that the star data is no good in this situation, and so perhaps the rescaling would help it out.

rob

Offline ian straton

  • Newcomer
  • Posts: 17
Re: photometric colour calibration error
« Reply #13 on: 2018 June 03 12:42:09 »
Ah, Ok I see a bit more now.  Thanks for the explanations :)

I'm just back from holiday so I'll have another look in a day or two.

Is there any issue with applying pcc to subframes before stacking?

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: photometric colour calibration error
« Reply #14 on: 2018 June 03 21:50:22 »
i don't think there is an issue with running PCC on the input frames per-se, however, due to the scaling process during integration the relative weights between the channels in the input frames would probably be different from the relative weights in the output frame. i think there's not much choice but to run color calibration on the integrated frame if you want to get correct colors. you might try the older color calibration flow and see what happens.

rob