Software Binning and WBPP

[fredvanner]

That's a great diagram!

Thanks. Sometimes a picture really is worth a thousand words!
The key point is that the shape of the profile has nothing to do with the star - it is entirely an artefact of the observing equipment and conditions, and in general is linear in the actual star brightness.

 

[StuartT]

FWHMEccentricity can output results to a file.

not quite sure which column to look at, but I assume the "median FWHM px" - in which case I seem to have a big variation in FWHM

 

[fredvanner]

not quite sure which column to look at

I remember this now! The CSV is a bit of a mess, so the headers don't line up properly when imported into Excel. This means both the summary statistics and the tabulated data are misaligned. So your Median FWHM is 6.86 pix, with a MAD of 0.8 pix. The equivalent values in the table are the sx and sy values (again misaligned by one column). These are the sigma values (standard deviation) in x and y. For a normal distribution the FWHM is about 2.355 sigma, so your median FWHM should be equivalent to a sigma of about 2.9. Looking at the tabulated data, the stars appear to have sx in the range 9..16 and sy in 9..10 (excluding outliers)...
... so the values really don't seem to match! I don't know what to make of that result - but I'm no expert on FWHMEccentricity.
A sample from one of my rather average images gives:

The value in red is the sigma calculated from my median FWHM, which looks a reasonable match for my sx and sy values.
Maybe using DynamicPSF, where you can see the data for a selected star, migh clarify this a bit.

 

[fredvanner]

OK, I think I know what the problem is. By default I always use the Gaussian model for my PSF analysis. You were using the Moffat6 model, which has the quite different FWHM scaling of FWHM = 0.7 sigma, so your median FWHM of 6.86 corresponds to a sigma of about 9.8 - in rough agreement with the tabular data.

 

[cloudbait]

ok, thanks. I can't seem to find a tool to measure the PSF of individual stars in an image. Is there a script?

I normally use DynamicPSF and click around the image on a few "average" stars (isolated, not saturated, not nearly lost in the noise). You can then examine the FWHM individually. You should see pretty similar values most of the time. Usually what I see will be pretty close to the median value reported by FWHMEccentricity, which looks at all the stars within the detection limits.

 

[fredvanner]

A nice feature of DynamicPSF is that you can calculate several different models for each star you look at. By looking the final (MAD) column you can assess which model is fitting your image best. This may help, for example, in picking a PSF model for deconvolution.

 

[StuartT]

Thanks for this. So are you saying that second row (starting with "Target" is the mean of the values in the column below?

Can I ask what you mean by sigma and MAD?

 

[fredvanner]

If you compare your screenshot with mine you will see that I have:

• removed the first four rows of summary data.
• removed the first word, "Header" from the fifth and seventh lines.
• this can be done in any text editor (you must remove "Header," - note the comma).

As result, the columns in my table align correctly. The resulting structure is actually two tables, one after the other:

• one row of headers (starting with the word "target"), listing summary data parameters.
• one row of summary data values (starting with the word "target"), giving the values of these parameters.
• one row of headers listing the star description parameters.
• a number of rows listing the parameters of individual stars.

The abbreviation "MAD" describes a number of related statistical measures; it can stand for "Mean Absolute Deviation" or "Median Absolute Deviation"; in either case this may be taken as deviation from the mean, or deviation from the median. In PixInsight I believe it always means "Median Absolute Deviation (from the median)".
Sigma - conventionally written as the small greek letter σ - is standard statistical terminology for the standard deviation (of a collection of data) - the square root of the variance (look up on Wiki if you want deeper info).
So in the summary data table, you will find the value "6.86E+00" aligned beneath the heading "Median FWHM px", indicating that your Median FWHM value (over the set of 115 stars measured) is 6.86 pixels. Later on the same line, the "MAD FWHM px" heading has the value 8.08E-01 = 0.808.
This tells you that the majority of the stars are less than one pixel away from the median FWHM value.
The table of individual star data does not give FWHM values. The nearest measures are the sx and sy columns. The x and y columns give the (x,y) coordinates of the centre of the star, and sx and sy give a measure of the "spread" of the star around this centre. For a Gaussian model, these are the standard deviations of the star data from the centre, in pixels. In the Moffat models they are an equivalent measure of spread. This is described in more detail in the documentation for DynamicPSF.
To estimate the FWHM from the sx and sy values in this table you need to use the relationships:

Where (G) is for Gaussian models, and (M) is for Moffat models, in which case β is the Moffat parameter (so β=6 for Moffat6).

 

[fredvanner]

As a simple aid:
FWHM = k * σ
where k is taken from this table:

(β = 1 is described as "Lorentzian" in DynamicPSF; β = 1.5, β = 2.5 are described as Moffat15 and Moffat25 in DynamicPSF)

 

[cloudbait]

The table of individual star data does not give FWHM values...

It is worth noting that in many or most cases we may not care about "true" FWHM, in any mathematical sense. If the goal is simply a quick assessment of seeing, or a comparison between images or regions of an image, a quick glance at the sx and sy values and knowledge of our imaging systems is usually sufficient. We can quickly learn that a value of 4 means it's a good night and a value of 8 means we might want to find another activity.

 

[fredvanner]

If you read the original paper by Moffat [1969], or the earlier paper by Gyldenkaerne [1950] to which it refers, the distribution due to atmospheric perturbation (seeing) is assumed to be Gaussian (based on yet earlier results by Danjon and Coude [1935] and Strömgren [1945]). The Moffat non-gaussian rational power function is mainly derived for analysis of scattering and absorption in photographic emulsions of finite depth - which has no relevance to modern CCD and CMOS cameras. The Moffat functions can still be used as abstract parametric models, but without any clear physical basis. I find that gaussian models are a good fit to most of my images. The exception tends to be undersampled wide field images where the FWHM is so small that stars are probably not sufficiently (photometrically) sampled.

Bottom line: Median FWHM from FWHMEccentricity with "Gaussian" selected is a pretty robust quantitative estimate of seeing.

 

[StuartT]

Thank you very much for going to the trouble to explain all this. You are very patient and I appreciate it. I can see now that my excel sheet had a frameshift error and once I removed the offending cells, it was fine.

So to estimate the seeing in arcsec I just multiply the FWHM in pixels by my image scale (for the image above that was 0.33"/px) so my seeing is 2.26"

Got there in the end. I have learned a lot!

 

[fredvanner]

and sx and sy give a measure of the "spread" of the star around this centre.

One last note, for anyone looking at this in detail. The "sx" and "sy" are not sigmas in the image (x,y) coordinates; they are in eliptical "PSF" coordinates, with x along the major axis and y along the minor axis. See the (increasingly out-of-date) reference documentation of DynamicPSF for details.

 

[Cygnusx1]

Hello, I’ve been going through this discussion with a lot of interest. I’m trying to analyze/evaluate the real-world effects of down-sampling un-binned image data using PI’s IntegerResample. I am using an ASI 2600MM with 3.76 micron pixel. I’m also using a C14EHD with .7 focal reducer so the OTA has a FL of 2738mm which gives a pixel resolution of .28”/px. Way over sampled for any of my typical seeing conditions. File size doesn’t concern me (maybe download speed…) and I don’t like losing any data if I don’t have a good reason.

As is SOP for me now I color calibrate all images with SPCC preceded with ImageSolver for an Astrometric solution. When I take an un-binned frame with .28”/px resolution it’s a dry day in California that it finds a solution (tried flushing the script to default values, hardwired parameters …). It always pulls the right metadata from the FITS info, and still fails.

But, when I do a 2x2 IntegerResample it solves the same (downsampled) image everytime. I can’t figure it out unless it’s just a scale thing. Last night I was able to solve the un-binned frame, but not anymore. Maybe it was the humidity.

Anybody have any thoughts about what might be going on?

 

[fredvanner]

Could you upload a problem image somewhere and post a link?

 

[Cygnusx1]

Hello, here's a frame (Ha) that has been WBPP'ed including auto cropped. It is the only frame out of all the LRGB Ha and S2 that I was able to solve. What's even 'curiouser ' is that I can't solve it twice. This is an IntegerResampled (unbinned to 2x2) image. I ran all the frames (~100) through Image and Process Containers. None but the Ha solved and it was one of the poorest S/N (and S2). I figure I'm missing something in how the Solver script works. I'd love to find some documentation. Any thoughts very much appreciated. Thanks. Here's the link : https://drive.google.com/file/d/17K6Ah3FWIWi5VYOwX4aLCmlQeo3yPlKT/view?usp=sharing

 

[cloudbait]

Hello, here's a frame (Ha) that has been WBPP'ed including auto cropped. It is the only frame out of all the LRGB Ha and S2 that I was able to solve. What's even 'curiouser ' is that I can't solve it twice. This is an IntegerResampled (unbinned to 2x2) image. I ran all the frames (~100) through Image and Process Containers. None but the Ha solved and it was one of the poorest S/N (and S2). I figure I'm missing something in how the Solver script works. I'd love to find some documentation. Any thoughts very much appreciated. Thanks. Here's the link : https://drive.google.com/file/d/17K6Ah3FWIWi5VYOwX4aLCmlQeo3yPlKT/view?usp=sharing

It solved fine for me, but it's a very marginal image with so few stars. It solved using 25 image stars tested against 800 Gaia stars. If I were imaging this target and had put together enough imaging time that I was ready to process, I'd expect hundreds of stars to be present.

 

[fredvanner]

It solved using 25 image stars tested against 800 Gaia stars.

Unsurprisingly, I get the same results.
I note that most of the brighter features are only about 2x - 3x the background level. I would have exepected more signal from 7 x 240s H-a. What camera is it?

here's a frame (Ha) that has been WBPP'ed including auto cropped. It is the only frame out of all the LRGB Ha and S2 that I was able to solve.

I assume the header history is correct; this is an integration of seven frames.
I'm slightly confused by the terminology "unbinned to 2x2"; 2x2 downsampling is "binning", not "unbinning".

 

[cloudbait]

Unsurprisingly, I get the same results.
I note that most of the brighter features are only about 2x - 3x the background level. I would have exepected more signal from 7 x 240s H-a. What camera is it?

I assume the header history is correct; this is an integration of seven frames.
I'm slightly confused by the terminology "unbinned to 2x2"; 2x2 downsampling is "binning", not "unbinning".

For comparison, here is a single 300 second red exposure of M101 I made a couple of years ago. It's raw- no calibration or anything, just a screen stretch. It's a little bit smaller field than the one under discussion, and it solved using 74 stars filtered from 151 valid PSF fitted stars and matched to 617 Gaia stars in the field. Obviously, an Ha image will have significantly attenuated stars compared with my broader red one, but still... yeah, more signal needed. (And I'd be more concerned with why the LRGB images didn't solve, where there should have been many more stars. Ha may not be the best test case for this failure.)

 

[Cygnusx1]

Hello. Excuse my terminology. By un-binning I meant its 1x1. By downsampling I meant I Integerresampled it to 2x2. My point about this frame was that it was by far the worst frame integrated (certainly S/N-wise) and I could still image solve it. Just for checking I tried to image solve this same (Ha) frame a second time and couldn't.
Here is an integrated RGB frame of NGC5457 (the Pinwheel).

RGB_NGC5457.xisf

drive.google.com

I cannot solve this or any other of the LRGB frames integrated.
The camera is an ASI2600MM pro and the OTA is a C14 EHD with .7 focal reducer. The mount is an AP1600GTO AE.
Does the ImageSolver script modify the FITS metadata ? Thanks