StarStatistics class and ReadPSF process module

[Carlos Milovic]

Hmmm, could you elaborate this further? I'm not sure I got what you said

[Niall Saunders]

Sorry Carlos,

What I meant was to have the Console display show what the fundamental process -setup was (the Radius of the circle, Symettry On/Off, etc.) at the time it was called.

Because this kind of process does NOT have an on-screen window (like the Statistics process, for example) you basically trigger the Process, and then have to look at the Console window output to interrogate the results.

In my case I tried a whole lot of settings, then looked at the Console output, and couldn't remember what I had actually applied for each activation !!

On a side note, in my glorious new multi-monitor environment, I have un-docked the Console completely, and have moved it off to an auxilliary monitor (at the LHS of my 28"WS, and now in 'portrait mode) - and can now leave it 'on' all the time. The console no longer pops up and away - it is just 'always there'. Similarly, I have the RHS monitor permanently configured to display the Statistics, Histogram and STF windows, which will I am hoping will remain displayed independent of Workspace selection once v1.6.1 comes out. Thanks Juan 

Cheers,

[Carlos Milovic]

What I meant was to have the Console display show what the fundamental process -setup was (the Radius of the circle, Symettry On/Off, etc.) at the time it was called.

Oh, ok. Anyway, you can know if it was symetric or not by the Sigma and SigmaMin outputs (the later should not be shown in the symetrical case). Similarly, Background sky value is only shown when it was internally calculated.

Another option to recover the parameters of old executions is to look at the processing history. It should be non empty.



By the way, some thoughts about the results, from my experiments and analysis:
- The centroid coordinates seems to be very robust and consistent. So, if no problems are detected here, the calculation should remain as it is right now.
- The main problem lies with the standard deviation. Since all other calculations depends on their values, this generates a cascade of errors. I managed to somehow limit that problem in my last code version (not uploaded) so it seems alwasy to generate smaller values than the real one. The reason is that the algorithm tries to fit the data with a full normal bivariate distribution, while in fact it is a truncated distribution. All those missing values yields a smaller sigma calculation. I'm working on new maximum likelihood stimators to deal with this problem. I'll try to create a radially truncated function, both internally and externally (to avoid problems from saturated pixels in the cores). Hopefully tomorrow I'll have something... I'll meet with a statistics professor and work on this. From the moment, I'm looking for references on published papers

So, that's for now... As I said, I'll upload a test image, with artificial stars tonight.

[Niall Saunders]

Carlos,

You are lucky that you have access to a Statistics Professor - the only one I have 'access to' is Juan

Going back to the current Console output - and I have printed out your actual code, but have not yet had time to 'follow it through' - given that my example star is definitely 'elongated', how do I interpret the 'Sigma and Sigma Min Axis' or the 'FWHM and FWHM Min Axis' results?

Do these actually correlate to the X and Y axes? Or do they correlate to the Major and Minor Axes of some 'best fit Ellipsoid', whose Major axis is 'rotated' form the usual cartesian zero-degree reference (i.e. along the positive x-axis)?

Am I also therefore correctly interpreting the Rotation Angle result, as a Float vlaue that should be multiplied by 360 to give the (clockwise) rotation from the positive x-axis?

(and I will have a closer think about the other results, and will get back to you on those as necessary)

Finally, would it be a case of writing a PJSR that would fist establish a list of 'star centroids', and then having it iterate through this array, working out FWHM stats (etc.) for each star loaction? If so, I see a (possible) problem concerning how PSFInfo is actually called - is there a 'lower level' means of calling the actual PSFInfo routine, whereby - for example - the image SwapFile does not need to be created, and the Console Output does not get triggered, and where, instead, the data is simply returned to the PJSR - to allow it to be formatted and output accordingly?

And, yes, what I am still getting at is the ability to create a 'new image', based on a source image, where the intensity of ADU is representative of FWHM (etc.) across the source image. In other words a calibrated, but still 'linear', image of a nice rich starfield allows the user to verify the performance of their optical train - answering questions like "is the CCD 'square' to the optical axis?", etc. The 'new image' could then be viewed in our existing "3-D viewer' application to allow strange anomalies to be better represented.

Cheers,

[Carlos Milovic]

Sigma and Sigma Min (also the fwhm values) are measured along the elipsoid axis. So, they give the maximum and minimum widths. Meanwhile, standard deviation refering to X and Y are measured in the image coordinate system. Actually, Sigma is calculated from those standard deviations and the correlation factor (and fwhm from sigma).
BTW, I'm not searching for elipsoids... although this may be done as al alternative method.

The rotation angle right now is measured in radians. I forgot to change it to degrees. (*180/Pi)
Another btw. I had to make some assumptions when resolving the equation system (4th degree), so there may be a problem with the sign of the angle. The magnitude is wrong, due the errors in the sigma values, but at least the orientation should be fine. I had not checked that... Focusing on the sigma problem right now.

About your questions final:
a) I don't see why not. It could be done thorugh another process, whose outputs are associated to variables instead of writing the results out. Another option is to implement a call to the Starstatistics class from the PSJR and do the job there (as I think is done with the statistics class).
b) Yes, console outputs may be disabled (not enabling console notifications, at the start of the instance). Not sure it it will cancel every output (like the execution time) but I think that this should work.
c) Yes, this is a very nice application. Also, a distortion map, based on rotation angles and aspect ratio could be used for other things...

[Carlos Milovic]

Just uploaded the sample image, with artificial stars. Also the zip file contains a text file with all the pixel math expressions used, so you can read the parameters from there. No rotation angle was used. Finally, there is a icon with a sample PM instance.

See the first post for the link.

[bitli]

Not sure it is possible, but if the result could be left in the process object so we can examine the values from a script, that would be great,
this could be used to take some decisions in another script.
-- bitli

[Carlos Milovic]

Yes, I'm planning this in the future... but first, I must make sure that the outputs are fine When the starstatistics class is ready, I'll work on a new process to read the PSFs from the image.

[Carlos Milovic]

Hi everybody

New upgrade of the source code and Win 64 release (if asked, I'll upload other versions... w32 and linux 32/64).

Changes:
- I decided to remove the automatic sky determination. The algorithm was not working right, and I feel that this problem needs a dedicated solution. So, once you determine the sky background value, you have to send it to the StarStatistics class. Another option, of course, is to extract the PSF on a background substracted image...
- The intensity calculation has been completely reworked. I was using a least squares approach from the predicted function, to find the best fit of the data. Because I had accuracy issues with sigma values, this yielded very poor estimations. Now I'm calculating it from statistical properties of the sample. I'm comparing the samples flux and total square flux per pixel to the truncated normal bivariate distribution's <f> and <f^2>.
New results are much more accurate.
- After a momentum research of the truncated normal bivariate distribution, I found a correction factor that is applied to the data's variance to find the complete distribution's standard deviation. Also I'm applying a bias correction factor. Both corrections yields a much better sigma value, still lower than the theoretical value for poorly resolved stars. I believe that this difference appears from the discretization of values, while I modeled the problem with continual functions. I'll try to fix that, but I think that I'm getting close to the inherent statistical error.

I have not checked the asymmetric outputs carefully (i.e., make sure that the angle calculation is right, and that the max and min sigmas have not been switched in any case ). Also I forgot to change the displayed angle value from radians to degrees. This will be done in the next release. From now on I'll try to focus more on the process itself, than in the  starstatistics class.

Oh, almost forgot to mention that the threshold parameter have become very critical. It should be high enough, so that it rejects more pixels than the radius (I need to redesign the boundary modelation of the truncated normal bivariate function), but low enough to include as much samples as possible. In my tests I found that a 0.1 value usually yields the better results. This may not be the case with real data...

Please, compare the results with the ones generated by other softwares. Also, it would be nice if you may provide me a pair of sample images (good real data, with known fwhm measurements [in pixels]).


PS: Any help with the code is highly wellcome

[Niall Saunders]

Hi Carlos,

You might want to update the first post in this thread - the following 'hyperlinks' don't 'link' !!

Source code:   http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-src-20100512.zip
Windows 64bits:  http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-pxm-20100512_w64.zip

Sample image (16bits fits):  http://pteam.pixinsight.com/pcldev/starstatistics/startest.zip

Cheers,

[Carlos Milovic]

Update your browser I deleted those old links, and replaced with the new ones (date ending with 15).

I don't know why the second and third links are not hyperlinks... but they point to existing files.

EDIT: I changed the coding of the first message... now the links are defined explicitly as hyperlinks

[Niall Saunders]

Hi Carlos,

In fact I had gone 'directly' to the web server, and downloaded the data from the 'FTP' representation of the web page. But, you picked up on the fact that the second two 'links' were not 'hyperlinks'.

I'm still trying to find time to 'enjoy reading' your PCL code - just too many other projects on the go at the moment. But, I have them printed out now, and will be attacking them with my yellow highlighter as I learn how you put them together. As I said before, if I find anything that I can contribute, I'll be back in touch!

Cheers,

[Carlos Milovic]

Thanks Niall
Now the important code is in StarStatistics.h and StarStatistics.cpp. The process implementation (ReadPSF) does almost nothing. In instance.cpp you'll see how a object of the starstatistics class is created, initialized, and then applied to the image. That's all.

[Niall Saunders]

Thanks for the clarification Carlos,

As I said, I am really looking forward to getting my head round the PCL environment as well as the PJSR one. All I need is more time

Cheers,

[Carlos Milovic]

Just added new compilations (win & linux 64bits) and the source code. Lots of changes (some minor, other not) to the class code. To summarize, I'm very happy with the current results, for every output parameter. My experiments have been very consistent (both symmetric and asymmetric synthetic stars), and the measurements are well within statistical error from the undersampled data (I mean, we do not have every point for a continuous gaussian function, just a limited set of pixels).

The threshold value still is critical, but now results are much more consistent along a wider range of values. Also I incorporated a "fail safe" function, to avoid problems with too low (or zero) thresholds.

Please, test it and compare results with other software.

Future plans: redesign the ReadPSF process. It will still be a single star reading tool, but I'll make it much more interactive, using readout data as sample coordinates input, and creating a graph of the result with the sampled data. Other ideas are highly welcome, as long as other interface proposals.