PixInsight Forum (historical)
Software Development => New Scripts and Modules => Topic started by: Carlos Milovic on 2010 May 03 21:34:26
-
StarStatistics is a PCL class (not included, yet) in development which performs a least squares fit of a gaussian or schwarzschild (asymmetrical) point spread functions.
To implement the class, and show examples of it's use, here is a process module designed for testing purposes, named ReadPSF. The user sets a initial (closer to the real one) center, and a few other parameters, and it gives the PSF statistics on the console.
Both codes are released under the GPL, with the intention to create a community development effort around them.
Download links:
Source code: http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-src-20100520.zip (http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-src-20100520.zip)
Windows 32bits: http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-pxm-20100520_win_32.zip (http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-pxm-20100520_win_32.zip)
Windows 64bits: http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-pxm-20100520_win_64.zip (http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-pxm-20100520_win_64.zip)
Linux 64bits: http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-pxm-20100520_x11_64.tar.gz (http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-pxm-20100520_x11_64.tar.gz)
(Will include 32bits releases if asked).
Sample image (16bits fits): http://pteam.pixinsight.com/pcldev/starstatistics/startest.zip (http://pteam.pixinsight.com/pcldev/starstatistics/startest.zip)
Current state:
StarStatistics: 2.2.0 2010/05/20
ReadPSF: 1.0.4 2010/05/20 (Angle in degrees).
Known issues:
- No known issues. Need user input.
Check the date on the filename to see if you have the latest version (format: YYYYMMDD).
EDIT: Uploaded new StarStatistics.cpp file. Changed the initial least square errors matrix. New values should give more accurate results. Please test.
EDIT2: New StarStatistics class and ReadPSF process. Changed the whole algorithm to a maximum likelihood method. Better results, and more consistent. Included a Windows 64bits compilation, for testing.
EDIT3: Added a sample image.
EDIT4: Sigma correction added. Better results right now. Intensity/Flux calculations redesigned.
EDIT5: Lots of changes to the StarStatistics class. Added iterative functions that follows a convergence criteria to determine the centroid, peak intensity and standard deviations. Results are very consistent right now. Code fully commented. New correction factors added, due to differences between the discrete data and a continuous distribution model. Linux 64bits compilation added.
-
Hi Carlos,
How is this installed?
bud
-
Hi Bud (and Carlos),
As I understand things, the zip file provided by Carlos needs to be COMPILED before it can be installed - and that means I am now WELL out of my 'comfort zone'.
What I would like is for someone (Carlos, Juan, Sander, etc.) to throw together a real simple 'step by step' tutorial to help the likes of me - who could at least 'read' the likes of your .c and .h files (and therefore can claim to 'understand' about 50% of the code at first pass through ;D) - and could then use your framework to be able to move on from the constraints of the PJSR world. If 'we' (i.e. the great unwashed) can figure out how to compile to PCL modules, then some more interesting stuff might get added to the world of PI (assuming we can also get a hold of the 28-hour days that Juan seems to use!).
Cheers,
-
Hi Niall
I kind of figured that it needed to be compiled.....I was/am going to ask my friend Mike Reid.....he is currently "test driving" the program and has quite a bit of programming experience. Perhaps he could compile it for us to be used on a windows machine.....I'm on a fairly powerful Vista 64.
I'll point him to this thread....
bud
-
Hi guys,
As I understand this.....much of this stuff needs to be beta tested.
Why not put it in a form that it can be installed?...ie precompiled as a module/process that the PI core program can recognize from...where?...the .bin folder"?
I have pointed my friend Mike at this thread and he might be able to help us non-programmers, but if you're needing beta testers to find those things that don't seem to work this might get a few more involved.
best regards,
bud
-
Hey, this is a good idea indeed. Maybe Juan can provide a repository of "Beta"-modules that the more advanturous amongst os can test at their own risk.
From my own experience I can say that it is almost impossible for a hobby programmer to compile modules for all those platforms that PI supports. Who of us has a build farm with Win32+Win64+Lin32+Lin64+Mac+FreeBSD plus assorted development tools available? Not me, for sure.
Georg
-
Hi Georg, Bud,
In fact I think that you have both got some valid points.
Yes, we COULD be doing with a 'BETA Bin' - somehwere for pre-compiled 'beta' modules to reside. Those of us who are able to compile such modules could simply 'make them available' for others to use, but they would at least be 'pre-compiled' with a name that advises which OS they should function with.
Then for others - perhaps like me - we would be happy to tinker with the C-level source code (generously provided by others, perhaps encouraging us to 'move up' to creating a new module from scratch). However, 'we' would need to be guided as to how to 'compile' such code into a useable module.
Importantly though, we would only need to compile it for our OWN Operating System (perhaps a second, if available, and if 'we' feel so inclined). The benefit then being that, if the code is taken up by someone else - who can compile it for a different OS, then this 'extra' compiled version could be added to the 'Beta Bin'.
Right now I see a gulf between PJSR and PCL, and I don't know how to cross it, because I don't have the skills. Whilst I 'can' write code in PJSR, and I (most likely) can get by coding in C (although it is a truly DREADFUL, and utterly NON-intuitive language to work with - IMHO, of course :P), I would definitely need someone to 'hold my hand' through the first attempts at compiling a PCL module.
Perhaps (Bud?) your you and chum could 'document' the steps needed - by actually installing all the necessary software components on YOUR computer (or mine - I am happy to make myself available at the end of a TeamViewer link). Alternatively - if the 'skilled programmers' amongst us could put together a quick and simple tutorial, the PixInsight community is bound to benefit in the long term.
Cheers,
-
Niall,
...
Perhaps (Bud?) your you and chum could 'document' the steps needed - by actually installing all the necessary software components on YOUR computer (or mine - I am happy to make myself available at the end of a TeamViewer link). Alternatively - if the 'skilled programmers' amongst us could put together a quick and simple tutorial, the PixInsight community is bound to benefit in the long term.
...
Is a recipe like http://pixinsight.com/forum/index.php?topic=1791.msg11361#msg11361 sufficient?
Georg
-
Hi Georg, Bud,
In fact I think that you have both got some valid points.
Yes, we COULD be doing with a 'BETA Bin' - somehwere for pre-compiled 'beta' modules to reside. Those of us who are able to compile such modules could simply 'make them available' for others to use, but they would at least be 'pre-compiled' with a name that advises which OS they should function with.
Then for others - perhaps like me - we would be happy to tinker with the C-level source code (generously provided by others, perhaps encouraging us to 'move up' to creating a new module from scratch). However, 'we' would need to be guided as to how to 'compile' such code into a useable module.
Importantly though, we would only need to compile it for our OWN Operating System (perhaps a second, if available, and if 'we' feel so inclined). The benefit then being that, if the code is taken up by someone else - who can compile it for a different OS, then this 'extra' compiled version could be added to the 'Beta Bin'.
Right now I see a gulf between PJSR and PCL, and I don't know how to cross it, because I don't have the skills. Whilst I 'can' write code in PJSR, and I (most likely) can get by coding in C (although it is a truly DREADFUL, and utterly NON-intuitive language to work with - IMHO, of course :P), I would definitely need someone to 'hold my hand' through the first attempts at compiling a PCL module.
Perhaps (Bud?) your you and chum could 'document' the steps needed - by actually installing all the necessary software components on YOUR computer (or mine - I am happy to make myself available at the end of a TeamViewer link). Alternatively - if the 'skilled programmers' amongst us could put together a quick and simple tutorial, the PixInsight community is bound to benefit in the long term.
Cheers,
dit, dit, dit, dit, dittos.....
bud
-
Hi Georg,
Thanks (also) for your quick reply. However - in the second sentence you had used the word "Linux" - and I had to stop there. Because I don't have time to learn Linux as well :'( (even though I have an old ex-works PC 'box' that I literally pulled out of the waste skip in the carpark, which I have allowed to drip-dry, and which has been set aside to try running Linux in - eventually).
So, I will re-read the link (trying to ignore the Linux references) and see if it means anything. But I still feel that I will (obviously) have to start with the 'freebie' Microsoft Visual C++ compiler - and so I need to know what I should be looking to download.
Then I need to take something like the StarStatistics / ReadPSF zip-file data, and be guided (for I doubt that I would 'figure it out myself' ::)) through the compilation process. (Yes, I know that, somewhere, there exists the SandBox code as well, but I can't even remember where that is hidden >:()
Heck, if someone want to link through to my machine, I could try grabbing the whole session on Camtasia and releasing THAT as a video tutorial ;D
Cheers,
-
Georg made a fine recipe for linux.
In windows, things are easier :P
1.- You have to install Microsoft Visual C++ Express 2008 (freeware, needs registration after trial time, but is costless; all you need is a live id account).
2.-Then, define all the environment variables (see Georg's post). They are under System (Control Panel), I believe in the advanced category. Just have to create a new variable, and enter the path. About the lib variables, since you'll have only one OS bit system, replace the other path with the current one (i.e. if you are in a 64bits system, set PCLLIBDIR32=$PCLDIR/lib/x86_64 since there is no x86 folder.).
3.- Run PixInsight, and open the Makefile script (development category). If all environment variables were successfully established, the window will open. Create the project for the code.
4.- Now you should have a MSVC project file inside a subfolder in the code directory. Run it to open MSVC.
5- To compile the code, select the Release option (change it from Debug), and then click on "Build Solution".
6.- If successful, the new module should be created in PixInsight's bin folder.
7.- Open PI and install the module.
I'll upload compiled versions to PI's server. Unfortunately, I'm having some problems linking the code with most OS, so I may provide only Win32 files right now.
-
Hi Niall
You really should try Linux. I'm enjoying a lot the new Ubuntu release. Also, you'll find that PI "orientation" is very similar to an Linux OS.
About Sandbox, look in the src/modules/processes folder under PinInsight (you need the PCL package, not just the PI Core). Take a good look at that folder, since there are lots of sample codes. I must confess that I always start a new project working over an existing one, which is the most similar to mine. Also, Sandbox provides nice examples on how to handle the most common variables.
-
Carlos,
Will it work with the Visual Basic 2010 Express.....or should I download the full deal?
A search for the 2008 redirects to the 2010 version.
bud
-
Thanks all,
Now, if one of you could send me a good excuse for me to use with my darling wife :-* - who will now not see me for the next few days, whilst I try and find time to 'play' with this new possibility.
And Georg, I think I am saving Linux for my retirement - I have used all the expansion slots in my Mk1 Brain at the moment, and feel that I will have to offload and archive my "Daily Grind" soft-drive before I can upload a new environment such as Linux.
Either that, or I will have to shell out for the 'Brain Overdrive' conversion - O0 - as used by Juan
Cheers,
-
Hi Bud,
Visual Basic 2010 Express
I think it would HAVE to be Visual C++, after all, the code is written in C.
Cheers,
-
And, Bud,
A quick Google took me to
http://www.microsoft.com/express/windows/ (http://www.microsoft.com/express/windows/)
From which, I think, I would then be going to
http://www.microsoft.com/express/Downloads/#2010-Visual-CPP (http://www.microsoft.com/express/Downloads/#2010-Visual-CPP)
and would be choosing to download Visual C++ 2010 Express, in English, as a free download.
That's what I'll be trying tonight, in any case, unless someone else knows otherwise, or tries it and tells me that I am wasting my time ;D
Cheers,
-
Bud: No, it won't work with it. I tried Visual C++ 2010 and the projects were not compatible. I think that a full Visual Studio 2008 will work, but have not tried (I think I can get a registered copy through the university).
Try this link: http://download.microsoft.com/download/7/3/2/73243975-91CC-4EE4-9365-2C705A410DF3/vcsetup.exe
Niall: Ubuntu is one of the more "user friendly" Linux distros. Their motto is "linux for humans" :D Somebody get used to windows should use it without any problems. Fedora is quite similar too, but I would say that Ubuntu is simpler.
-
Also here: http://www.microsoft.com/express/Downloads/#2008-Visual-CPP
-
Also here: http://www.microsoft.com/express/Downloads/#2008-Visual-CPP
Thanks Carlos, that is where I went....I didn't see the 2008 tab at the top of the page.
bud
-
Hi Guys,
This is a good thread. I look forward to trying this out later this evening. Unlike my retired chum I actually have to work for a living so unless Bud if golfing this afternoon he will have probably gotten this all worked out by the time I get around to it!
I must say as a newbie to PixInsight I am thrilled by the open platform aspects of this software.
Now I have to return to addressing 2 news bugs that have been filed against me just this morning by my paying customers. :(
Mike
-
Hi everyone
I've been thinking on ways to improve the results. I guess that changing the magnitude of the errors, giving a more relative weight to pixels with higher value might be a solution. I uploaded a new StarStatistics.cpp file that incorporates that change.
Also, I can't remember if the ReadPSF process allowed to change the threshold parameter. If not, this should be another improvement in accuracy (values larger than 0, but low enough to include several "good pixels").
See the first post for the file.
-
A little more info for first time module compilers...
I managed to track down the installation files for Visual C++ 2008 Express with google but there is another wrinkle. I am currently running PixInsight 64-bit on my Windows 7 machine as it has the fastest CPU and most RAM. I assume that to one must compile modules for 64-bit target to use them with the 64-bit version of PixInsight. But Visual C++ Express edition does not support 64 bit compilation target. You have to buy the full Monty from Micro$oft for that privilege. There is a hack that is described here,
http://jenshuebel.wordpress.com/2009/02/12/visual-c-2008-express-edition-and-64-bit-targets/
I probably won't try this as I own the full version of Visual Studio 2005 which I used to develop my Roll-off roof ASCOM driver a couple of years ago. I just need to install it on my new machine.
Mike
-
I was afraid of that... tried installing several expansion packages to give 64bits support but neither worked. I always have linking errors at the end. I think this answers things...
Well, I'll try to install the full version, using a licence I got from the university's contract with micro$oft, and then upload the libraries for W32 and W64 (sorry, but had not entirely solved my problems with eclipse in linux... it hates me).
Back to the main topic, I'm working on a full redesign of the fitting algorithm, using the maximum likehood method (instead of the log least squares method). It seems to get superior results in terms of accuracy. Also the code may be breaked down and simplified a bit. Need to study it more to implement the code :)
-
Well,
From what appears to have been said, dedicating my entire evening, last night, to the download of MS Visual Studio Express 2008 and 2010 would now seem to have been a total waste of time >:(
Having paid 'over the odds' for a full-blown version of Vista64, it does seem rather pointless having to work in a 32-bit environment in order to be able to compile PCL. So, it looks like I will not be wasting any further time on this.
And then Microsnot wonder why there is such a high level of 'black-marketeering' and hacking that takes place with their products (on a matter of principle, I considered ALL Microsoft products as 'Beta' code up until the release of XP)
Well - I suppose I will get to see my sweetheart :-* more often, if I am not hard-wired to the keyboard!
Cheers,
-
Hi everybody
There is a new upload of the StarStatistics class and the ReadPSF process.
I rewrote the whole class, changing the algorithm. Now it yields much better results. Very consistent, specially the centroid coordinates. I think that the standard deviation needs a little more work, since still depends a lot of the selected parameters (threshold and radius). Anyway, it is an improvement over the previous version.
I'll upload a compiled version for Win64 tomorrow. Just have to upload the files to the ftp server, but it is too late now. :P Also I'll upload a sample image, with artificial stars and their parameters.
-
Well, the compiled module is up... I need more time for the sample image (create the list, etc.) so this one indeed will be tomorrow :D
http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-pxm-20100512_w64.zip
-
Hi,
I was able to install it and run it.
I just wonder what the x and y coordinate are? Do I have to find a star manually and enter its coorrdinate? This seems to give some results. typically:
*** PSF results ***
Sigma...........: 3.48417
FWHM............: 8.2046
Total Flux......: -1.#IND
Peak Intensity..: -1.#IND
Center..........: 1500 , 1168
Sky Level.......: -1.#IND
*** Distribution data ***
Sigma X.........: 3.48417
Sigma Y.........: 3.48417
Correlation.....: 0
0.0270 s
-- bitli
-
Hi Carlos,
Same here - a painless dowload and install, and I didn't need to do anything that wasn't 'intuitive', and I didn't have to read up on the instructions of 'how to' ::)
I ran the process three or four times, tweaking the <Radius> as I went, and then finally un-ticking the <Simmetry> box (Sorry, it is actually Symmetry, if you want to correct the spelling !!)
This is the console output
ReadPSF: Processing view: r_dkflcal_16_bit_M82_IIP_Lu_600s_1_0C_0004
Writing swap files...
13.17 MB/s
*** PSF results ***
Sigma...........: 3.28131
FWHM............: 7.72689
Total Flux......: -1619.59
Peak Intensity..: -23.9404
Center..........: 419.029 , 434.927
Sky Level.......: 0.30461
*** Distribution data ***
Sigma X.........: 3.28131
Sigma Y.........: 3.28131
Correlation.....: 0
0.0460 s
ReadPSF: Processing view: r_dkflcal_16_bit_M82_IIP_Lu_600s_1_0C_0004
Writing swap files...
21.10 MB/s
*** PSF results ***
Sigma...........: 10.4815
FWHM............: 24.6821
Total Flux......: 1.434
Peak Intensity..: 0.00207741
Center..........: 419.006 , 434.985
Sky Level.......: 0.261422
*** Distribution data ***
Sigma X.........: 10.4815
Sigma Y.........: 10.4815
Correlation.....: 0
0.0470 s
ReadPSF: Processing view: r_dkflcal_16_bit_M82_IIP_Lu_600s_1_0C_0004
Writing swap files...
26.56 MB/s
*** PSF results ***
Sigma...........: 6.90163
FWHM............: 16.2521
Total Flux......: -5.34982
Peak Intensity..: -0.0178754
Center..........: 419.007 , 434.982
Sky Level.......: 0.270227
*** Distribution data ***
Sigma X.........: 6.90163
Sigma Y.........: 6.90163
Correlation.....: 0
0.0460 s
ReadPSF: Processing view: r_dkflcal_16_bit_M82_IIP_Lu_600s_1_0C_0004
Writing swap files...
7.55 MB/s
*** PSF results ***
Sigma...........: 4.55957
Sigma min axis..: 4.5181
FWHM............: 10.737
FWHM min axis...: 10.6393
Rotation Angle..: -0.801138
Total Flux......: -9.06128
Peak Intensity..: -0.0700051
Center..........: 419.007 , 434.982
Sky Level.......: 0.269997
*** Distribution data ***
Sigma X.........: 4.53823
Sigma Y.........: 4.53954
Correlation.....: -0.00913196
0.0620 s
And this was based on the following single star, whose position I determined 'by eye'
Cheers,
-
Hey guys!
Thanks for downloading and trying. It seems that the intensity and flux parameters are not working well... I'll take a look at them. Also I'm worried about the lack of consistency for the std dev calculations... it should not be that dependent on the input parameters.
A quick note about the module: yes, you have to set the initial centroid search, as close as you can (no need to be too exact, just 2-3 pixels around, depending on your search radius).
Well, I'll upload a new release today. Thanks Niall for the spelling check ;)
-
Hi Carlos,
For your code update, might I make a suggestion?
It would be useful to have a quick summary of the parameters actually being used - in the first few lines of the Console output. This would make the listing more 'understandable' when the user takes the time to inspect it in detail.
Cheers,
-
Hmmm, could you elaborate this further? I'm not sure I got what you said
-
Sorry Carlos,
What I meant was to have the Console display show what the fundamental process -setup was (the Radius of the circle, Symettry On/Off, etc.) at the time it was called.
Because this kind of process does NOT have an on-screen window (like the Statistics process, for example) you basically trigger the Process, and then have to look at the Console window output to interrogate the results.
In my case I tried a whole lot of settings, then looked at the Console output, and couldn't remember what I had actually applied for each activation !!
On a side note, in my glorious new multi-monitor environment, I have un-docked the Console completely, and have moved it off to an auxilliary monitor (at the LHS of my 28"WS, and now in 'portrait mode) - and can now leave it 'on' all the time. The console no longer pops up and away - it is just 'always there'. Similarly, I have the RHS monitor permanently configured to display the Statistics, Histogram and STF windows, which will I am hoping will remain displayed independent of Workspace selection once v1.6.1 comes out. Thanks Juan :D
Cheers,
-
What I meant was to have the Console display show what the fundamental process -setup was (the Radius of the circle, Symettry On/Off, etc.) at the time it was called.
Oh, ok. :) Anyway, you can know if it was symetric or not by the Sigma and SigmaMin outputs (the later should not be shown in the symetrical case). Similarly, Background sky value is only shown when it was internally calculated.
Another option to recover the parameters of old executions is to look at the processing history. It should be non empty.
By the way, some thoughts about the results, from my experiments and analysis:
- The centroid coordinates seems to be very robust and consistent. So, if no problems are detected here, the calculation should remain as it is right now.
- The main problem lies with the standard deviation. Since all other calculations depends on their values, this generates a cascade of errors. I managed to somehow limit that problem in my last code version (not uploaded) so it seems alwasy to generate smaller values than the real one. The reason is that the algorithm tries to fit the data with a full normal bivariate distribution, while in fact it is a truncated distribution. All those missing values yields a smaller sigma calculation. I'm working on new maximum likelihood stimators to deal with this problem. I'll try to create a radially truncated function, both internally and externally (to avoid problems from saturated pixels in the cores). Hopefully tomorrow I'll have something... I'll meet with a statistics professor and work on this. From the moment, I'm looking for references on published papers :P
So, that's for now... As I said, I'll upload a test image, with artificial stars tonight.
-
Carlos,
You are lucky that you have access to a Statistics Professor - the only one I have 'access to' is Juan :laugh:
Going back to the current Console output - and I have printed out your actual code, but have not yet had time to 'follow it through' - given that my example star is definitely 'elongated', how do I interpret the 'Sigma and Sigma Min Axis' or the 'FWHM and FWHM Min Axis' results?
Do these actually correlate to the X and Y axes? Or do they correlate to the Major and Minor Axes of some 'best fit Ellipsoid', whose Major axis is 'rotated' form the usual cartesian zero-degree reference (i.e. along the positive x-axis)?
Am I also therefore correctly interpreting the Rotation Angle result, as a Float vlaue that should be multiplied by 360 to give the (clockwise) rotation from the positive x-axis?
(and I will have a closer think about the other results, and will get back to you on those as necessary)
Finally, would it be a case of writing a PJSR that would fist establish a list of 'star centroids', and then having it iterate through this array, working out FWHM stats (etc.) for each star loaction? If so, I see a (possible) problem concerning how PSFInfo is actually called - is there a 'lower level' means of calling the actual PSFInfo routine, whereby - for example - the image SwapFile does not need to be created, and the Console Output does not get triggered, and where, instead, the data is simply returned to the PJSR - to allow it to be formatted and output accordingly?
And, yes, what I am still getting at is the ability to create a 'new image', based on a source image, where the intensity of ADU is representative of FWHM (etc.) across the source image. In other words a calibrated, but still 'linear', image of a nice rich starfield allows the user to verify the performance of their optical train - answering questions like "is the CCD 'square' to the optical axis?", etc. The 'new image' could then be viewed in our existing "3-D viewer' application to allow strange anomalies to be better represented.
Cheers,
-
Sigma and Sigma Min (also the fwhm values) are measured along the elipsoid axis. So, they give the maximum and minimum widths. Meanwhile, standard deviation refering to X and Y are measured in the image coordinate system. Actually, Sigma is calculated from those standard deviations and the correlation factor (and fwhm from sigma).
BTW, I'm not searching for elipsoids... although this may be done as al alternative method.
The rotation angle right now is measured in radians. I forgot to change it to degrees. (*180/Pi)
Another btw. I had to make some assumptions when resolving the equation system (4th degree), so there may be a problem with the sign of the angle. The magnitude is wrong, due the errors in the sigma values, but at least the orientation should be fine. I had not checked that... Focusing on the sigma problem right now.
About your questions final:
a) I don't see why not. It could be done thorugh another process, whose outputs are associated to variables instead of writing the results out. Another option is to implement a call to the Starstatistics class from the PSJR and do the job there (as I think is done with the statistics class).
b) Yes, console outputs may be disabled (not enabling console notifications, at the start of the instance). Not sure it it will cancel every output (like the execution time) but I think that this should work.
c) Yes, this is a very nice application. Also, a distortion map, based on rotation angles and aspect ratio could be used for other things...
-
Just uploaded the sample image, with artificial stars. Also the zip file contains a text file with all the pixel math expressions used, so you can read the parameters from there. No rotation angle was used. Finally, there is a icon with a sample PM instance.
See the first post for the link.
-
Not sure it is possible, but if the result could be left in the process object so we can examine the values from a script, that would be great,
this could be used to take some decisions in another script.
-- bitli
-
Yes, I'm planning this in the future... but first, I must make sure that the outputs are fine :D When the starstatistics class is ready, I'll work on a new process to read the PSFs from the image.
-
Hi everybody
New upgrade of the source code and Win 64 release (if asked, I'll upload other versions... w32 and linux 32/64).
Changes:
- I decided to remove the automatic sky determination. The algorithm was not working right, and I feel that this problem needs a dedicated solution. So, once you determine the sky background value, you have to send it to the StarStatistics class. Another option, of course, is to extract the PSF on a background substracted image...
- The intensity calculation has been completely reworked. I was using a least squares approach from the predicted function, to find the best fit of the data. Because I had accuracy issues with sigma values, this yielded very poor estimations. Now I'm calculating it from statistical properties of the sample. I'm comparing the samples flux and total square flux per pixel to the truncated normal bivariate distribution's <f> and <f^2>.
New results are much more accurate.
- After a momentum research of the truncated normal bivariate distribution, I found a correction factor that is applied to the data's variance to find the complete distribution's standard deviation. Also I'm applying a bias correction factor. Both corrections yields a much better sigma value, still lower than the theoretical value for poorly resolved stars. I believe that this difference appears from the discretization of values, while I modeled the problem with continual functions. I'll try to fix that, but I think that I'm getting close to the inherent statistical error.
I have not checked the asymmetric outputs carefully (i.e., make sure that the angle calculation is right, and that the max and min sigmas have not been switched in any case :P ). Also I forgot to change the displayed angle value from radians to degrees. This will be done in the next release. From now on I'll try to focus more on the process itself, than in the starstatistics class.
Oh, almost forgot to mention that the threshold parameter have become very critical. It should be high enough, so that it rejects more pixels than the radius (I need to redesign the boundary modelation of the truncated normal bivariate function), but low enough to include as much samples as possible. In my tests I found that a 0.1 value usually yields the better results. This may not be the case with real data...
Please, compare the results with the ones generated by other softwares. Also, it would be nice if you may provide me a pair of sample images (good real data, with known fwhm measurements [in pixels]).
PS: Any help with the code is highly wellcome ;)
-
Hi Carlos,
You might want to update the first post in this thread - the following 'hyperlinks' don't 'link' !!
Source code: http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-src-20100512.zip
Windows 64bits: http://pteam.pixinsight.com/pcldev/starstatistics/ReadPSF-pxm-20100512_w64.zip
Sample image (16bits fits): http://pteam.pixinsight.com/pcldev/starstatistics/startest.zip
Cheers,
-
Update your browser ;) I deleted those old links, and replaced with the new ones (date ending with 15).
I don't know why the second and third links are not hyperlinks... but they point to existing files.
EDIT: I changed the coding of the first message... now the links are defined explicitly as hyperlinks :)
-
Hi Carlos,
In fact I had gone 'directly' to the web server, and downloaded the data from the 'FTP' representation of the web page. But, you picked up on the fact that the second two 'links' were not 'hyperlinks'.
I'm still trying to find time to 'enjoy reading' your PCL code - just too many other projects on the go at the moment. But, I have them printed out now, and will be attacking them with my yellow highlighter as I learn how you put them together. As I said before, if I find anything that I can contribute, I'll be back in touch!
Cheers,
-
Thanks Niall :)
Now the important code is in StarStatistics.h and StarStatistics.cpp. The process implementation (ReadPSF) does almost nothing. In instance.cpp you'll see how a object of the starstatistics class is created, initialized, and then applied to the image. That's all.
-
Thanks for the clarification Carlos,
As I said, I am really looking forward to getting my head round the PCL environment as well as the PJSR one. All I need is more time :'(
Cheers,
-
Just added new compilations (win & linux 64bits) and the source code. Lots of changes (some minor, other not) to the class code. To summarize, I'm very happy with the current results, for every output parameter. My experiments have been very consistent (both symmetric and asymmetric synthetic stars), and the measurements are well within statistical error from the undersampled data (I mean, we do not have every point for a continuous gaussian function, just a limited set of pixels).
The threshold value still is critical, but now results are much more consistent along a wider range of values. Also I incorporated a "fail safe" function, to avoid problems with too low (or zero) thresholds.
Please, test it and compare results with other software.
Future plans: redesign the ReadPSF process. It will still be a single star reading tool, but I'll make it much more interactive, using readout data as sample coordinates input, and creating a graph of the result with the sampled data. Other ideas are highly welcome, as long as other interface proposals.
-
Other ideas are highly welcome, as long as other interface proposals.
What about extract the X.Y from mouse position?
-
Yes, this is one of the first features I'll include. Is quite easy to implement.
-
Added win 32 compilation, and fixed broken links (sorry).
-
Installed the Win32. FWHM values seem reasonable. Will have to compare figures with some other software.
Agree X,Y extracted from mouse click will be nicely eventually.
-
5- To compile the code, select the Release option (change it from Debug), and then click on "Build Solution".
6.- If successful, the new module should be created in PixInsight's bin folder.
7.- Open PI and install the module.
Carlos thank you for step-by-step instruction. Compilation successful. But I got some warning... see log:
1>------ Build started: Project: ReadPSF-src-20100520, Configuration: Release Win32 ------
1>Compiling...
1>StarStatistics.cpp
1>ReadPSFProcess.cpp
1>C:\PCL\include\pcl/Control.h(1308) : warning C4521: 'pcl::Control' : multiple copy constructors specified
1>ReadPSFParameters.cpp
1>ReadPSFModule.cpp
1>C:\PCL\include\pcl/Control.h(1308) : warning C4521: 'pcl::Control' : multiple copy constructors specified
1>..\..\ReadPSFModule.cpp(43) : warning C4003: not enough actual parameters for macro 'PCL_STRINGIFY'
1>ReadPSFInterface.cpp
1>C:\PCL\include\pcl/Control.h(1308) : warning C4521: 'pcl::Control' : multiple copy constructors specified
1>ReadPSFInstance.cpp
1>Linking...
1> Creating library Win32\Release\ReadPSF-src-20100520.lib and object Win32\Release\ReadPSF-src-20100520.exp
1>Generating code
1>Finished generating code
1>Embedding manifest...
1>Build log was saved at "file://c:\PCL\src\modules\processes\ReadPSF-src-20100520\windows\vc9\Win32\Release\BuildLog.htm"
1>ReadPSF-src-20100520 - 0 error(s), 4 warning(s)
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
-
Hi Nikolay,
Just disregard those warnings; they are completely harmless (and useless, too).
-
And unavoidable ;) Those warning are inherited from the PCL definition of Control. Just ignore them.
-
Guys thanks.
-
I found a bug with the ReadPSF process, when creating process icons (standalone ReadPSF ones, or inside a ProcessContainer, like a ProcessHistory). The isSimmetric parameter is not stored adequately. This will be fixed next time I upgrade the code.
-
How to get access to *** PSF results *** from script?
-
Right now, you can't. For the next upload I'll create a few new variables that will hold the psf results, and then they should be accessed by a script. Sorry but I have been busy at the university, so I had no time to do this before. I'll try to perform that this weekend.
-
Thank you.
-
Carols is there any recent update on this tool? What is the current version for Win64bit?
Are all process's for PI written in Cpp?
Edit update
Ok I tried to download your compiled Dll file and placed it in \prgoram files\pixinsight\bin\ folder. Tried to Install it from Install Module but I get errors about a possible hack etc.
Installing 1 module(s):
C:/Program Files/PixInsight/bin/ReadPSF-pxm.dll
<* failed *>
*** PixInsight API Error: Module: C:/Program Files/PixInsight/bin/ReadPSF-pxm.dll: Module initialization error: Exit code: 1
*** Warning: No modules were installed.
What am I Doing wrong here?
Regards
Fahim
-
Hi Fahim
The latest version is in my thread collecting all the development modules I've wrote
http://pixinsight.com/forum/index.php?topic=2275.0
The ReadPSF process is inside the CMSegmentation module. There are few changes. In terms of GUI, I added a "readout mode" that allows you to select the coordinates using the mouse position and clicking over a star.
There are the Windows and Linux 64bit releases.
Yes, all the modules are done using the C++ language. I think that Juan has plans to include others, but don't expect that to be any-time soon.
Hmmm. That is strange. I'm going to download the same file and try to install it.
-
Do you have the latest CMSegmentation release? Are you running PI Core 1.7? I checked on my Windows system, and I've got the process running without problems.
-
Carlos,
Yes I am running PI.
Thanks jusd downloaded all your Developmetn modules from your other link. Will try them and See. Now I note CMSegmentation module seems to include ReadPSF, so do I still need to install that seperately?
BTW is it possible to store Dev modules in a seperate BinDev folder? Or must they be in \pixisnight\bin\?
Thanks and Regards
Fahim
-
No. Just install that module. It is compiled with the latest PCL release, so it should work without problems in PI 1.7. The module that is in this threat is an older version, outdated, and no longe compatible.
Yes, they could be installed anywhere. You just have to select the appropiate folder when you are installing them with the module manager.
Looking forward for your feedback :)
-
All is good and working now. I noted that all you modules do not go into any Process sub-menu's. They all appear only the "all process" menu.
ReadPSF is great little tool I worked out how to use it but it would be great if it could automatically find the stars and give you an Average FWHM value. Or maybe let you select multiple stars (of similar size) and give you a average FWHM from those.
I am currently also in parrallel looking at SExtractor, tried to compile it on Mint Debian but it was just going nowhere to many dependencies missing etc etc. I will try on a Fedora VM i have ass there is a RPM for it.
BTW will these DLL file from the other thread work in Linux also for PI?
-
No, for Linux you must use the .so files. I've compiled everything for Linux 64bits, and the latest PCL/PI release. My computers run Fedora both at home and at work, so it is my main development platform.