PixInsight Forum (historical)
Software Development => PCL and PJSR Development => Topic started by: mschuster on 2012 March 17 18:07:15
-
I'd like to share a new script. Something like this may already exist, but I am new to PI and easily may have missed it.
PSFEstimation provides a simple way to estimate the point spread function (PSF) and other parameters of a target image.
Stars are detected automatically by StarAlignment's star detector. You can specify parameters to that process. The script generates a map of detected stars so you can double check your settings. The script then applies the DynamicPSF process to the detected stars and reports the medians and median deviations of the results.
In contrast to running DynamicPSF manually, you loose the ability to be selective in your choice of stars to fit, but you gain the ability to quickly analyze subexposures, calibrations, registrations and integrations.
Strictly speaking, the script should only be applied to linear images. But it does seem to provide some useful information for nonlinear images as well. RGB images are converted to Grayscale prior to PSF estimation.
When you click OK, the script runs its process and then opens a second dialog with the results, and optionally an additional window with visualizations. The result fields have tool tips that provide helpful information.
Comments welcome.
The script dialog.
(http://img191.imageshack.us/img191/5779/psfestimatordialog032.png)
Results dialog.
(http://img4.imageshack.us/img4/7200/psfestimatorresults032.png)
Visualization (generated when the Mosaic size parameter is 3 or larger):
(http://img440.imageshack.us/img440/6339/psfestimatorvisualizati.png)
Thanks,
Mike
Known bugs:
- Due to a previously reported issue, StarAlignment will abort PSFEstimation if it finds no stars. After clicking OK to dismiss the error dialog, try again with more permissive star detection parameters.
0.2 changes:
Added source view to dialog.
Added AvgDev eccentricity to results.
Decreased default detection sensitivity value.
0.3 changes:
Fixed non 32-bit floating point sample bug.
Increased limit on maximum number of stars fitted.
Changed fitted star map to Grayscale.
Improved performance.
0.4 changes:
Fixed zero fitted stars bug.
0.5 changes:
Increased default detection sensitivity value.
Fixed StarAlignment stars view id bug.
Cleanup code.
0.6 changes:
Added new instance button to parameters dialog.
0.7 changes:
Added new instance button to results dialogs.
0.8 changes:
Allow dialog to open with no active view.
0.9 changes:
Fixed fitted star map bug.
0.10 changes:
Fixed no target view bug.
Preserve STF in star map views.
Removed duplicate fits.
0.11 changes:
Added Save As.. CSV file button.
0.12 changes:
Fixed missing directory bug.
0.13 changes:
Handle 'Execute in the global context' command.
0.14 - 0.18 changes:
Many improvements.
0.19 - 0.28 changes:
Many improvements.
0.29 changes:
Fix MacOS dialog layout bugs.
Impose maximum per panel and total fitted star limits.
0.30 - 0.32 changes:
Use MedDev rather than AvgDev for robustness.
Set defaults to Moffat4 and Eccentricity.
Add MRS Noise and SNR Weight.
Remove Median (background) visualization.
0.33 changes:
Added per star PSF information to the .csv file.
-
Impressive! Do you feed the star locations to the PSF process or do you calculate the PSF yourself? I realize I could review your code but maybe you'll humor me and tell us :)
-
Oh yes I use DynamicPSF. I then calculate the median and avgdev across its output table myself. I was hoping to use its synthetic export to avoid having to do my own reduction but I can't find a way to force it to export via pjsr.
-
Excellent! I'll give it a try and let you know how it goes. Capturing some subs right now.
-
Another note of a script update above, I hope you don't mind again. There are a lot of bug fixes and new features, including the ability to create an icon that you can drag to a view for a quick analysis.
I am finding the tool useful for evaluating the star quality on subs. It provides a quick way to check the roundness of your stars across the field. Just create a preview of your area of interest and drag the icon into the preview. Then check the aspect ratio and eccentricity values. You can also evaluate star FWHM differences across the field easily. You can also save the result as a CSV file for documentation or post-processing purposes.
Some suggestions for usage. You will want to double check the detected star map to see just which stars are being detected. On noisier subs you may need to increase the sensitivity parameter to avoid false star detection on noise structures. On smoother integrations you may want to decrease sensitivity to help detect enough stars. The script fits at most 1000 of the detected stars, so if you have detected more than this number you can double check the fitted star map to see just which stars were fitted.
The script works best when there are a fair number of stars detected. The median values are usually robust and reliable. However, StarAlignment's star detector and DynamicPSF's star fitter can sometimes provide bad results on fields containing lots of noise or large scale nebular and galactic structures. If you see relatively large average deviation values try fitting regions that do not contain these structures.
Finally, this tool is intended only for quick analysis. If you need to be selective on the set of stars analyzed then please use the DynamicPSF process directly.
-
Here is more analysis of the collimation problem that motivated my work on this script. I divided the subexposure into a 5x5 grid and ran the script on each cell to detect stars and estimate PSF FWH, aspect ratio and azimuth parameters. I then used Mathematica to plot the arrays as height/vector fields. See the attachments. The contour plots turned out to be a nice way to visualize the data.
-
I made some improvements to the script. It now creates visualizations of FWHM and other parameters, so you can easily see how the PSF parameters vary across your subs and integrations. I posted the updates in this thread's head post.
As before the script uses StarAlignment for star detection and DynamicPSF for PSF fitting. The visualizations are created using Resample, PixelMath and custom scripts that do the layout and labeling. The visualizations are similar to the Mathemetica generated versions.
I recommend running the script with Mosaic size set to 1 first to verify that stars are being detected properly and that PSF medians for the whole view seem reasonable. You can modify the detection and fitting parameters if necessary. If things look good rerun with Mosaic size set to 5 or 7. You will then get the visualizations that show how the PSF parameters vary across your view. The labeling makes it easy to see what is going on.
Thanks,
Mike
PS: I have not figured out an easy way to visualize vector fields. I would like to match what Mathematica does for azimuth. If anybody has a suggestion please let me know.
-
Very good script, thank you. It is interesting to see the corelation between FWMH and gradient in the image.
I have attached an image with a (strongly stretched) gradient and the result of PSF estimator
Now I have to test if the fwmh measured after background equalization is better or worse... and understand the meaning.
-- bitli
-
The visualizations are created using Resample
Good job and excellent idea!
-
Thank you Mike. This a very useful script.
-
Hi Mike,
Very nice work. Can we distribute this script as an official one through the update system?
-
Thank you Juan, that would be great. Please go ahead.
Andres, NKV, bitli, thank you also for you kind comments.
NKV, yes, I was happy when I found that Resample's excellent results were exactly what I needed. Credit goes to PixelMath also, it is helpful to say the least. I had actually had started a custom implementation of all of this and then realized that these tools could do a better, faster job with much less work.
bitli I would like to hear your conclusions.
Mike
-
Hi Mike,
I am impressed by your slick programming and the result.
At the same time: In evaluating one sub the screen of my macbook is cluttered by the many windows this script is producing.
Is this the way to go or do you suggest another workflow to assess the starquality of many subs in one set?
I expect that your script have the potential to make my ccdinspectorsoftware (my current tool for subselection) obsolete.
But for now I must say that the ease of work of ccdinspector beats yours.
It would be nice if the script could pick the files from a given directory and produce some sort of list with scorings per sub.
I know ... I am ungrateful and selfish but since you are obviously a great javascriptguru I take my chance ... ;-)
But again, many thanks for the effort and result sofar: I will certainly use this script many times.
Hans
-
Hi Hans,
Thank you for your comments and feedback. I agree, reducing the number of windows and adding a workflow for multiple subs would be big improvements. What sort of scoring per sub would you find most useful? A basic one of course might be per sub FWHM information (median, minimum, maximum). Beyond that what would you like to see?
Thanks,
Mike
-
Hi Mike,
I agree: FWHM is essential.
The evaluation mechanism could be further enhanced by:
(1) aspect ratio (roundness)
(2) background level
(3 , optional) contrast (brightest star pixels versus background).
The ultimate prizewinning tool would be that the user is able to give these four criteria a weight. Something along the way of 1 being important and -1 unimportant. The script calculates an overall score. A maximum score of 100 represents the best theoretical values like when the FWHM equals the seeing, stars are completely round etc ...
The report ... maybe a list with columns: subnr/ evaluated total score / fwhm / aspectratio & weight / bckgrndlevel & weight / contrast & weight //
Hans
-
Thank you Hans. These things sound good and are doable I think.
Mike
-
I posted an update for Hans. It runs faster, has more options and generates fewer windows. It is still limited to a single image however, I hope to work on a batch version soon.
Thanks,
Mike
PS: I added a visualization for azimuth (DynamicPSF's theta parameter), the position angle of the median PSF FWHM major axis. The optics used for this example have a bit of astigmatism that shows up as a small elongation with varying position angle along on the left side of the frame. A visualization for background is new also.
-
Mike,
This special treatment is much appreciated.
... and testing ....
Hans
-
Let me add my thanks and admiration of this extremely useful script. I have wanted something like this to
characterize my camera lens behavior for a long time. I'm afraid I may now spend more time testing and
doing experiments than gathering data for an image!
One question tho ... what is the meaning background in this context?
-carl
-
Thank you Carl. Credit goes to the PI team for all their hard work on PI functionality.
Although Background is somewhat unrelated to PSF, I included it at Hans' request. It is the image's median value expressed in normalized real values from 0 to 1, scaled by the exponent given in the title text. When mosaic size is 1, background equals the Median value given by the Statistics process. The plot attempts to show how the median value varies across the image.
Background then is probably some sort of mix of sky flux, extended object structure flux, optical vignetting and reflections, ccd pattern noise, etc and their variations across the image. It may or may not be something that turns out to be useful.
If you can think of improvements, to background or anything else, please let me know. Hans' other suggestions, batching and some sort of image grading, are on my to do list.
Mike
-
Since you asked ... :D Would it be useful to take the results and make a synthetic PSF image like the
DynamicPSF process does? I know that can be used in the Deconvolution process, maybe others.
Ultimately, it seems that it would be possible to have a evaluation/measurement script like yours to
feed into a star "fixer" script (or deconvolution) to address problems by zone. I'm not suggesting this
for your script ... I'm just thinking out loud.
Thanks for the explanation of background. I'll have to puzzle a bit how I would use that information to
judge sub quality. I'm not at the point where I am tossing out subs anyway - all of mine are equally poor :)
-
Looking at the DynamicPSF script documentation, I don't see a way to export the synthetic PSF. Maybe a wish list item?
FYI something to check out: I found this an interesting paper on CCD calibration for the Cassini mission. Rather involved residual bulk image and flat field processing and a complex model function.
astro.cornell.edu/~mmhedman/papers_published/ISS_calibration_PSS.pdf (http://astro.cornell.edu/~mmhedman/papers_published/ISS_calibration_PSS.pdf)
all of mine are equally poor
I can relate. I tried to collimate a fast catadioptric recently. It is way worse now than what I started with. :-[
-
I posted a new version of this script, see the head post. Primary changes are (1) switched to median deviation (MedDev) instead of average deviation (AvgDev) for robustness, (2) added MRS noise and SNR weight values.
MSR noise is the same value produced by the Noise Evaluation script. It is a measure of noise in the image's structure free regions. To convert to electrons (e-) multiply this value by 65535 * gain. I like to compare this value to that of a dark with the same exposure.
SNR weight is the value (MedDev / MSRNoise)^2. SNR weight is an unnormalized estimate of the weight used by ImageIntegration's Noise Evaluation method. (ImageIntegration currently uses AvgDev clipped to (0.0, 0.98] rather than an unclipped MedDev). You can estimate II's weight via the ratio of the SNR weights of the target and reference. The intuition behind this formula is this (assuming equally exposed subs): Smaller MRSNoise values are better (less light pollution and air glow). Larger MedDev values are better (better transparency and contrast).
On the issue of sub evaluation, here is what I am doing now. All my evaluations are done on calibrated but unregistered subs.
I discard subs with relatively poor FWHM. What is poor? For this example set I consider a FWHM 10% or more larger than the best FWHM in the set poor. So subs 1 and 24 got discarded. (These are undersampled subs, hence the small FWHM in pixels.)
(http://img713.imageshack.us/img713/9416/calibratedsubdatafwhm.png)
I discard subs with poor Eccentricity. Collimation and tracking is usually good on my setup, eccentricities are usually smaller than 0.43, which is OK in my opinion. None got discarded.
(http://img15.imageshack.us/img15/328/calibratedsubdataeccent.png)
I discard subs with relatively poor SNR weight. Sub 4 is an obvious outliner (heavy smoke from forest fire), it got discarded. Actually all subs were smoke affected, but not too badly.
(http://img836.imageshack.us/img836/1007/calibratedsubdatasnrwei.png)
Finally, for integration weighting purposes, I use the SNR weights (or Noise Evaluation weights) of the calibrated but unregistered subs. This avoids weighting artifacts due to undersampled sub interpolation.
Regards,
Mike
-
Hi Mike,
First of all congratulations on this nice script. You're doing a really excellent work.
In second place, please forgive me for overlooking it, even after having asked you some time (months!) ago if we could release it as an official update. Now I have to ask you if we can include this script in the official 1.8 release :)
I don't see a way to export the synthetic PSF. Maybe a wish list item?
There's no way to export it from a DynamicPSF instance. However, the synthetic PSF is just a rescaled accumulation of a set of PSF fits. You can use the following snippet to perform the same operation in JavaScript:
/*
* Define the columns of the DynamicPSF.psf output table parameter.
*/
#define PSF_starIndex 0
#define PSF_function 1
#define PSF_circular 2
#define PSF_status 3
#define PSF_B 4
#define PSF_A 5
#define PSF_cx 6
#define PSF_cy 7
#define PSF_sx 8
#define PSF_sy 9
#define PSF_theta 10
#define PSF_beta 11
#define PSF_mad 12
/*
* Render all PSF fits in a DynamicPSF instance as a grayscale image.
*
* This function performs the same operation as the 'Export synthetic PSF'
* routine of the DynamicPSF interface.
*/
function renderSyntheticPSF( DPSF_instance )
{
var R = new Matrix;
for ( var i = 0; i < DPSF_instance.psf.length; ++i )
{
var psf = DPSF_instance.psf[i];
var r;
if ( psf[PSF_function] == DynamicPSF.prototype.Function_Gaussian )
r = Matrix.gaussianFilterBySize( psf[PSF_sx], 0.01, psf[PSF_sy]/psf[PSF_sx], Math.rad( psf[PSF_theta] ) );
else // Moffat PSF
r = Matrix.moffatFilterBySize( psf[PSF_sx], psf[PSF_beta], 0.01, psf[PSF_sy]/psf[PSF_sx], Math.rad( psf[PSF_theta] ) );
if ( R.rows < r.rows )
R.swap( r );
for ( var i0 = 0, k = (R.rows-r.rows)>>1, i = k; i0 < r.rows; ++i0, ++i )
for ( var j0 = 0, j = k; j0 < r.cols; ++j0, ++j )
R.at( i, j, R.at( i, j ) + r.at( i0, j0 ) );
}
var I = R.toImage();
I.rescale();
return I;
}
The renderSyntheticPSF function above returns an Image object that you can use in your script. It works with the list of PSF fits returned by a DynamicPSF instance as its psf output table parameter. However, the routine is very easy to adapt to other data structures, if necessary. You can also use goodness-of-fit values (the PSF_mad column) to filter out bad fits. Disclaimer: I have not tested the code, but it should work without problems. Let me know if you find it useful.
-
Thank you Juan. I will try your PSF rendering code with a bad fit filter. Please feel free to use the script. I will put together a testing project for you like bitli did for VaryParams when I return from my dark sky week.
A follow-up question on II Noise Evaluation weights: does the MRSNoise used by this weighting clip to (0, 0.98] like ADev?
Mike
-
Revisiting your great script ... wonderful additions! Silly question in case I'm overlooking something. On your sub evaluation exercise did you need to run the script on each sub (separately or in a process container) and write down/copy/cut the values and then plot outside PI? Or is there a way to work with all your subs and generate a tabulation for easy observation/graphing?
-carl
-
Hi Mike,
Sorry to pile on the questions, but I'm experimenting with using your script's output as the *input* for Deconvolution. The PI Deconvolution module has an input slider for the `Shape' of the PSF. They say this is the kurtosis of the PSF. Apparently the kurtosis of a Gaussian model PSF is 2.0.
What is the kurtosis of a different model fit, such as a Moffatt4 model? Should the `Shape' slider be set to 4 in that case?
- Marek
-
Thank you Carl. The script only does one image and I do copy values. However, I am working on a new batch version. I hope to have it ready by the end of October.
Hi Marek, I don't know. It may be that the Moffats can only be approximated, but what value is best I don't know. I have not Deconed so I have not faced this issue.
Mike
-
Awesome! I look forward to it. The script is incredibly useful as it. Batch functionality will bring it to
another level. Another experiment I'll be looking forward to trying is learning how star sizes and shapes
vary through focus and f ratio (I image with camera lenses). You can see why I might be interested
in the batch feature :P
-carl
-
Mike, is the version 0.33 download the latest (from the link in the first post of the thread)?
Dave
-
Yes Dave, 0.33 is the latest, thanks.
Mike
-
Here is a link to a batch version of the script.
https://dl.dropbox.com/u/109232477/SubframeSelector.0.4.zip (https://dl.dropbox.com/u/109232477/SubframeSelector.0.4.zip)
You can add the script to the menu by selecting the src directory in Feature Scripts, or choose SubframeSelector.0.4.js in Execute Script File. You will find documentation in the doc/scripts/SubframeSelector.0.4 directory, open the html file in your browser. The tool tip of the info icon at the bottom of the script's dialog has helpful information. You can also install the documentation by running the complier on the pidoc file, but for some unknown reason the resulting documentation is not available from within PixInsight, so please open the html file.
The script presents subframe measurements in a table and in plots. You can also save the measurements in a csv file for postprocessing. The script also has features for subframe approval and weighting, these are optional.
Thank you Hans Pleijsier, Georg Viehoever and Juan for your help and code.
Thanks,
Mike
(http://imageshack.us/a/img132/7228/subframeselector04.png)
-
Mike, it is a fantastic scrip. Thanks you! :)
I have installed it and works perfect. I tried it a set of 94 images from differents days and I could reject the images that are far form the media in FWHM and SNRWeight and now i am trying to understand how to determinate the best images to use in StarAlignment. Thanks for the documentation, it is very clear.
Saludos. Alejandro.
-
Alejandro thank you for the comments. :)
It is safe now to install the documentation. I made a change a while ago to fix a typo and did not realize that this change also fixed the doc install problem.
Run Scripts>Development>Documentation Compiler, click Add Files, select SubframeSelector.0.4.pidoc in the script's pidoc directory and click OK. Now docs will be avaliable in Process Explorer>Scripts>Batch Processing>SubframeSelector.0.4. (Note that you can't access Process Explorer when the script is running due to the modal nature of PJSR.)
Mike
-
Hi Mike, installed perfectly. Thanks again!
Saludos, Alejandro.
-
Hi Mike
Script seems to work fine, but when I try to install the documentation I get an "Access Denied" message. Any suggestions?
Geoff
-
Geoff, maybe an admin account premissions issue? Can you run PI as an administrator?
Mike
-
Geoff, maybe an admin account premissions issue? Can you run PI as an administrator?
Mike
That works.
Thanks Mike
-
Hi,
I'm just testing now this tool. I've found it would be very useful to set a cropping parameter for all the images. Usually you need only a small crop of all the frames to decide which are the best ones. In my case, I have about 500 frames and each one is a 16-megapixel image. So processing the images at original size is not a good idea... Now I'm cropping the images to 512x1536 with an image container, just after processing them. But this doesn't allow to automatically rename the files, of course.
Regards,
Vicent.
-
Another idea. Could it be possible to automate this script with a watch dog pointing to a folder? The idea is to have the script running and, every time there's a new image, measure the PSF and update a text file in the hard drive with the PSF readings.
Regards,
Vicent.
-
A plot a just made from the 541 image data set:
(http://astrofoto.es/astrofoto/foros/SubframeSelector_plots_FWHM.png)
Could it be possible to choose the graph size in pixel?
Thank you,
Vicent.
-
Wow. Cropping is a good idea, I will add this. Also, I will add something Juan suggested that will help performance.
Yes, a graph size is needed and maybe a scroll bar also. I will add this. I will think about a watch dog too.
Thank you,
Mike
-
Thank you,
A watch dog would be very useful for real-time monitoring of your image session. Also, for professional observatories, this could be useful to compare image seeing to seeing monitor measurements. The trick is to output a text document to let the user to graph himself the values (in Linux with GNUPlot, for example).
Regards,
Vicent.
-
holy smokes, this is great, thanks for the batch version. also i just realized the original version is something of a replacement for CCDInspector.
i hate to make feature requests but i wonder if the script could move bad frames into a different directory rather than copying good frames? unless something's gone horribly wrong i usually only have a few bad frames. this would save on disk space. or maybe it already does this and i did not notice.
-
Thanks pfile.
I will add options so you will be able to copy/move the good/bad frames in any combination that you want. I hope to post a new version at the end of this month that will include your idea and also Vicent's requests, or at least as many of them as I have time for.
Thanks,
Mike
-
Here is a new version of the script with some of pfile's and Vicent's suggestions. Install it like before, use Feature Scripts to remove the old version. pfile, you can move or copy files to different directories. Vicent, you can specify a rectangular measurement area, plot legibility is better, no watch dog.
Note: All subframes should be observations of the same target. Small variations in targeting are acceptable, such as those due to dithering and meridian flipping. Larger variations in targeting will result in incomparable property measurements. Add at most several hundred subframes. Adding more than that may result in poor performance of the script.
Thanks,
Mike
https://dl.dropbox.com/u/109232477/SubframeSelector.0.5.zip (https://dl.dropbox.com/u/109232477/SubframeSelector.0.5.zip)
A variation: Rather than replacing a ringing pixel with its bilinear equivalent, it seems better to blend the two pixels. This maintains more "sharpness". With this tweak FWHM is even smaller.
-
Hi Mike,
Just tested the latest version and it worked flawlessly, thank you very much. I've noted that cropping the images to 0.15 megapixels is not faster than cropping to 1.5; anyway, it's much faster than working with 16 mp images!
I tested also how the registering process affects the FWHM values. In the image set the stars where always 2 - 5 pixel wide. The graph below shows the relation of FWHM values between the images before and after the registering process:
(http://astrofoto.es/astrofoto/foros/FWHM.png)
The result is very good. The green line represents the 1 to 1 ratio. The red line is a quadratic fit of the FWHM value data set. As you see, the registering process increases the FWHM values when the images are sharper. IMO, this weighting must be calculated after the registering if your typical FWHM are below 2 pixels.
Best regards,
Vicent.
-
Hi again,
Another graph to compare Lanczos-3 vs Bicubic Spline. In this one I compare the FWHM of the original images versus the FWHM values after registering with Bicubic Spline interpolation:
(http://astrofoto.es/astrofoto/foros/FWHM_spline.png)
Regards,
Vicent.
-
thanks so much for hacking in my suggestion, i appreciate it.
i am debugging collimation issues so this script is timely.
-
Thanks pfile and Vicent.
Vicent, here are data for my 4.2 arcsec/pixel subs. The increase in FWHM is about 0.6/0.9 arcsec. MRSNoise of registered subs whose FWHM approaches 1 pixel is a problem. Basically NoiseEvaluation weights of registered subs in ImageIntegration are random numbers. This is the reason why I prefer non-registered measurements. As a workaround I record NoiseEvaluation weights of calibrated, non-registered subs in FITS headers, carry these values through registration and use them in ImageIntegration.
Thanks,
Mike
(http://imageshack.us/a/img203/8339/fwhm.png)
(http://imageshack.us/a/img42/3221/mrsnoise.png)
-
Hi,
Yes, of course, the noise weighting is far more accurate if done before registration. This is specially true if there is a slight rotation of the images because of moire effects, even with Lanczos.
One suggestion: Try Lanczos 4 or 5 to see what happens with your FWHM values.
In your system seeing effects are negligible (if not under very poor conditions). So, once you include the watch dog, this could be useful for focus monitoring while acquiring data.
Regards,
Vicent.
-
Lanczos 4 rings worse than Lanczos 3, FWHMs are nearly equal. I didn't try 5, IMO it will be even worse.
Mike
-
Hi,
I was trying to run the script and generate the text output. The output I got contained somehow strange values, so I tried to run DynamicPSF manually using the same stars. It appears, that manual run gave different output than script. Any idea what happened? The values are totally different. For example script gave Sx=9.89 and manual run Sx=1.46 (see screenshot).
milosz
-
Hi milosz,
Yes, I understand what has happened. I believe you ran the script a second time with the fitted star map as target rather than the original image as target. The fitted star map image contains the cross marks, and trying to fit those marks gives the completely nonsensical results. Please make sure that your original image is the active target and rerun the script. This is an easy mistake to make, running with the wrong target, when there are many open windows in the workspace.
You might also try the batch version of the script. It runs on a set of files and so it is much less likely that mistakes like this happen.
Thanks for posting, sorry for the trouble,
Mike
-
Mike,
You are perfectly right. My mistake. The name of the image is incorporated in the file name with the results :) I ran it again and it gave good results.
There is one improvement you can do in the script. Please add the column names on top of the results. It helps reading it. I had to dig into the code to learn which columns represents which value.
milosz
-
Hi Milosz,
Column names are there just above Star #1. But probably too terse and too easy to miss.
Mike
-
Mike,
I don't see the column names there:
Target view,broom_ha_600s2
Star detection layers,4
Noise reduction layers,2
Hot pixel filter radius (px),1
Log(detection sensitivity),0.00
Star peak response,0.80
Maximum star distortion,0.50
Generate detected stars map,false
Model function,Moffat4
Circular point spread function,false
Scale mode,Pixels
Elongation mode,Eccentricity
Generate fitted stars map,false
Mosaic size,1
Generate visualization,false
Aggregate visualization,false
Generate tabulation,true
Stars fitted,829
Star,1,0.018834,1.387844,5.21,315.79,1.30,1.22,0.933,16.33,1.346e-003
...
Do I have an outdated version of the script? I'm using 0.33.
I also don't see them in the code itself:
file.outTextLn("Stars fitted," +
rowColumn + format("%d", psfEstimates[0][i][PSFEstimate_StarsFitted]));
if (psfEstimates[0][i][PSFEstimate_StarsFitted] != 0) {
if (parameters.circularPSF) {
for (var j = 0; j != psfEstimates[1][i][Barycenters_Points].length; ++j) {
file.outTextLn(
"Star," +
rowColumn +
format("%d,", j + 1) +
format("%.6f,", psfEstimates[1][i][Barycenters_Points][j][PSF_B]) +
format("%.6f,", psfEstimates[1][i][Barycenters_Points][j][PSF_A]) +
format("%.2f,", psfEstimates[1][i][Barycenters_Points][j][PSF_X]) +
format("%.2f,", psfEstimates[1][i][Barycenters_Points][j][PSF_Y]) +
format("%.2f,", scaleModeUnits[0] * FWHMPerStdDev[parameters.modelFunction] *
psfEstimates[1][i][Barycenters_Points][j][PSF_Sx]) +
format("%.3e", psfEstimates[1][i][Barycenters_Points][j][PSF_MAD])
);
}
I mean something like
file.outTextLn("Stars fitted," +
rowColumn + format("%d", psfEstimates[0][i][PSFEstimate_StarsFitted]));
if (psfEstimates[0][i][PSFEstimate_StarsFitted] != 0) {
if (parameters.circularPSF) {
file.outTextLn("Name,No,B,A,X,Y,Sx,MAD");
for (var j = 0; j != psfEstimates[1][i][Barycenters_Points].length; ++j) {
file.outTextLn(
"Star," +
rowColumn +
format("%d,", j + 1) +
format("%.6f,", psfEstimates[1][i][Barycenters_Points][j][PSF_B]) +
format("%.6f,", psfEstimates[1][i][Barycenters_Points][j][PSF_A]) +
format("%.2f,", psfEstimates[1][i][Barycenters_Points][j][PSF_X]) +
format("%.2f,", psfEstimates[1][i][Barycenters_Points][j][PSF_Y]) +
format("%.2f,", scaleModeUnits[0] * FWHMPerStdDev[parameters.modelFunction] *
psfEstimates[1][i][Barycenters_Points][j][PSF_Sx]) +
format("%.3e", psfEstimates[1][i][Barycenters_Points][j][PSF_MAD])
);
}
I would also remove all the summaries from the CSV. IMO CSV should contain only the data for the fitter stars + column names.
milosz
-
Oh, sorry. I added the column labels in the batch script output and forgot that I had not in this script. 0.33 is the latest. I like having the summary to help document the data, but please go ahead and delete that code if you wish.
Mike
-
Dear Mike,
Hi - I have been using both your PSFEstimator and Subframeselector scripts for some time - they are fantastic.
I've just upgraded to Ripley RC3, and find that both produce the same error. The scripts run, identify a number of stars using the star alignment module, but then were unable to detect the stars in the StarDetection part of the algorithm.
I had a play with the PSFEstimator script. Very ingenious - you convolved the cross pattern that the StarAlignment uses with the same pattern, and where you get a 1, that's the star centre.
I took a look at the images generated by the StarAlignment map - they now generate the cross pattern with a value of 0.5. so you now get a 0.5, rather than a 1 post convolution, and hence the binarize(1.0) does not pick out the stars.
I set the binarize threshold to 0.5, and this does now work, but I can see cases where you might get eroneous stars (if the value was > 0.5 over a space of 11 pixels). Not sure if there is a work around, other than to ask Juan to set the value to 1.0 from the cross from the StarAlignment module?
Colin
-
Hi Mike
Your PSFEstimation was really nice under PI 1.7
I used it many time and most when you have launch SubFrameSelector.
Under PI 1.8, the PSFEstimator returns ZERO because it doesn't find stars.
If you could work on it or see if Juan can help you
Also our member BITLI could help you too !
;)
-
FWIW I have the same issue with PSFEstiation in 1.8RC5 (Win 7 64 bit).
The new aperture photometry script also fails with zero stars detected so maybe there is a common root cause.
Regards
Derrick
-
Hi Derrick,
Which are your configuration parameters for detecting stars?
Thanks,
Vicent.
-
Hi Vicent,
Thanks for responding, I have tried various settings witout success; here is a copy of the Process console output that includes the star detection parameters for a failed attempt:
Reading image:
D:/WIP/M81 Feb13/PI_Cal/calibrated/light/M81-003L_c.fit
Reading FITS: 32-bit floating point, 1 channel(s), 3362x2537 pixels: done
run --execute-mode=auto "C:/Program Files/PixInsight/src/scripts/PSFEstimator.0.33.js"
Processing script file: C:/Program Files/PixInsight/src/scripts/PSFEstimator.0.33.js
PSFEstimator 0.33:
Target view: M81_003L_c
Star detection layers: 5
Noise reduction layers: 2
Hot pixel filter radius: 1px
Log(detection sensitivity): -0.20
Star peak response: 0.90
Maximum star distortion: 0.50
Generate detected stars map: false
Model function: Moffat4
Circular point spread function: false
Scale mode: Pixels
Elongation mode: Aspect ratio
Generate fitted stars map: false
Mosaic size: 1
Generate visualization: false
Aggregate visualization: false
Generate tabulation: true
StarAlignment: Processing view: M81_003L_c_target
M81_003L_c_target:
521 stars found.
4.165 s
StarDetection: Processing view: M81_003L_c_target_stars
0 stars detected.
4.243 s
PixelMath: Processing view: M81_003L_c_target
Calculating view statistics...
Solved 1 invariant subexpression(s).
Executing PixelMath expression: combined RGB/K channels: abs($target - med($target)): done
0.687 s
DynamicPSF: Global context
1 view(s)
0 star(s)
0 PSF fittings
0.171 s
PSFEstimation: Processing view: M81_003L_c
Model function: Moffat4
Stars fitted: 0
0.000 s
Warning: One or more mosaic panel has fewer than 30 fitted stars.
I'm more than happy to try any suggestions you might have to help understand what might cause this problem.
Regards
Derrick
-
hi Derrick
I don't have any problem with this AperturePhotometry script. It seems working fine.
So, your problem could not be linked to the PSFEstimation script (not present in RC5).
-
Hi Philippe,
I resolved my issue with the Aperture photometry script by changing some parameters.
I downloaded the PSFEstimation script v0.33 from the first page of this thread and loaded it in RC5.
Derrick
-
Dear Mike,
Hi - I have been using both your PSFEstimator and Subframeselector scripts for some time - they are fantastic.
I've just upgraded to Ripley RC3, and find that both produce the same error. The scripts run, identify a number of stars using the star alignment module, but then were unable to detect the stars in the StarDetection part of the algorithm.
I had a play with the PSFEstimator script. Very ingenious - you convolved the cross pattern that the StarAlignment uses with the same pattern, and where you get a 1, that's the star centre.
I took a look at the images generated by the StarAlignment map - they now generate the cross pattern with a value of 0.5. so you now get a 0.5, rather than a 1 post convolution, and hence the binarize(1.0) does not pick out the stars.
I set the binarize threshold to 0.5, and this does now work, but I can see cases where you might get eroneous stars (if the value was > 0.5 over a space of 11 pixels). Not sure if there is a work around, other than to ask Juan to set the value to 1.0 from the cross from the StarAlignment module?
Colin
The issue is a change in the StarAlignment module. The map now returns a value of 0.5 rather than 1.
As a work around, try editing the script and look for binarize. Set the value to 0.5 rather than 1, and it will work. There could be cases of false stars detected, but this will at least get you going.
Colin
-
Enclosed script with change to Binarize setting which makes it work with new star alignment module.
Colin
-
Thanks Colin, the binarize may sometimes create false positives, many will be rejected out of hand by DPSF, and the script's median processing will usually eliminate the influence of the remainder.
Thanks,
Mike
-
Yeeeeeessssssss !!!!
Working again !!! thaaaaannnnnnnks
Maybe the SubFrameSelector will work too if we replace this script ?
-
Hi Mike,
As promised long time ago, attached to this post is a JavaScript version of the star detection routines used in the StarAlignment tool. With this code you no longer need StarAlignment for star detection, and your scripts will be more robust and independent.
The code is documented. The main object is StarDetector, whose constructor is:
new StarDetector( [int structureLayers[,
int noiseLayers[,
int hotPixelFilterRadius[,
Number sensitivity[,
Number peakResponse[,
Number maxDistortion[,
Boolean invert[,
Image mask]]]]]]] )
where the function parameters are similar to the corresponding StarAlignment parameters:
structureLayers
Number of wavelet layers for structure detection (default=5)
noiseLayers
Number of small-scale wavelet layers for noise suppression (default=1)
hotPixelFilterRadius
Half size in pixels of a morphological median filter used for hot pixel removal (default=1).
sensitivity
Sensitivity of the star detector device. Smaller values provide more sensitivity (default=0.1).
peakResponse
Peak response of the star detector device. Larger values are more tolerant with relatively flat structures (default=0.75)
maxDistortion
Maximum distortion allowed, relative to a perfect square. The distortion of a perfect circle is pi/4 (default=0.5)
invert
Detect dark structures over a bright background, instead of bright structures over a dark background (default=false)
mask
Optional mask image. If defined, star detection will be restricted to nonzero mask pixels (default=undefined)
StarDetector's main method is:
Array StarDetector.stars( Image image )
which returns an array of Star objects (also defined in StarDetector.js). Note that star coordinates are referred to a coordinate system where the center of the top-left pixel is {0.5,0.5}. This is different from StarAlignment's coordinate system, which refers coordinates to {0,0}.
If you uncomment the first line of StarDetector.js, you can execute the script on the active image to generate a copy with a red overlay where all detected stars are plotted as circles.
There's only one limitation: Unfortunately, the method Image.morphologicalTransformation() is unusable in versions 1.8.0 RCx, due to a regression bug. This means that the hot pixel removal functionality, which implements a median filter, is not available for now (see the relevant code at lines 170-179 of StarDetector.js). Of course this bug will be fixed in the final 1.8.0 release. Other than this problem, the star detector is fully operational and reasonably fast. Let me know if you find it useful.
-
Thank you Juan! I am integrating now.
Mike
-
Here's a new version of the StarDetector object:
- The structureLayers parameter has now a default value of 4 wavelet layers instead of 5. This is for coherence with the StarAlignment tool, and mainly because according to our experience, 4 layers seems the best option for most DS images.
- The StarDetector object has a new parameter: progressCallback. This is an optional function that will be invoked by the star detection task (the stars() method), so you can provide feedback to the user and make the task abortable. This is fully documented in the source code, and you have an example at the end of the StarDetector.js file, which you can see in action if you #define the __TEST__ macro.
-
Hi Juan,
Sorry for the delay in replying.
StarDetector and StarAlignment differ on results and performance more than desired.
I will study your code and look for possible improvements.
Data for RC6 Win7 with and without hot pixel filter enabled.
Mike
All runs with the same parameters for both StarDetector and StarAlignment:
var starDetector = new StarDetector(
4, // structureLayers
2, // noiseLayers,
1, // hotPixelFilterRadius
Math.pow(10.0, -1.0), // sensitivity
0.8, // peakResponse
0.5 // maxStarDistortion
);
var stars = starDetector.stars(imageWindow.mainView.image);
Subs are relatively small, 2 megapixels.
StarAlignment:
Sub #1, 1514 stars found, 1.6 seconds
Sub #2, 1632 stars found, 1.5 seconds
StarDetector script, hot pixel filter disabled:
Sub #1, 1108 stars found, 14.3 seconds
Sub #2, 1054 stars found, 13.4 seconds
StarDetector script, hot pixel filter enabled:
Sub #1, 808 stars found, 9.8 seconds
Sub #2, 865 stars found, 9.7 seconds
Note: The StarAlignment performance numbers do not include the time my old script requires to convolve and peak detect the star map (equals 3 seconds on these subs). With this overhead included, StarDetector is about 2-3x slower. This performance is loss is possibly OK, but I need to figure out why fewer stars are detected.
-
One more note:
Although StarDetector detects fewer stars overall, some of them were not detected by StarAlignment. So these schemes seem to be working "differently" somehow. By looking at which stars were detected by which, it is not clear to me what is going on.
Mike
-
does this script not yet exist for 1.8? or is my 1.8 installation broken? because i don't see PSFEstimation, or SubframeSelector in the scripts menu, and regenerating the scripts did not fix anything.
rob
-
or maybe this was never released officially? i just can't remember.
-
And the wait became too long...Too bad my demo license expired yesterday - I guess PixInsight won't figure in my future :-[ Automation is a very important part of my automated observatory and these scripts were part of the solution.
Cheers
-
Here is a release of PSFEstimator for 1.8. Sorry for the delay.
https://dl.dropboxusercontent.com/u/109232477/PixInsight/PSFEstimator.0.34.js (https://dl.dropboxusercontent.com/u/109232477/PixInsight/PSFEstimator.0.34.js)
The change uses Juan's and Collin's convolution threshold suggestion, along with an additional false positive rejection technique. Also some minor 1.8 dialog styling changes.
You may see slightly different sets of stars selected for the analysis, this is to be expected. The results should match 1.7 closely. The script continues to use the StarDetector process rather than Juan's star detector script.
Tested on Win7 and MacOSX 10.7.
SubframeSelector is not yet ready.
Thanks,
Mike
-
thanks Mike.
-
Great. Tnx, Mike-
-Jeff
-
many thank, Mike
Cheers
-
Here is a release of SubframeSelector for 1.8. Again, sorry for the delay.
https://dl.dropboxusercontent.com/u/109232477/PixInsight/SubframeSelector.0.6.zip (https://dl.dropboxusercontent.com/u/109232477/PixInsight/SubframeSelector.0.6.zip)
Changes are similar to those for PSFEstimator mentioned above, along with a Tabulation/Plot/SaveAs file naming bug fix. Tested on Win7 and MacOSX 10.7.
To install the script, first unzip the downloaded file, choose Scripts>Feature Scripts..., click Add, navigate to SubframeSelector.0.6/src and click Select Folder. The script should appear in Scripts>Batch Processing.
To read documentation, open SubframeSelector.0.6/doc/scripts/SubframeSelector.0.6.html in your browser.
You may also install the documentation into PixInsight. Choose Scripts>Development>Documentation Compiler, click Add Files, and add SubframeSelector.0.6/pidoc/SubframeSelector.0.6.pidoc. Click OK and the file and its assets will be compiled and installed in PixInsight's system install directory.
Once installed, you may view the documentation within PixInsight via Process Explorer>Scripts>Batch Processing>SubframeSelector.0.6. Make sure the Show/Hide Extensions button at the bottom of the explorer window is configured to show the documentation.
Thanks,
Mike
-
Can these be released via Juans update mechanism, or are they still experimental?
-
Georg, not ready for official release.
Mike
-
Here is another script that measures FWHM on a single sub. It is easier to use than PSFEstimator. It produces a nice, simple table and nice plots. I use it more often than PSFEstimator now.
https://dl.dropboxusercontent.com/u/109232477/PixInsight/FWHMEccentricity.0.4.js (https://dl.dropboxusercontent.com/u/109232477/PixInsight/FWHMEccentricity.0.4.js)
Select your view, click Measure to make the measurements, and click Support to see the plots. You will see values for star FWHM and Eccentricity. Round stars have eccentricities less than about 0.44, much larger than that and they will start looking oval.
You can choose your star profile function and detection sensitivity. The defaults work well on my subs.
Tested with PixInsight 1.8RC7 on Win7 and MacOSX 10.7.
Thank,
Mike
-
Hi Mike
That's right, this new one is simple and very useful ;)
Thank you very much for sharing !
-
Thanks Philippe.
As a companion to FWHMEccentricity, here is another new script that estimates a sub's contrast, noise and their ratio. The script focuses on trying to get a more robust estimate of background noise and gives a contrast-noise-ratio for sub quality evaluation purposes. I have had difficulty getting reliable noise estimates for my binned, narrowband subs, hence this script.
https://dl.dropboxusercontent.com/u/109232477/PixInsight/ContrastBackgroundNoiseRatio.0.4.js (https://dl.dropboxusercontent.com/u/109232477/PixInsight/ContrastBackgroundNoiseRatio.0.4.js)
Choose your sub, click Measure to see the measurements, and click Support to get a bicolor mask. By default, the script measures noise in only the dimmest 2 percent of the sub. You can select a different percentile.
The resulting noise measurement will typically be a composite of noise from several sources, including read noise, dark current noise, shot noise, confusion noise (i.e. noise from unresolved stars and galaxies), and also fixed pattern noise if the sub has not been calibrated. The script uses simple, robust statistical methods to eliminate contributions from object structures and cosmic rays to get a more reliable noise estimate. By restricting measurements to the dimmest percentile of intensity, the impact of the strongly intensity dependent shot noise contribution to the noise estimate is controlled.
The bicolor mask shows areas of the sub used for the measurement. You can composite this mask with the sub to better see these areas as follows. First stretch your sub, then rename the bicolor mask "support", then apply to the stretched sub the PixelMath "$target + 0.5 * support" with the options rescale off, create new image on and color space set to RGB.
Tested with PixInsight 1.8RC7 on Win7 and MacOSX 10.7.
Thanks,
Mike
-
I'm kind of embarrassed to ask this. I've been reading about PSF for ages, but have no idea what it is used for. From reading this thread it seems that the script is powerful and useful. Trouble is, I have no idea what for. :-[
Could it be used for star shaping? eg I have some rotation in my field around the guide star due to slightly off polar alignment. Could this be used to correct that?
Sorry for the dumb question.
-
it's kind of for testing your OTA/camera combination to see if your OTA is in collimation and your camera is not tilted. the shape of the stars is pretty much the Point Spread Function for your optical train, which is a measure of how the optics distort a point source of light.
CCDInspector is a commercial product which does something similar to the mosaic mode of PSFEstimation, you can check that out too.
the related script SubframeSelector measures the average PSF and a whole host of ancillary statistics about the PSFs across images so that you can judge which subexposures are good. it's a bit more objective than eyeballing them.
-
Hi Mike,
I have attached a new version of StarDetector.js to this post. This is an adaptation of the star detection routine that I have written for the next generation of the StarAlignment tool, which I'm going to release in a few days.
This version has the following main features:
* Much faster. You can use it now without performance problems in your scripts. Example 1: 1800x1300 image, 3100 stars detected in 1.40 seconds. Example 2: 4000x5000 image, 40,000 stars detected in 8.3 seconds. Benchmarked on a 6-core i7 990X workstation.
* More robust detection. The new algorithm is much more robust to detection of false star structures (such as small nebular features) and multiple/crowded stars.
* More accurate. The algorithm converges to valid centroid coordinates under difficult conditions such as nearby stars or nebulae, and diffraction spikes.
* The new detection algorithm is invariant to scale. This means that the same stars are now detected consistently for the same image, on small crops or on the whole image. This means also that the same stars are now detected for the same image under a wide range of scaling factors. Scale dependency was a severe limitation of previous StarAlignment versions.
The script now generates better control images (when the __TEST__ macro is #defined to a value > 0). In particular, the script generates a star mask with circles showing the dimensions of the detected star structures. Just select the star mask as a mask for the original image and apply an automatic screen stretch as necessary. The stars will be shown as red circles.
The algorithm no longer uses wavelet transforms to isolate structures (as this was the cause of the scale dependency that I have commented above). However, for the sake of compatibility, the structureLayers and noiseLayers parameters are still expressed as wavelet layers. I have written the structure detection phase in a way that mimics the wavelet-based implementation. Despite that, the structureLayers and noiseLayers parameters are much less sensitive in this version. This means that the difference between 4, 5 and 6 detection layers is not really important for most images, which is a good property because it contributes to the scale invariance of the algorithm.
Let me know if it helps, and keep up the nice work!
-
Thank you so much Juan, I will test! This sounds like a big improvement over the older script which I decided not to use due to its results and performance.
Mike
-
Troy,
Star diameter appears to vary with star brightness, the FWHM metric on the other hand, because it normalizes flux profiles, tends to give the same number independent of brightness.
Here is the result of my FWHMEccentricity script on a sub taken last year.
(http://mschuster.zenfolio.com/img/s9/v96/p1841781371.png)
The median FWHM is about 1 pixel, which means that one half of a typical star's flux is concentrated within a diameter of 1 pixel. The sub was binned 2x2 with a fairly coarse image scale of 4.2 arcseconds per pixel. Seeing, focus, tracking and collimation were all good, which accounts for the small FWHM value.
Median Eccentricity is a measure of typical star shape, the degree of its roundness or ovalness. Values less than 0.44 are usually considered round.
Of course, FWHM and Eccentricity may vary across the sub due to optical aberrations, tilt, field rotation and what not. The script produces plots to help you see this. You can see that FWHM was slightly higher in two corners.
(http://mschuster.zenfolio.com/img/s8/v78/p1662823045.tif)
(http://mschuster.zenfolio.com/img/s10/v110/p1674531263.tif)
The script also produces a plot that shows which stars were measured, and also how well the chosen model function fit the star's profile, from negative (better than median fit), to 0 (median fit) to positive (worse than median fit).
(http://mschuster.zenfolio.com/img/s9/v97/p1612977684.tif)
The other scripts do similar things, one of them is a batch version, and they also provide information about noise and contrast. For star measurements on single sub I prefer FWHMEccentricity due to its simplicity.
Thanks,
Mike
-
Hi Juan,
Your new script is working extremely well so far. Thank you!
I concur, faster than existing SA and much faster than the old SD script, and better at finding stars than existing SA and much better in this regard to old SD.
I concur, robustness appears superior, but have not tested sufficiently. Have not yet tested scale invariance.
I concur, layer parameters are much less sensitive. For the time being I will continue to use 4/2 as with existing SA. (FYI: defaults are actually 5/0 but comments say defaults are 4/1).
With 4/2 script detects more stars (1.25x to 2x more) in less time than SA on certain subs.
I am testing more now. So far no problems.
Thanks,
Mike
-
Here is an update to the script FWHMEccentricity, which measures subexposure star shapes. It includes Juan's new star detector that he describes above. This new detector is a significant improvement in results, robustness and performance. The script also includes Juan's script programming recommendations (no with statements, no unrooted controls, and no restyle() calls). Tested for 1.8 RC7 on Win7 and MacOSX 10.7. The script produces results slightly different than earlier versions, this is to be expected due to the differences in star detection.
Thanks,
Mike
https://dl.dropboxusercontent.com/u/109232477/PixInsight/FWHMEccentricity.0.6.js (https://dl.dropboxusercontent.com/u/109232477/PixInsight/FWHMEccentricity.0.6.js)
-
Thanks for the reply, Mike. If I understand correctly from your post, and also some PM correspondence as well, it's possible that if your script estimates the PSF for all stars over the image, a script to deconvolve using it could be developed to correct star profile shapes?
Say you've used a star in the bottom right of your image and there's a slight field rotation. Stars top left show the worst effects of circumferential lengthening about the guide star. If your PSF script can represent what's going on mathematically, a script could be written to correct the rotation and tighten the stars up? Might be like a gradient correction because the most needing it would be top left while bottom right would need nominal correction?
-
Hi Troy, yes, spatial varying deconvolution is possible. I have implemented one technique (see reference) in Matlab. It is work in progress. With the current version on my undersampled subs, I like very much how it enhances subtle, low contrast nebula detail, but I don't like the ringing on the small stars. I don't yet have a solution for this problem.
Thanks,
Mike
Oliveira, et al., "Adaptive Total Variation Image Deconvolution: A Majorization-Minimization Approach", Signal Processing, vol. 89, issue 9, pp. 1683-1693, Sep 2009.
-
Mike,
This script looks fantastic. I tried to download it today, and the dropbox link seems to be broken. Is there an alternative way to obtain it?
-Scott
-
Hi Scott,
Please see this post (http://pixinsight.com/forum/index.php?topic=5712.0).
Please try the script FWHMEccentricity. It is an easy to use replacement for PSFEstimator. The batch version SubframeSelector is also available from the post.
Please also be sure that your 1.8 RC7 version is upgraded to the most recent version.
Thanks,
Mike
-
Any way to still download this script somewhere? The dropbox link isn't working (anymore).
-
Hi Chris, please try the script FWHMEccentricity, which is now available as part of the standard PI 1.8 update mechanism. PSFEstimation is obsolete.
Thanks,
Mike
-
Excuse my ignorance, but where should I be able to find it? I don't have it listed in my scripts overview?
-
Chris,
Please first try Resources > Updates > Check for Updates and update if necessary.
The script requires 1.8 RC7 Version 01.08.00.1022 or later so the update is important.
The script should appear as Scripts > Image Analysis > FWHMEccentricity.
If not there, please download it from the list in this post (http://pixinsight.com/forum/index.php?topic=5712.0).
Install the download via Scripts > Feature Scripts... > Add. Select the folder containing the download. After an OK it should appear as above.
If this doesn't work, please let me know!
Regards,
Mike
-
Thanks Mike, I needed to install updates. This is a new install of the latest version of PI, so I wrongly assumed it was up to date :)
-
First of all, thanks for making this incredible tool avalaible for all.
On the other hand, anyone have tried this with DSLR raw files? My first approach is to use the BatchFormatConversion script using "raw cfa" as input format hint and then using the output files with the SubframeSelector script. It have been useful to discard some subframes, but I don't know if it is correct to use the raw non-debayered data.
A more long workaround would be to debayer the images and then extracting the L component. One of the problems I have with this approach is the increase in file size from BatchDebayer because it saves the files in 32-bit format. I'm using a Canon 550D (14 bit depth) and the files grows to more than 200Mb (32 bit) from 35Mb (16 bit). I have an old computer and have several problems with memory. I can use again BatchFormatConversion script to feed the SubframeSelector, but...
In my head is to use the SubframeSelector script to help me with focus and need a fast method (my field computer is older :-))
Thanks in advance,
Sergio
-
well, if using an OSC you have to debayer the frames first...
for focusing, you might be better off using the various focus quality metrics built into BackyardEOS. if you have a bright enough star available you can actually use live view to achieve focus. in addition the latest version can also understand a bahtinov mask diffraction pattern and give you feedback about the focus.
also there's some freeware for windows called "bahtinov grabber" that can take a (user defined) screenshot, and given the focal length of your instrument can determine if you are in the critical focus zone.
if you want to try this script then maybe the best thing to do is just set the camera to take JPGs while trying to focus?
rob
-
Thank you Sergio,
I agree with Rob.
I didn't envision focusing with the scripts, I think they won't be efficient. Does FocusMax work with DLSR?
Mike
-
well, if using an OSC you have to debayer the frames first...
Yes, that's sound reasonable, the stars of a non-debayered image are not the best to perform this task. :)
for focusing, you might be better off using the various focus quality metrics built into BackyardEOS. if you have a bright enough star available you can actually use live view to achieve focus. in addition the latest version can also understand a bahtinov mask diffraction pattern and give you feedback about the focus.
also there's some freeware for windows called "bahtinov grabber" that can take a (user defined) screenshot, and given the focal length of your instrument can determine if you are in the critical focus zone.
I'm now using live view for focusing in APT, but I want more precision and be more seeing independent. APT implements bahtinov grabber, I will try to make a mask and use it.
if you want to try this script then maybe the best thing to do is just set the camera to take JPGs while trying to focus?
Yes, that's a nice idea. Thanks
Thank you Sergio,
I agree with Rob.
I didn't envision focusing with the scripts, I think they won't be efficient. Does FocusMax work with DLSR?
Mike
The problem with FocusMax is that you need Maxim to control the camera, and that's expensive. :-[
-
Sergio,
The B-mask is worth trying, but it might not be accurate on OSC, I don't know. The B-mask diffraction pattern may be less legible on a bayer array.
I printed the mask pattern on a sheet of mylar, that did not work, it was not accurate. I am not sure why, maybe its flexibility.
The metal mask from spike-a.com works wonderfully for focusing both wideband and narrowband. It is what I use for focusing my f/5 setup.
It is seeing insensitive and accurate. A 10 micron change in focus is visible with the mask (measured with my Atlas digital focuser).
Based on a phase-detect principle, both the direction of defocus and its magnitude are measurable from a single B-mask exposure, unlike contrast-detect used by FocusMax for example which requires multiple exposures.
Also the B-mask gives information across the entire frame. With a set of reasonably bright stars across the frame (e.g. an open cluster), focal surface curvature and tilt is also made visible.
Regards,
Mike
-
Hello Mike, thanks for the very useful script! It was not immediate to figure out what did you mean by "eccentricity", but I understand that the standard geometry definition for the ellipse applies. I also realized that, if one wants, your script can even provide the same information as CCD Inspector: one just needs to perform a best fit of the measured function FWHMm(i,j) with the mathematical expression of FWHM for a tilted plate and a miscollimated optics. The least-square fits would return the optical center on the plate (xc, yc), the curvature of the focal plane and the tilt angles in the x and y directions (or even a single tilt angle and its direction in the x-y frame).
Regards,
Antonio
-
Thank you, Antonio.
Yes, the standard geometry definition for the ellipse applies to eccentricity. I included formulas and a table in SubframeSelector documentation relating eccentricity, aspect ratio and a flatness measure. Stars with eccentricity values less than 0.42 to 0.44 look mostly round. Much larger than that and they start to look elongated.
Presenting focal surface parameters like CCD Inspector would be very helpful. Spatial FWHM variations may be due to various causes unrelated to focal surface curvature/tilt (e.g. astigmatism, coma, field rotation, guiding errors, flexure, etc). These other causes can be a source of confusion.
My setup suffers from tilt, verified by refocusing along the tilt axis. Here is a plot. The tilt runs diagonally, the upper left is inside of focus, the center is in focus, the lower right is outside of focus. I have replaced the entire imaging train but the tilt remains nearly the same. So I think it may be due to optical misalignment and/or sensor tilt.
(http://mschuster.zenfolio.com/img/s9/v87/p1713230052.png)
In hopes of solving this tilt problem, I bought one of these (http://www.gerdneumann.net/english/astrofotografie-parts-astrophotography/ctu-camera-tilting-unit.html) and am now awaiting delivery of adapters.
Regards,
Mike
-
Mike, I agree with you: the parameters summarizing "field correction" in CCD Inspector can be misleading sometime as in the cases you mentioned. For sure, the 2D maps of the FWHM spatial dependence are most truthful as "real data": from these maps one can optimize his scope setup without knowing the exact field curvature.
The map of your ccd FWHM does show clear indication of a tilt effect, but I wonder whether this is clearly discernible by visual inspection of the stars in the image: after all you go from 0.96 to 1.06 pixels which does not seem so much. Just curious!
Regards,
Antonio
-
Hi Antonio,
You are right. I have to admit that looking at the raw frames the tilt is not discernible, well maybe only very, very slightly.
But with a Bahtinov mask on a test exposure, the tilt is annoyingly obvious. The diffraction patterns show a rather large portion of the sensor outside the critical focus zone. It's something I want to fix if possible.
Regards,
Mike
-
With some work and PixInsight's help with measuring stars, I think I have finally eliminated my setup's sensor tilt and temperature dependent focus drift.
Here is an map of a 40 minute light without the tilt fix, the tilt runs from upper-left to lower-right:
(http://mschuster.zenfolio.com/img/s9/v87/p1713230052.png)
Here is a map of a 40 minute light with the tilt fix, nearly no tilt:
(http://mschuster.zenfolio.com/img/s10/v104/p389043874.png)
Here is a plot comparing FWHM's of short 20 second exposures with no guiding to the FWHM's of long 40 minute guided exposures. One short was captured immediate prior to starting each long exposure. FWHM's are nearly the same with nearly no focus drift:
(http://mschuster.zenfolio.com/img/s9/v87/p677342581.png)
Regards,
Mike
-
nice work - how did you eliminate the focus drift? i thought this was purely a function of the expansion of materials under a change in temperature.
i have a long way to go with my reflectors… currently far from properly collimated and/or tilt free.
rob
-
nice work - how did you eliminate the focus drift? i thought this was purely a function of the expansion of materials under a change in temperature.
Hi Rob,
Thank you. My setup doesn't truly eliminate focus drift, rather it compensates for the drift by using active focusing. It works just like an Optec TCF focuser, but uses a different focuser, thermistor, and my own drivers and modeling software. Challenges were good modeling for a mobile setup in varying weather conditions, and image shift when the focuser makes small mechanical movements.
Regards,
Mike
-
If PixInsight Magazine were still alive this would make for a very nice article. Really good stuff.
-
If PixInsight Magazine were still alive this would make for a very nice article. Really good stuff.
A stand-alone article perhaps?
-
The idea behind PixInsight Magazine - high quality well redacted articles - still has a lot of appeal for me. But ultimately, any magazine has to feed its editors (i.e. earn money). I wonder if it is possible to build a business case for such a PI centered product, maybe a PI magazine, a PI book, a PI (meta) blog, a PI newletter, ... . Any opinions?
Georg
-
Thank you. I have been playing more with the data:
Having used Pixinsight to measure stars and write the measurements to .csv files, I used Mathematica to plot smoothed histograms of star FWHMs. One histogram for each integration, with combined FWHM measurements from all of its subframes.
My setup at 4.2 arcsec per pixel is undersampled, so FWHMs can't be much smaller than about one pixel. But how close to one pixel can they be in good seeing?
Here is the progression over the past year and a half in three charts, each with just four histograms to make them more legible. Since this past August, the histograms have shifted leftward and become more peaky. This of course means that FWHMs are getting smaller and more equal across the frame. This improvement is due to better focus and better collimation.
The charts together have data for almost 300 forty minute subs, almost 200 hours of exposure.
Regards,
Mike
(http://mschuster.zenfolio.com/img/s9/v91/p474937325.png)
(http://mschuster.zenfolio.com/img/s5/v124/p484222313.png)
(http://mschuster.zenfolio.com/img/s5/v120/p133453392.png)