In a previous post from July, I described the new features we had implemented in the StarAlignment tool. One of the most relevant of these features is StarAlignment's new local distortion correction capabilities. Now I'll describe an important improvement that is now available in the latest version of this tool, which makes distortion correction even more accurate and robust, especially in difficult cases.
As you know if you've read my previous post, StarAlignmen't distortion correction algorithm uses two-dimensional surface splines, also known as thin plates, as a non-rigid mathematical model to describe the geometrical transformation necessary to register two images. Surface splines are extremely adaptable to local function variations, which makes them particularly appealing as flexible interpolating devices in many image analysis and processing tasks. We already have been using them for a long time in tools such as DynamicBackgroundExtraction and DynamicAlignment.
So far, StarAlignment's distortion correction has been using interpolating surface splines. In an interpolating spline, each data point is reproduced exactly by evaluation of the spline functions. In the case of image registration, this means that for each matched star pair, the target image star is registered exactly on its reference image counterpart. This sounds good at first sight, since after all, "perfect" star matches guarantee a "perfect" image registration. But as you surely know if you've been involved in this image processing business for some time, the words "perfect" and "exactly" almost invariably denote a questionable procedure or implementation.
As implemented in our StarAlignment tool, distortion correction by interpolating surface splines works very well most times. However, it can work not so well sometimes, and I'm going to put a "nice" example of this problem here. Before you continue reading and getting scared by some image comparisons, the good news is that (1) these problems are now fixed with the new implementation, and (2) these problems only happen in particularly unfavorable cases.
The following screenshot shows PixInsight 1.8.0 Ripley while working on a two-frame mosaic by Vicent Peris, who has released the final image on the Gallery forum board. The mosaic covers a nice region of the Large Magellanic Cloud, close to the Tarantula nebula.
Although not evident at first sight, this data set has two properties that expose the weakness of interpolating splines used as image registration models:
1 - It is a wide-field image crowded of stars.
2 - Seeing was not particularly good, so the stars have quite flat profiles.
The first property increases the probability of overlapped, colliding, or very close star images. Although StarAlignment's star detection routine does a good job trying to avoid such problematic star images as registration references, decreasing the average distance between nearby star images doesn't help to compute accurate star positions. The second property makes things even worse for star detection, since a flat star profile increases the uncertainty in the computed star position. As a result, the registration model is less accurate, more uncertain, and contains a non-negligible amount of mistakes. If we use interpolating surface splines to reproduce all star pair matches exactly, the result is a very good example of the difference between numerical accuracy and model adequacy:
Image animation: Registration with distortion correction, interpolating surface splines.
To generate the above animation, we have registered both mosaic frames with StarAlignment working in Register/Union - Separate mode and the new Spline smoothness parameter set to zero. This effectively forces StarAlignment to use pure interpolating splines, which has been the only option available until now. To combine both mosaic frames in a way that maximizes registration differences visually, we have applied the maximum operator with the following PixelMath expression:
on a full preview on one of the mosaic frames. What you are looking at on the above animation is the result of excessive local adaptation, or excessive flexibility in the registration model.
Enter approximating surface splines. In contrast to interpolating splines, approximating splines don't reproduce all data points exactly. Instead, approximating splines introduce some degree of smoothness in the computed registration model. This is a more realistic approach because it assumes that there are errors in the computed star positions, or in other words, that the star positions are really not known exactly, but are just approximations.
To understand how approximation works as opposed to interpolation, consider the following 256x256 pixels grayscale image.
For a moment, forget that this is an image of stars; just take it as a set of scattered measurements of some process represented as varying intensities on a square matrix.
With a simple script in PixInsight, I have generated an interpolating 2-D surface spline from 1024 pixels of the above image, distributed on a regular lattice at intervals of 8 pixels (or a grid of 32x32 interpolation points). When we use this spline to interpolate the original image, we get the following result:
Since the interpolating spline reproduces all data points exactly, we get a wildly varying representation because the original data also vary strongly at small scales. This is an accurate representation of the original data set, but it is not necessarily an accurate representation of the underlying true function.
With approximating surface splines we can get a much better view of the data from a less local, more global perspective. A nice feature of approximating surface splines is that we can control the degree of regularization, or smoothness, with a single numerical parameter:
Now that you know how it works, let's repeat the same mosaic with the new Spline smoothness parameter of StarAlignment set to its default 0.25 value:
Image animation: Registration with distortion correction, approximating surface splines.
Here are animations of the same area zoomed 8:1:
Image animation: Registration with distortion correction, interpolating surface splines (zoom 8:1).
Image animation: Registration with distortion correction, approximating surface splines (zoom 8:1).
For a better comparison, this is the same region of the mosaic generated using a projective registration model without distortion correction:
Image animation: Registration without distortion correction, projective model.
The best part of this new feature is that it works very well with its default parameter value in virtually all cases, so you won't have to worry about a new parameter: StarAlignment remains a tool that "just works", and that's even more true with approximating surface splines.
As you know if you've read my previous post, StarAlignmen't distortion correction algorithm uses two-dimensional surface splines, also known as thin plates, as a non-rigid mathematical model to describe the geometrical transformation necessary to register two images. Surface splines are extremely adaptable to local function variations, which makes them particularly appealing as flexible interpolating devices in many image analysis and processing tasks. We already have been using them for a long time in tools such as DynamicBackgroundExtraction and DynamicAlignment.
So far, StarAlignment's distortion correction has been using interpolating surface splines. In an interpolating spline, each data point is reproduced exactly by evaluation of the spline functions. In the case of image registration, this means that for each matched star pair, the target image star is registered exactly on its reference image counterpart. This sounds good at first sight, since after all, "perfect" star matches guarantee a "perfect" image registration. But as you surely know if you've been involved in this image processing business for some time, the words "perfect" and "exactly" almost invariably denote a questionable procedure or implementation.
As implemented in our StarAlignment tool, distortion correction by interpolating surface splines works very well most times. However, it can work not so well sometimes, and I'm going to put a "nice" example of this problem here. Before you continue reading and getting scared by some image comparisons, the good news is that (1) these problems are now fixed with the new implementation, and (2) these problems only happen in particularly unfavorable cases.
The following screenshot shows PixInsight 1.8.0 Ripley while working on a two-frame mosaic by Vicent Peris, who has released the final image on the Gallery forum board. The mosaic covers a nice region of the Large Magellanic Cloud, close to the Tarantula nebula.
Although not evident at first sight, this data set has two properties that expose the weakness of interpolating splines used as image registration models:
1 - It is a wide-field image crowded of stars.
2 - Seeing was not particularly good, so the stars have quite flat profiles.
The first property increases the probability of overlapped, colliding, or very close star images. Although StarAlignment's star detection routine does a good job trying to avoid such problematic star images as registration references, decreasing the average distance between nearby star images doesn't help to compute accurate star positions. The second property makes things even worse for star detection, since a flat star profile increases the uncertainty in the computed star position. As a result, the registration model is less accurate, more uncertain, and contains a non-negligible amount of mistakes. If we use interpolating surface splines to reproduce all star pair matches exactly, the result is a very good example of the difference between numerical accuracy and model adequacy:
Image animation: Registration with distortion correction, interpolating surface splines.
To generate the above animation, we have registered both mosaic frames with StarAlignment working in Register/Union - Separate mode and the new Spline smoothness parameter set to zero. This effectively forces StarAlignment to use pure interpolating splines, which has been the only option available until now. To combine both mosaic frames in a way that maximizes registration differences visually, we have applied the maximum operator with the following PixelMath expression:
Code:
max( $T, the_other_mosaic_frame )on a full preview on one of the mosaic frames. What you are looking at on the above animation is the result of excessive local adaptation, or excessive flexibility in the registration model.
Enter approximating surface splines. In contrast to interpolating splines, approximating splines don't reproduce all data points exactly. Instead, approximating splines introduce some degree of smoothness in the computed registration model. This is a more realistic approach because it assumes that there are errors in the computed star positions, or in other words, that the star positions are really not known exactly, but are just approximations.
To understand how approximation works as opposed to interpolation, consider the following 256x256 pixels grayscale image.
For a moment, forget that this is an image of stars; just take it as a set of scattered measurements of some process represented as varying intensities on a square matrix.
With a simple script in PixInsight, I have generated an interpolating 2-D surface spline from 1024 pixels of the above image, distributed on a regular lattice at intervals of 8 pixels (or a grid of 32x32 interpolation points). When we use this spline to interpolate the original image, we get the following result:
Since the interpolating spline reproduces all data points exactly, we get a wildly varying representation because the original data also vary strongly at small scales. This is an accurate representation of the original data set, but it is not necessarily an accurate representation of the underlying true function.
With approximating surface splines we can get a much better view of the data from a less local, more global perspective. A nice feature of approximating surface splines is that we can control the degree of regularization, or smoothness, with a single numerical parameter:
Now that you know how it works, let's repeat the same mosaic with the new Spline smoothness parameter of StarAlignment set to its default 0.25 value:
Image animation: Registration with distortion correction, approximating surface splines.
Here are animations of the same area zoomed 8:1:
Image animation: Registration with distortion correction, interpolating surface splines (zoom 8:1).
Image animation: Registration with distortion correction, approximating surface splines (zoom 8:1).
For a better comparison, this is the same region of the mosaic generated using a projective registration model without distortion correction:
Image animation: Registration without distortion correction, projective model.
The best part of this new feature is that it works very well with its default parameter value in virtually all cases, so you won't have to worry about a new parameter: StarAlignment remains a tool that "just works", and that's even more true with approximating surface splines.
