Updated 2020 January 16, core version 1.8.8-4Hi all,
This announcement is relevant for PixInsight users working with raw data acquired by some DSLR cameras. It describes a serious incompatibility introduced by recent versions of LibRaw, an open-source software library which our RAW module uses as a back-end to support raw formats of digital cameras.
Description of the ProblemThe RAW module included in the standard PixInsight distribution since core version 1.8.8 uses
LibRaw 201910 snapshot. Since this version, LibRaw has changed the way some raw formats are decoded. The changes we are interested in here refer to the coordinates of the visible region of the raw image with respect to the entire (uncropped) raw frame. As you probably know, many (most?) DSLR raw formats define unused areas of the raw frame, which correspond to rows and columns of pixels that are never exposed to light. Obviously, these unused regions are located at the edges of the raw frame, and their dimensions and location depend on the particular formats used by each camera model. Our RAW module provides a preferences option to enable/disable automatic cropping of unused areas, which is enabled by default.
Unfortunately, the changes introduced by LibRaw since the 201910 snapshot version for some cameras create an incompatibility with master calibration frames generated from raw frames of the affected formats with PixInsight 1.8.7 and earlier versions. This problem affects mainly to some Canon cameras, where the visible region of the image begins at odd vertical pixel coordinates from the upper left corner of the entire raw frame. This includes, probably among others, the following cameras:
EOS 1D Mark IV
EOS 5D Mark II
EOS 7D
EOS 60D
EOS 550D
EOS 600D
EOS 1200D
See the
original bug report where this problem has been first described (to my knowledge) on this forum. The report refers to the Canon EOS 600D, which is quite popular among PixInsight users working with DSLR cameras. As you can see, in this camera the visible region of the frame starts at vertical pixel coordinate y=51, where x=y=0 corresponds to the pixel at the upper left corner of the uncropped raw frame.
LibRaw versions before the 201910 snapshot version reported frame coordinates correctly for all Canon cameras, that is, the
actual coordinates of the visible image within the raw frame, as well as the actual Bayer pattern, which is always RGGB for these cameras.
Since the 201910 snapshot version, LibRaw intentionally removes the first row of pixels from the visible image for these raw formats. In the case of EOS 600D for example, LibRaw reports now a visible region starting at y=52 with a height of 3464 pixels, instead of the actual 3465 pixels. Since the cropped frame starts at y=52 instead of y=51, the reported Bayer pattern is now GBRG instead of RGGB.
The authors of LibRaw have good reasons to implement this change. As a software developer, I can understand their reasons from a technical point of view. The repercussions of this change for daylight or 'normal' photography applications are as minimal as a single row of pixels cropped at the top of the image. However, the problems generated are severe for applications requiring calibration of raw data.
LibRaw is not going to revert these changes. Since there is no alternative to LibRaw for a multiplatform application, we have to find a suitable solution.
Our SolutionThe solution we have implemented is quite simple: a special
update-compat repository, where you can opt-out of the new RAW module in PixInsight 1.8.8-1 and newer versions, replacing it with a module that is compatible with existing master calibration frames generated with PixInsight 1.8.7 and older versions for the affected cameras (see a partial list above).
If you
really need to use master calibration frames incompatible with the latest version of LibRaw, here is the procedure to opt-out of the current RAW module:
1. Make sure that you are using PixInsight core version 1.8.8-4 or later.
2. Make sure that you have applied all available updates for your current PixInsight version. Note that this
might include an update to the standard RAW module at the time you are reading this. In such case you must apply that update before continuing.
3. From the main menu, select
Resources > Updates > Manage Repositories.
4. On the Manage Update Repositories dialog, click the Add button.
5. Enter the following repository URL:
6. Click OK, Then OK again on the Manage Update Repositories dialog.
7. From the main menu, select
Resources > Updates > Check for Updates.
8. The
Select and Download Updates dialog should now be open. Make sure that the RAW_compat update is available, then click Apply.
9. After downloading and extracting the update, as usual, exit the PixInsight core application and apply the update.
10. Check that you have the RAW_compat module in Format Explorer.
Here is a small video showing the above procedure:
After the update, the standard RAW module will be replaced by a special RAW_compat module that uses an improved version of
LibRaw 0.19.5-Release. The RAW_compat module is basically the same RAW module included in PixInsight 1.8.7. You can use it while you have to use old master calibration frames for Canon cameras suffering from this incompatibility.
Our recommendation, however, is to return to the standard RAW module once you are ready to work with newly acquired data.To return to the standard RAW module, remove the update-compat repository enabled with the above procedure. Then add the following repository URL:
and repeat the check for updates process, etc. This will reinstall the standard RAW module using the latest LibRaw version available.
I hope this solution will be useful for all users that have to suffer from this serious compatibility issue. Please realize that we are not causing this problem. For obvious reasons, we cannot stick to an old version of one of our fundamental third-party support libraries. Thank you for your patience and understanding.
