Author Topic: New automatic FITS header cleanup feature  (Read 7766 times)

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
New automatic FITS header cleanup feature
« on: 2013 April 30 07:43:54 »
Hi all,

I have just released an update with version 1.0.14 of the FITS format support module for PixInsight 1.8.0. This version comes with a new cleanup headers feature.

The FITS standard allows only printable ASCII characters in header keywords (ISO/IEC 8859-1 code points from 2016 to 7E16). Unfortunately, some applications ignore this rule and write illegal characters to FITS keyword comments and values. For example, the degree sign, whose code is B016 (decimal 176), is written by some applications to represent temperatures and angles in FITS keyword comments. Other common cases are copyright signs (A916 = 16910) and Latin characters with graves, acutes, dieresis, etc., in user names (SWOWNER keywords). For the sake of compatibility, PixInsight has always loaded these nonstandard FITS files without problems, but we'll never allow writing them unless the illegal characters are either removed or replaced with legal ASCII values. Our policy is to never contribute to propagate or distribute nonstandard files in any format.

Users of software applications that generate nonstandard FITS files for the reason described above have always had problems to save their images in standards-compliant applications such as PixInsight. The only solution to these problems is to edit or replace the wrong headers, and so far this had to be done either manually or using batch tasks implemented as scripts or with the ImageContainer and FITSHeader tools. These issues have been recurring, and it seems that the applications causing them have no intention to fix them, after many years. Hence, we have decided to implement a new feature to repair wrong headers automatically in PixInsight.

The new cleanup headers option is available from the FITS Format Preferences dialog:


If this option is enabled, any illegal characters in the header keywords of a FITS file will be replaced automatically by their best legal counterparts before writing the file to disk. For example, an illegal character C016 (Latin capital letter A with grave = À) will be replaced by 4116, or a Latin capital letter A. Similarly, an illegal copyright sign (A916 = ©) will be replaced by the standard three-character sequence "(C)". Note that the cleanup headers option is disabled by default. This has been done on purpose because we don't want nonstandard FITS files to pass inadverted.

The cleanup headers FITS feature can also be controlled using the "cleanup-headers" and "no-cleanup-headers" format hints to override current module settings by enabling and disabling, respectively, the cleanup headers option. For example, the following screenshot shows how to use the BatchFormatConversion script to fix a set of FITS files:


The same output hint can be used directly in tools such as ImageCalibration and StarAlignment.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: New automatic FITS header cleanup feature
« Reply #1 on: 2013 April 30 07:52:56 »
Cool.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Josh Lake

  • PixInsight Old Hand
  • ****
  • Posts: 424
Re: New automatic FITS header cleanup feature
« Reply #2 on: 2013 May 01 18:20:05 »
Nice work, Juan, great idea! Do you know some specific programs that this will help coming from? CCDStack?

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: New automatic FITS header cleanup feature
« Reply #3 on: 2013 May 03 00:07:42 »
Hi Josh,

Quote
CCDStack?

I don't think so. AFAIK this application generates standards-compliant FITS files.

With very special exceptions, I never comment on specific competing products.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/