XISF Version 1.0 Specification (DRAFT 3)

By Juan Conejero (PixInsight Development Team)


Formal definition of the Extensible Image Serialization Format (XISF) version 1.0. [more]

Keywords: XISF, image serialization, image file format, file format, serialization, formal specification

Contents

[hide]

1 Introduction

[hide]

Extensible Image Serialization Format (XISF) is a free file format for storage, management and interchange of digital images and associated data.

XISF has been originally conceived and implemented as the native file format of PixInsight, an image processing software platform designed specifically for astronomical imaging, and developed by the company Pleiades Astrophoto S.L. However, our hope is that XISF serves as an efficient tool for the development of imaging software, including not only software specialized in astronomy, but image processing software in a wide range of technical and general fields.

Two key elements in the design of XISF can be found in its title: extensible and serialization. Extensibility is crucial to adapt the format easily and efficiently to the requirements of present and future software applications. The architecture of XISF has to facilitate the development of extensions to the core format specification, and for this purpose XISF headers are standard XML documents. Serialization denotes the ability of XISF to store not just image data, but also data structures associated with the environments where the images evolve as living objects. These data structures can be deserialized to recreate the images along with their working contexts. We formalize the resources to store data structures and objects as properties of a variety of predefined data types. XISF properties can be directly associated with images, with entire XISF units, or be defined as standalone components.

Finally, XISF is a free format open to the contributions of anyone interested, including users of PixInsight and other applications, as well as individuals and groups from other development teams, institutions and companies involved or interested in image processing software.

1.1 Contributors

  • Jean-Marc Lugrin (Switzerland)
  • Mike Schuster (United States)
  • Georg Viehoever (Germany)

1.3 Disclaimer

This document and the information contained herein is provided on an "AS IS" basis and PLEIADES ASTROPHOTO S.L. DISCLAIMS ALL WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, ANY ACTUAL OR ASSERTED WARRANTY OF NON-INFRINGEMENT OF PROPRIETARY RIGHTS, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. NEITHER PLEIADES ASTROPHOTO S.L. NOR ITS CONTRIBUTORS SHALL BE HELD LIABLE FOR ANY IMPROPER OR INCORRECT USE OF INFORMATION. NEITHER PLEIADES ASTROPHOTO S.L. NOR ITS CONTRIBUTORS ASSUME ANY RESPONSIBILITY FOR ANYONE'S USE OF INFORMATION PROVIDED BY PLEIADES ASTROPHOTO S.L. IN NO EVENT SHALL PLEIADES ASTROPHOTO S.L. OR ITS CONTRIBUTORS BE LIABLE TO ANYONE FOR DAMAGES OF ANY KIND, INCLUDING BUT NOT LIMITED TO, COMPENSATORY DAMAGES, LOST PROFITS, LOST DATA OR ANY FORM OF SPECIAL, INCIDENTAL, INDIRECT, CONSEQUENTIAL OR PUNITIVE DAMAGES OF ANY KIND WHETHER BASED ON BREACH OF CONTRACT OR WARRANTY, TORT, PRODUCT LIABILITY OR OTHERWISE.

2 Version History

[hide]

Table 1
Version History of the XISF Specification Document

Version

Date

Description

July-October 2014

First internal working documents about XISF.

0.1

December 2014

First public draft of the XISF specification.

0.2

January 2015

Second public draft of the XISF specification.

0.3

January 2015

Third public draft of the XISF specification.

3 Terminology and Conventions

[hide]

3.1 Requirement Levels

The key words must, must not, required, shall, shall not, should, should not, recommended, may, and optional in this document are to be interpreted as described in RFC 2119.[3]

3.2 Typographical Conventions

Monospaced text represents literal source code fragments.

Italicized monospace text represents metalanguage fragments describing source code elements:

name="value"

In this example, the word value is a metasymbol used to describe the value of a name XML attribute.

In code fragments, an ellipsis represents a variable-length list of items:

item1:...:itemN

This example shows a list of N items separated by literal colon ':' characters. Note that the itemi elements are metasymbols in this case, not literal fragments.

Portions of this document enclosed by gray dashed lines represent literal source code examples:

   This is a source code example.

Literal source code fragments not necessarily correspond to working code examples. When necessary or not obvious, this is indicated in the text describing the examples.

The subindexes 2 8 10 and 16 indicate base 2 (or binary), base 8 (octal), base 10 (decimal), and base 16 (hexadecimal) integers, respectively. For example:

7610 = 4C16

Figures are for illustration only; unless explicitly stated, they are not reference renderings.

4 Availability and Use of XISF

[hide]

XISF has been conceived as a free, open format since its origin. The words free and open formally mean the following:

  1. The entire definition of XISF shall be publicly available without any restrictions or monetary cost to anyone.
  2. Anyone shall be able to use or implement XISF freely without any monetary cost for any purpose.
  3. XISF shall not be subject to patents or royalties of any kind that could limit its availability or impose a monetary cost for its availability or its use.
  4. These conditions shall constitute a legally binding assignment, and no individual or organization that implements, extends or redefines XISF in the future will be able to revoke or limit them under any circumstance.

A copy of this specification document is available online at the following URL:

http://pixinsight.com/doc/docs/XISF-1.0-spec/XISF-1.0-spec.html

This document has been authored in the PIDoc markup language, and has been generated by the PixInsight Documentation Compiler script. The entire PIDoc source code of the latest version of this specification is available at the following GitHub repository:

https://github.com/PixInsight/XISF-specification

For discussion on the XISF specification and implementations, please join us at PixInsight Forum:

http://pixinsight.com/forum/index.php?board=20.0

5 Overview of XISF 1.0 Features

[hide]

The following is an informal overview of features in the initial XISF version 1.0 specification—not all parts of the specification are described, and this overview is not part of the specification proper.

Two storage models

  • Monolithic model, where all image metadata, pixel sample data and image properties are stored in a single file.
  • Distributed model, where image definitions and metadata are stored in a single header file, while pixel sample data and image properties are stored as separate units, including local and remote files.

Relocatable distributed units

A distributed XISF unit can be built so that it can be moved or copied to any location in a local file system and across different machines.

Multiple images, metadata and image properties

An XISF unit can store an unlimited number of images with different geometries, color spaces, and associated metadata and properties.

Multidimensional images

XISF supports arbitrary image dimensions, including one-dimensional, two-dimensional and three-dimensional images, as well as images of higher dimensions such as tesseracts and other hypercubes.

Multichannel images

XISF supports an unlimited number of image hyperplanes (not to be confused with the dimensionality of the image), such as nominal color channels and alpha channels, for each stored image.

Sequential and random access

Uncompressed XISF data blocks support sequential and random access to read or write individual pixel sample rows or sets of contiguous rows.

Eight pixel sample data types

XISF supports integer pixel samples stored as unsigned 8-bit, 16-bit, 32-bit and 64-bit integers, plus real and complex samples in 32-bit and 64-bit IEEE 754 floating point format.

Colorimetrically defined color spaces

XISF supports images represented in colorimetrically defined grayscale, RGB and CIE L*a*b* color spaces.

RGB working spaces

XISF can serialize the parameters of colorimetrically defined RGB working spaces associated with stored images.

ICC color profiles

XISF can store standard ICC color profiles associated with stored images.

Unambiguous floating point data ranges

Floating point real and complex images can be stored using arbitrary values and ranges. However, an unambiguous specification of the numeric range to which floating point real image data is referred—that is, the black and white points of the image—is mandatory. This prevents any ambiguity in the interpretation of floating point image data and guarantees interoperability between different implementations and applications.

Data compression

The XISF specification supports data compression codecs, including lossless and lossy compression schemes. The deflate/zlib algorithm is the standard compression codec of XISF.

Digital signatures

The XISF specification supports signed files with standard X.509 certificates and XML signatures. Digital signatures protect both data integrity and authenticity.

XML file headers

XISF headers (in both the monolithic and distributed models) are plain text in standard XML 1.0 format encoded as UTF-8.

Full Unicode support

Image metadata and textual properties fully support Unicode.

Properties

An XISF property is a data block accessible through a unique identifier (name-value pairs). Properties can be associated with images, with the whole XISF unit, or be serialized as standalone elements. XISF properties are typed objects of scalar, complex, string, time point, vector and matrix types.

FITS format compatibility

FITS header keywords can be stored in XISF files and retrieved from XISF files as if the reader were working with actual FITS files.

6 Definitions

[hide]

The following subsections form a collection of important terms and concepts identified during the production of this document. They are included here for informational/reference purposes only.

Byte

An 8-bit unsigned binary integer.

File

A file is a sequence of N 8-bit contiguous bytes, where N is the file length and bytes are numbered from 0 to N–1. The theoretical maximum file length is 264–1 = 18,446,744,073,709,551,615 bytes.

Byte offset / byte position

Number of bytes from the beginning of a file, or from the beginning of a given section of a file.

File system

A file system allows software applications to store and retrieve files on storage devices. Files are usually organized in a hierarchical structure of directories (also known as folders). Files and directories can be accessed through identifying names.

Storage unit

A file or a set of files storing an XISF unit or part of an XISF unit.

XISF unit

A set of one or more files defining contents in the XISF format.

Monolithic XISF unit

An XISF unit structured following the monolithic storage model.

Distributed XISF unit

An XISF unit structured following the distributed storage model.

XISF header

A valid XML 1.0[11] document encoded in UTF-8[16] [17] that describes and locates the entire contents of an XISF unit.

XISF header file

An XISF header stored as a single file.

XISF data block

A file or a subset of a file storing data that can be accessed through an XISF unit.

XISF data blocks file

A file that allows indexed access to XISF data blocks, available as a local or a remote resource.

Encoder

Any system, application, utility or device able to generate XISF units according to this specification.

Decoder

Any system, application, utility or device able to read and interpret the contents of an XISF unit according to this specification.

Scalar

A single value without internal structure, such as an integer or a floating point number.

Complex number

A number that can be expressed in the form , where and are real numbers and is the imaginary unit, which is defined by the equation . The components and of a complex number are called its real and imaginary parts or components.

Vector

A homogeneous sequence of zero or more contiguous scalars or complex numbers.

Matrix

A homogeneous set of contiguous scalars or complex numbers, structured as a two-dimensional array with a fixed number of rows and columns.

String

A homogeneous sequence of zero or more contiguous Unicode code points represented as integer scalars.

Array

A contiguous sequence of zero or more objects, where any object can be accessed through an array index.

Homogeneous structure

A structure that can only store objects of the same class or data type.

Heterogeneous structure

A structure that can store objects of different classes or data types.

Contiguous sequence / contiguous objects

Zero or more objects or structures stored consecutively without any gaps.

Case-sensitive

A text property, such as the name or identifier of an object or data item, which distinguishes between uppercase and lowercase letters. For example, the words "thisIsFoo" and "thisisfoo" are different if considered as case-sensitive.

Case-insensitive

A text property, such as the name or identifier of an object or data item, which does not distinguish between uppercase and lowercase letters. For example, the words "FooBar" and "foobar" are equivalent if considered as case-insensitive.

Little-endian

A data encoding where the bytes within a 16-bit, 32-bit or 64-bit value are addressed from the least significant to the most significant as the byte address increases.

Big-endian

A data encoding where the bytes within a 16-bit, 32-bit or 64-bit value are addressed from the most significant to the least significant as the byte address increases.

7 Conformance

[hide]

We define several conformance levels for any encoder or decoder that claims conformance with this XISF specification:

7.1 Level 1 XISF Conformance

A level 1 XISF decoder shall implement this specification to read XISF units, including all optional components and features, with the following allowed exception:

A level 1 XISF encoder shall implement this specification to generate XISF units, including all optional components and features, with the following allowed exception:

7.2 Level 2 XISF Conformance

A level 2 XISF decoder shall implement this specification to read XISF units, including all optional components and features, with the following allowed exceptions:

  • Read pixel data in the UInt64, Complex32 and Complex64 pixel sample formats.
  • Read and process XML signatures to verify the integrity and authenticity of the data in an XISF unit.

A level 2 XISF encoder shall implement this specification to generate XISF units, including all optional components and features, with the following allowed exceptions:

  • Write pixel data in the UInt64, Complex32 and Complex64 pixel sample formats.
  • Write XML signatures to guarantee the integrity and authenticity of the data in an XISF unit.

7.3 Level 3 XISF Conformance

A level 3 XISF decoder shall implement this specification to read XISF units with the following allowed exceptions:

  • Read files and blocks not stored in the local file system.
  • Read pixel data in the UInt64, Complex32 and Complex64 pixel sample formats.
  • Read and process XML signatures to verify the integrity and authenticity of the data in an XISF unit.

A level 3 XISF encoder shall implement this specification to generate XISF units with the following allowed exceptions:

In addition, other optional features and components of the specification may be excluded in a level 3 XISF encoder or decoder.

7.4 Baseline XISF Conformance

A baseline XISF decoder shall have at least the following abilities to read XISF units:

A baseline XISF encoder shall have at least the following abilities to generate XISF units:

8 XISF Data Objects

[hide]

Two high-level data objects have been defined in this XISF specification: Property and Image. In this section we define the structural and functional properties of these objects, independently on the structure of an XISF unit. In successive sections we'll describe how these objects are represented and how they play their roles for data serialization. As a result of the extensibility of XISF, you can expect new high-level objects to be defined in future versions of this specification.

8.1 Fundamental Scalar Types

Table 2 lists the fundamental data types used to define all objects serializable by XISF. All of these types are fixed-length scalar data types.

Table 2
XISF Fundamental Scalar Types

Type name

Size (bytes)

Lower bound

Upper bound

Decimal precision

Unsigned 8-bit integer

1

0

255

Unsigned 16-bit integer

2

0

65,535

Unsigned 32-bit integer

4

0

4,294,967,295

Unsigned 64-bit integer

8

0

18,446,744,073,709,551,615

Signed 8-bit integer

1

–128

+127

Signed 16-bit integer

2

–32,768

+32,767

Signed 32-bit integer

4

–2,147,483,648

+2,147,483,647

Signed 64-bit integer

8

–9,223,372,036,854,775,808

+9,223,372,036,854,775,807

IEEE 754[26] binary32 floating point

4

± 1.18 × 10–38

± 3.40 × 1038

1.19 × 10–7

IEEE 754 binary64 floating point

8

± 2.23 × 10–308

± 1.80 × 10308

2.22 × 10–16

Fundamental scalar types can be unsigned integer numbers, two's complement signed integer numbers, and floating point numbers. Floating point types correspond to the IEEE 754[26] binary32 and binary64 basic formats, also known as single precision and double precision, respectively. For floating point types, the lower and upper bound columns in Table [2] refer to the approximate smallest and largest representable normal values, and the decimal precision column gives the relative rounding error for each type, also known as machine epsilon.

IEEE 754 non-numeric entities, including infinities and NaN (Not a Number), are supported by this specification and must be correctly handled (in an implementation-specific manner) by XISF encoders and decoders.

8.2 Endianness

All the data stored in an XISF unit, including XISF headers, XISF data blocks files and all XISF data blocks, shall be encoded as little-endian.

The only exception to this rule occurs when an object in other format requiring big endian encoding has to be embedded in an XISF unit. ICC color profiles[28] are the most frequent instances of this exception.

The contents of external files not encoded as XISF data blocks, which can be referenced from XISF headers, are out of the scope of this specification. Therefore, the endianness of these files and the way they are used are the responsibility of the decoders and applications accessing them.

8.3 Plain Text Serialization of Scalars

Serializations of scalar values as plain text—for example, in XML attribute values—shall comply with the following rules:

8.3.1 Plain Text Serialization of Decimal Integers

A serialization of a decimal (base 10) integer value as plain text shall satisfy the following regular expression:[19]

\s*[+-]?[1-9][0-9]*\s*

Examples:

987
-123
+45678
8.3.2 Plain Text Serialization of Binary, Octal and Hexadecimal Integers

Serializations of binary (base 2), octal (base 8) and hexadecimal (base 16) integer values as plain text shall satisfy the following regular expressions:[19]

Binary:

\s*0[bB][0-1]+\s*

Octal:

\s*0[oO][0-7]+\s*

Hexadecimal:

\s*0[xX][0-9a-fA-F]+\s*

Examples:

0b10100111100101
0o570261
0x80E950AB

In the first example, 101001111001012 = 1072510.

In the second example, 5702618 = 19268910.

In the third example, if the represented value is a two's complement signed 32-bit integer, the value is 80E950AB16 = –213219310910. However, if the represented value is an unsigned integer, the value is 216277418710.

8.3.3 Plain Text Serialization of Floating Point Values

A serialization of a numeric floating point scalar as plain text, including both IEEE 754[26] binary32 and binary64 formats, shall satisfy the following regular expression:[19]

\s*([-+]?(([0-9]+)?\.)?[0-9]+([eE][-+]?[0-9]+)?)|(NaN)|([-+]Inf)\s*

where the non-numeric IEEE 754 entities NaN (Not a Number), +∞ and –∞ are serialized as NaN, +Inf and -Inf, respectively.

Examples:

123
123.456
-123.456
.123
+.123
NaN
1e0
-0.123e+02
8.3.4 White Space

For textual serializations of all scalar types, white space (represented as the \s character class in regular expressions) is irrelevant, should not be generated by encoders, and must be ignored by decoders.

8.4 XISF Property

An XISF property is an association of two data structures, identifier and value. A property may also have an optional associated format specifier to control its representation as readable text. An XISF property may be associated, either with an object stored in an XISF unit, or with an XISF unit as a whole. An XISF property may also be stored in an XISF unit as a standalone object. Valid associations between properties and objects will be defined later in this document.

8.4.1 Property Identifier

A property identifier must be a sequence of one or more 8-bit ASCII characters[18] satisfying the following regular expression:[19]

[_a-zA-Z][_a-zA-Z0-9]*(:([_a-zA-Z][_a-zA-Z0-9])+)*

Examples:

MyFirstProperty
mySecondOne234
_this_1_is_a_test
Namespace:Property
foo:bar:Foo2_Bar3

The colon separator (':', ASCII code point 5810 = 3A16) may be used to organize a set of properties into customary groups known as namespaces. Namespaces can be useful to improve structuration of properties by defining logical and semantic categories.

A property identifier must be unique for the object with which the property is associated. For example, all the properties of a given image must have unique identifiers, but two properties associated with different images may have the same identifier, or a property associated with the whole XISF unit may have the same identifier as another property associated with an image.

There is no specific limit for the length of a property identifier.

8.4.2 Property Value

A property value is a data object associated with a property identifier. A property value has a predefined XISF property type, which determines its structure and behavior.

8.4.3 Property Format Specifier

A property format specifier may be associated with a property to control generation of readable plain text representations of the property value in a standardized way.

A property format specifier is a list of one or more format tokens separated by semicolon ASCII characters (';', ASCII code point 5910 = 3B16):

token1;token2;...;tokenn

White space is irrelevant in a property format specifier and must be ignored. The following format tokens are supported by this specification:

width:count

count is a plain text representation of an unsigned integer. If the length n in characers of the represented value is less than the value of count, the represented value shall be padded with (ncount) padding characters—see the fill token below. The location of padding characters depends on the current alignment—see the align token below.

If no width token is specified, the value shall be represented using the minimum number of characters necessary, that is, no padding will be used by default.

fill:char

char is a single ASCII character that shall be used as a filling character for padded representations—see the width token above.

When no fill token is specified, the default padding character shall be the ASCII space character (ASCII code point 3210 = 2016).

align:mode

mode defines the alignment used for padded representations of all scalar types (see the width token above). mode must be one of:

left

Padding characters shall be appended at the end of the represented value.

right

Padding characters shall be prepended at the beginning of the represented value.

center

If the length n of the padding space is even, the same number n/2 of padding characters shall be prepended and appended at the beginning and the end of the represented value, respectively. If the padding length n is odd, n/2+1 padding characters shall be prepended at the beginning and n/2 padding characters shall be appended at the end of the represented value.

When not specified, the default alignment shall be right, as described above.

sign:mode

mode defines how a sign character is used for representations of numeric values. mode must be one of:

auto

A minus sign ASCII character ('-', ASCII code point 4510 = 2D16) shall be prepended to a representation of a scalar if the represented value is less than zero. If the represented value is zero or greater than zero, a sign character shall not be prepended.

force

A minus sign ASCII character ('-', ASCII code point 4510 = 2D16) shall be prepended to a representation of a scalar if the represented value is less than zero, and a plus sign ASCII character ('+', ASCII code point 4310 = 2B16) shall be prepended if the value is greater than zero.

In all cases, if the numeric value is being represented as zero (after a possible truncation or rounding operation), a sign character must not be prepended to the represented value.

When not specified, the default sign mode shall be auto, as described above.

A decoder must ignore the sign format token for representations of non-numeric values.

precision:count

count is a plain text representation of an unsigned integer, whose value is the precision of the representation of a floating point value. The precision shall be interpreted according to the current floating point mode (see the float token below):

  • For a representation of a floating point scalar when the floating point mode is scientific or fixed, the precision is the minimum number of digits to appear after the decimal separator.
  • For a representation of a floating point scalar when the floating point mode is auto, the precision is the maximum number of significant digits.

The represented floating point scalar shall be rounded to the nearest value with the required number of decimal digits.

If no precision token is specified, the default precision shall be 6.

A decoder must ignore the precision format token for representations of non-numeric or integer numeric values.

float:mode

mode defines the floating point mode used for representations of floating point numeric scalars. mode must be one of:

auto

The floating point value shall be represented using printf's 'g' mode.[20] The floating-point value shall be represented in scientific or fixed floating point mode, depending on the value being converted and the current precision, to generate the shortest possible text representation.

scientific

The floating point value shall be represented in exponential notation. Ignoring possible padding characters, the representation shall satisfy the following regular expression:[19]

[-+]?[0-9]+\.[0-9]+[eE][-+][0-9]+
fixed

The floating point value shall be represented in fixed-point decimal notation. Ignoring possible padding characters, the representation shall satisfy the following regular expression:

[-+]?[0-9]+(\.[0-9]+)?

When not specified, the default floating point mode shall be auto, as described above.

A decoder must ignore the float format token for representations of non-numeric or integer numeric values.

bool:mode

mode defines how Boolean values are represented. mode must be one of:

alpha

Boolean values shall be represented as the plain text words false and true.

numeric

The integers 0 and 1 shall be used to represent the Boolean values false and true, respectively.

When not specified, the default mode shall be alpha.

A decoder must ignore the bool format token for representations of non-Boolean values.

base:radix

radix defines the numeric base for the representation of an integer value. radix must be one of:

bin

The integer value shall be represented as a binary number using exclusively the digits '0' and '1'.

oct

The integer value shall be represented as an octal (base 8) number using the digits from '0' to '7'.

dec

The integer value shall be represented as a decimal number using the digits from '0' to '9'.

hex

The integer value shall be represented as an hexadecimal (base 16) number using the digits from '0' to '9' and from 'a' to 'f' to represent the numbers from 0 to 9 and from 10 to 15, respectively.

When not specified, the default value of radix shall be dec, that is, decimal representations shall be generated for integer numbers by default.

A decoder must ignore the base format token for representations of non-integer values.

unit:text

text is a Unicode text fragment that, if present, must represent the unit in which the property value is expressed. The multiplication, division and raise operators, '*', '/' and '^' respectively, can be used to represent units. For example:

"unit:m/s"
"unit:erg*s^−1*cm^2*Hz^−1"

In the second example the represented unit must be interpreted as .

A decoder shall follow these rules to apply a format specifier to structured numeric properties:

  • A format specifier applied to a complex property shall apply to the individual complex and imaginary parts.
  • A format specifier applied to a vector property shall apply to all vector components.
  • A format specifier applied to a matrix property shall apply to all matrix elements.

For examples of property format specifiers, see the Property core element.

8.4.3.1 Date and Time Format

Property format specifiers are not applicable to TimePoint XISF properties. Representation of dates and times is left as implementation-defined in this version of the XISF specification.

8.4.4 Property Types

XISF property types are organized into six categories: scalar, complex, string, time point, vector and matrix properties.

8.4.4.1 Scalar Property Types

A scalar is a single numeric value without internal structuration.

Table 3
XISF Scalar Property Types
Type name Alternate name

Description

Boolean

An unsigned 8-bit integer representing one of the logical values false and true. The logical value false is represented by the integer value 0, and the logical value true is represented by the integer value 1.

Int8

A signed two's complement 8-bit integer.

UInt8

Byte

An unsigned 8-bit integer.

Int16

Short

A signed two's complement 16-bit integer.

UInt16

UShort

An unsigned 16-bit integer.

Int32

Int

A signed two's complement 32-bit integer.

UInt32

UInt

An unsigned 32-bit integer.

Int64

A signed two's complement 64-bit integer.

UInt64

An unsigned 64-bit integer.

Float32

Float

An IEEE 754[26] binary32 floating point value (single precision).

Float64

Double

An IEEE 754 binary64 floating point value (double precision).

8.4.4.2 Complex Property Types

A complex number is a homogeneous, ordered sequence of two contiguous scalars:

  • The two scalars that form a complex number are its components.
  • The first component of a complex number is its real part.
  • The second component of a complex number is its imaginary part.
Table 4
XISF Complex Property Types
Type name Alternate name

Description

Complex32

A complex number whose components are encoded as two Float32 values.

Complex64

Complex

A complex number whose components are encoded as two Float64 values.

8.4.4.3 String Property Type

A string is a homogeneous sequence of zero or more contiguous Unicode[15] code points encoded as unsigned integer scalars.

  • The Unicode code points contained in a string are string elements or characters.
  • The number of elements or characters in a string is the string's length.
  • An empty string is a string with zero length.
  • There is no specific limit for the length of a string.
Table 5
XISF String Property Type
Type name Alternate name

Description

String

A Unicode string encoded in UTF-8,[16] [17] whose elements are UInt8 scalars. Can be also a sequence of characters in an 8-bit encoding compatible with Unicode, such as ASCII[18] or ISO/IEC 8859-1.[33]

It must be pointed out that the length of a string is in general not the same as (and usually smaller than) the length in bytes of the data block required to store it. This happens because UTF-8 is a variable-length encoding scheme where a single code point may require from one to four bytes to be represented.

8.4.4.4 TimePoint Property Type

In the context of this specification, a time point is an instant in the Coordinated Universal Time (UTC)[39] time scale.

Table 6
XISF Time Point Property Type
Type name Alternate name

Description

TimePoint

A time point encoded as an ASCII[18] string that shall comply with the ISO 8601 extended specification for representations of dates and times.[32] [5]

In ISO 8601 extended representations of time points, decimal fractions shall be divided from integer parts exclusively by the full stop or dot character ('.', ASCII code point 4610 = 2E16).

A time zone should always be specified in a time point representation to eliminate any possible ambiguity about the difference of a time point with respect to UTC. If no time zone is specified in the ISO 8601 extended representation, the time point shall be assumed to be UTC, that is, the "Z" time zone shall be assumed by default.

TimePoint properties don't have a property format specifier. As stated above, representation of dates and times is left as implementation-defined in this version of the XISF specification.

8.4.4.5 Vector Property Types

A vector is a homogeneous sequence of zero or more contiguous scalars or complex numbers.

  • The scalars or complex numbers contained in a vector are vector components.
  • The number of components in a vector is the vector's length.
  • An empty vector is a vector with zero length.
  • There is no specific limit for the length of a vector.
Table 7
XISF Vector Property Types
Type name Alternate name

Description

I8Vector

A vector whose components are Int8 values.

UI8Vector

ByteArray

A vector whose components are UInt8 values.

I16Vector

A vector whose components are Int16 values.

UI16Vector

A vector whose components are UInt16 values.

I32Vector

IVector

A vector whose components are Int32 values.

UI32Vector

UIVector

A vector whose components are UInt32 values.

I64Vector

A vector whose components are Int64 values.

UI64Vector

A vector whose components are UInt64 values.

F32Vector

A vector whose components are Float32 values.

F64Vector

Vector

A vector whose components are Float64 values.

C32Vector

A vector whose components are Complex32 values.

C64Vector

A vector whose components are Complex64 values.

8.4.4.6 Matrix Property Types

A matrix is a homogeneous set of contiguous scalars or complex numbers, structured as a two-dimensional array with a fixed number of rows and columns.

  • The scalars or complex numbers contained in a matrix are matrix elements.
  • The number of rows and columns in a matrix are the matrix dimensions.
  • The total number of elements in a matrix is its length, which is equal to the product of its dimensions.
  • An empty matrix is a matrix with zero length.
  • There are no specific limits for the dimensions or the length of a matrix.
  • A matrix shall be stored as a contiguous sequence of matrix elements in row order: all matrix elements of the first row shall be stored by column order at consecutive locations, followed by all matrix elements of the second row, and so on.
Table 8
XISF Matrix Property Types
Type name Alternate name

Description

I8Matrix

A matrix whose elements are Int8 values.

UI8Matrix

ByteMatrix

A matrix whose elements are UInt8 values.

I16Matrix

A matrix whose elements are Int16 values.

UI16Matrix

A matrix whose elements are UInt16 values.

I32Matrix

IMatrix

A matrix whose elements are Int32 values.

UI32Matrix

UIMatrix

A matrix whose elements are UInt32 values.

I64Matrix

A matrix whose elements are Int64 values.

UI64Matrix

A matrix whose elements are UInt64 values.

F32Matrix

A matrix whose elements are Float32 values.

F64Matrix

Matrix

A matrix whose elements are Float64 values.

C32Matrix

A matrix whose elements are Complex32 values.

C64Matrix

A matrix whose elements are Complex64 values.

8.5 XISF Image

8.5.1 Structure and Properties

In the context of this specification, an image is an object with the following structure and properties:

  • An image is a set of scalars or complex numbers structured as a non-empty collection of -dimensional arrays.[45]
  • The dimensionality of the image is .
  • Each -dimensional array is known as a channel or (hyper)plane of the image.
  • All image channels have identical lengths in each one of their dimensions.
  • Let be the number of channels in the image. Any channel must be addressable by its channel index , an integer such that .
  • Let be the length of any channel in the -th dimension, .
  • An image with any length for is an empty image. Empty images cannot be serialized in an XISF unit.
  • A pixel coordinate is an integer array index such that .
  • Any -tuple formed with the set of scalars or complex numbers at the same pixel coordinates is a pixel.
  • The components of a pixel are pixel samples.
  • Any pixel must be addressable by its -tuple of pixel coordinates.
  • Any pixel sample must be addressable by the -tuple of channel index and pixel coordinates.
  • The total number of pixels in an image is .
  • The total number of pixel samples in an image is .
  • For images with a visual representation role, pixel samples must be representable in a color model.[49] [38]
  • The color model where pixel samples are represented, if defined, should be augmented with a number of viewing conditions and colorimetric properties, which define the color space[50] of the image.
  • The first channels strictly required to define the color model (or color space) of an image, with indexes , are nominal channels.
  • Excess channels beyond nominal channels, with indexes , are collectively known as alpha channels.
  • Although the roles of alpha channels are implementation-defined, the first alpha channel, at index , should define the transparency of the image for alpha-compositing operations.[53]

The mathematical symbols and terminology introduced in the above list will be used in the rest of this section.

Figure 1 — Structure of a two-dimensional image.

8.5.2 Geometrical Conventions

In a two-dimensional image we have and the lengths and are known as the width and height of the image, respectively. Conventionally, the dimensions and are known as the X-axis and Y-axis, respectively. It is customary to use the symbols and to represent coordinates in the first and second dimensions, respectively. We also say that is the horizontal axis and is the vertical axis. The point with pixel coordinates is said to be the top-left corner or the origin of the image. Note that all of these words don't necessarily imply any particular orientation of the represented image; they are just conventional designations.

In a three-dimensional image, and the dimensions , and are the width, height and depth of the image, respectively. The dimensions , and are known conventionally as the X-axis, Y-axis and Z-axis, respectively. It is customary to use the symbols , and to represent coordinates in the first, second and third dimensions, respectively. is the horizontal axis, is the vertical axis and is the depth axis. The point with pixel coordinates is said to be the origin of the image. Again, these designations are purely conventional and don't imply any particular representation of the image.

For one-dimensional images and images with dimensionalities higher than three, such as tesseracts and other hypercubes, there are no conventional names for axes and lengths. Normally these structures are manipulated as mathematical objects in purely abstract terms without a perceptual meaning or a visual role.

8.5.3 Pixel Storage Models

Two pixel storage models are formalized by this specification: planar and normal. The planar storage model is appropriate for applications that store and organize images by separate channels in memory. The normal storage model is best suited for applications that organize images by pixels, that is, applications that store the components of each pixel at consecutive memory locations.

Irrespective of the pixel storage model, the total space in bytes required to store an uncompressed image channel equals , where is the size in bytes of a pixel sample. Similarly, the total space in bytes required to store an uncompressed image is .

8.5.3.1 Planar Pixel Storage Model

In this model each channel is stored in a separate block:

  • In the planar storage model, each channel of an image shall be stored as a contiguous sequence of pixel samples.
  • For a given channel index , all pixel samples shall be stored in pixel coordinate order. The coordinates of the stored sequence of pixel samples can be represented as:

    [1]

Figure 2 — A two-dimensional image in the planar pixel storage model. Each square represents a pixel sample with its coordinates written vertically, where is a channel index and and are pixel coordinates. All pixel samples are stored as a single contiguous block from top to bottom and left to right (arrows indicate storage order).

8.5.3.2 Normal Pixel Storage Model

In this model all pixel samples are stored in a single block:

  • In the normal storage model, all pixel samples of an image shall be stored as a contiguous sequence.
  • All pixels shall be stored in pixel coordinate order. The coordinates of the stored sequence of pixel samples can be represented as:

    [2]

Figure 3 — A two-dimensional image in the normal pixel storage model. As in Figure [2], each square represents a pixel sample with its coordinates written vertically. Colors have been used to indicate image channels. As you can see by comparing both figures, in the normal model pixel samples are grouped by coordinates (by rows and columns in the case of a 2-D image) instead of by channels.

8.5.4 Color Spaces

The following color spaces are formally supported by this specification for storage of pixel data in XISF units: Grayscale (monochrome), RGB, and CIE L*a*b*.

An in-depth description of color spaces is beyond the scope of this specification. The reader should refer to the literature for further information on this subject; for example, see references [49], [50] and [54].

To remove any possible ambiguity in the specification of CIE L*a*b*, we'll describe the necessary color space transformation algorithms in the next subsections.

8.5.4.1 RGB Working Space

All color space transformations shall be performed relative to a colorimetrically defined RGB working space (RGBWS) associated with the image. An RGB working space is a set , where:

  • is a reference white. In this specification, all RGB working space parameters shall be relative to the standard D50 reference white.[31]
  • is an exponent for linearization of RGB components.
  • and are the chromaticity coordinates of the RGB primaries.
  • are the luminance coefficients of the RGB primaries.

All chromaticity coordinates and luminance coefficients shall be represented normalized to the [0,1] range.

For spaces not originally defined with respect to the D50 reference white, a chromatic adaptation algorithm must be applied to convert coordinates and coefficients to D50. The Bradford chromatic adaptation transform[29] is recommended.

When no RGB working space is specified for a given image, the default RGBWS associated with the image shall be the sRGB space.[27] The chromaticity coordinates and luminance coefficients of sRGB relative to D50 (after chromatic adaptation from the D65 reference white with the Bradford algorithm) are the following:

The sRGB space uses a special linearization function instead of a gamma exponent, as described below.

8.5.4.2 RGB to CIE XYZ Conversion

Let R, G, B be the nominal components of a pixel in the RGB color space. The nominal components R', G', B' in the linear RGB space are

[3]

If the RGB working space is the sRGB space,[27] then a special function must be used instead of the equations above. Define the linearization function

[4]

Now we have, for the sRGB space:

[5]

The CIE XYZ components are given by

[6]

with the transformation matrix

[7]
8.5.4.3 CIE XYZ To RGB Conversion

Let X, Y, Z be the nominal components of a pixel in the CIE XYZ color space. The components R', G', B' in the linear RGB space can be found by multiplication with the inverse transformation matrix:

[8]

Then the RGB components are given by

[9]

If the RGB working space is the sRGB space,[27] then a special function must be used instead of the equations above. Define the delinearization function

[10]

Now we have, for the sRGB space:

[11]
8.5.4.4 CIE XYZ to CIE L*a*b* Conversion

Define the function

[12]

Then the CIE L*a*b* components are:

[13]
8.5.4.5 CIE L*a*b* to CIE XYZ Conversion

Define the function

[14]

Then the CIE XYZ components are:

[15]
8.5.4.6 RGB to Grayscale Conversion

A colorimetrically defined grayscale component shall be computed as the L* component of the CIE L*a*b space.

9 XISF Structure

[hide]

9.1 XISF Unit

An XISF unit shall consist of a unique XISF header and zero or more XISF data blocks. Two storage models are supported and formally defined in this specification: monolithic and distributed:

The difference between the monolithic and distributed storage models is that a monolithic XISF unit must store the header and all of the data pertaining to the XISF unit in a single file, while a distributed XISF unit must store the header and the data separately in different files.

Figure 4 — Structure of a monolithic XISF unit.

Figure 5 — Structure of a distributed XISF unit.

9.2 File Name Suffixes

The following file name suffixes, also known as file name extensions in some operating systems, are mandatory for files pertaining to XISF units:

9.3 XISF Header File

An XISF header file is a file that must contain exclusively the unique XISF header of a distributed XISF unit.

9.4 XISF Data Block

An XISF data block is a sequence of one or more contiguous bytes pertaining to an XISF unit.

The location and role of an XISF data block must be completely and unambiguously defined by the unique XISF header of the XISF unit to which it pertains.

An XISF data block must be either attached to a monolithic XISF file, stored in an XISF data blocks file, serialized directly in the XISF header, or accessible as an external resource. See the section on XISF data block location for detailed information.

9.5 Attached XISF Data Block

An attached XISF data block is an XISF data block stored in a monolithic XISF file.

The position of an attached data block from the beginning of the monolithic XISF file where it is stored, as well as its length in bytes, must be completely and unambiguously defined by the XISF header of the XISF unit.

9.6 External XISF Data Block

An external XISF data block is an XISF data block stored separately from the XISF header of a distributed XISF unit. An external XISF data block can be stored either in an XISF data blocks file, or in a generic file.

The position of an external data block and its length in bytes must be completely and unambiguously defined by the XISF header and, when applicable, by the XISF data blocks file where the block is stored.

9.7 Monolithic XISF File

A monolithic XISF file shall have the following structure:

Signature

A sequence of eight contiguous bytes whose values must form the set 'XISF0100' of ASCII characters,[18] that is, the sequence 8810 7310 8310 7010 4810 4910 4810 4810 of unsigned 8-bit integers.

Header length

An unsigned 32-bit integer whose value must be the length in bytes of the XISF header.

Reserved field

Four contiguous bytes reserved for future use. The values of these bytes shall be zero.

XISF header

The unique XISF header of the XISF unit.

Unused space (optional)

A sequence of zero or more contiguous bytes that may exist immediately after the XISF header. The bytes in an unused space must be zero.

Attached XISF data blocks (optional)

Zero or more attached XISF data blocks that may exist, either immediately after the XISF header, or after the optional unused space if it exists. Each attached data block is a sequence of one or more contiguous bytes. Attached data blocks may be separated by unused space, with the same definition as above.

Figure [6] shows a graphical representation of a monolithic XISF file. An XISF decoder or a file management application can use the XISF signature (the first eight bytes of the file) to quickly determine if a given file can be a valid monolithic XISF file. The header length field has the advantage that an XISF decoder can load the entire XISF header in memory using a single file read operation. Without the header length element, a decoder would have to scan the XML source code sequentially in order to find the end of the XISF header. Unused spaces can be useful to optimize I/O buffering and access to attached data blocks in some devices.

Figure 6 — Structure of a monolithic XISF file.

The following C++ code snippet shows a practical implementation of a structure to represent the initial components of a monolithic XISF file, including construction and validation.

#include <cstdint>

/*
 * Monolithic XISF file header.
 */
struct XISFMonolithicFileHeader
{
   uint8_t  magic[ 8 ];   // 'XISF0100'
   uint32_t headerLength; // length in bytes of the XML file header
   uint32_t reserved;     // reserved - must be zero

   /*
    * Constructs a monolithic XISF file header with the specified XML header
    * length in bytes.
    */
   XISFMonolithicFileHeader( uint64_t _headerLength )
   {
      magic[0] = 'X'; magic[1] = 'I'; magic[2] = 'S'; magic[3] = 'F';
      magic[4] = '0'; magic[5] = '1'; magic[6] = '0'; magic[7] = '0';
      headerLength = _headerLength;
      reserved = 0;
   }

   /*
    * Returns true if this is a valid monolithic XISF file header.
    */
   bool IsValid() const
   {
      return magic[0] == 'X' && magic[1] == 'I' && magic[2] == 'S' && magic[3] == 'F' &&
             magic[4] == '0' && magic[5] == '1' && magic[6] == '0' && magic[7] == '0' &&
             headerLength >= 65; // minimum length of an empty header from "<?xml..." to </xisf>
   }
};

9.8 XISF Data Blocks File

An XISF data blocks file is a file that may contain zero or more XISF data blocks and may contain unused space, as described below. XISF data blocks files can be available as local or remote resources.

An XISF data blocks file shall have the following structure (Figure [7]):

Signature

A sequence of eight contiguous bytes whose values must form the set 'XISB0100' of ASCII characters,[18] that is, the sequence 8810 7310 8310 6610 4810 4910 4810 4810 of unsigned 8-bit integers.

Reserved field

Eight contiguous bytes reserved for future use. The values of these bytes shall be zero.

Block index

The block index of an XISF data blocks file is a singly linked list[46] [47] [48] of block index nodes (Figure [8]). A block index node contains a sequence of block index elements. Each block index element may define the position and length of an XISF data block stored in the data blocks file.

A block index node shall have the following structure:

Length

An unsigned 32-bit integer whose value must be the number of block index elements available in this block index node.

Reserved field

Four contiguous bytes reserved for future use. The values of these bytes shall be zero.

Next node

An unsigned 64-bit integer whose value must be the byte position, measured from the beginning of the XISF data blocks file, of the next block index node. If the value of this element is zero, then this is the last block index node of the block index.

Elements

A sequence of contiguous block index elements. A block index element must define the position and length of an XISF data block stored in the XISF data blocks file. We say that a block index element points to the XISF data block whose position and length it defines.

A block index element shall be a sequence of 40 contiguous bytes with the following structure (Figure [9]):

Unique Identifier

An unsigned 64-bit integer. This identifier must be unique in the XISF data blocks file, that is, no two index elements can have the same identifier in a given data blocks file.

Block position

An unsigned 64-bit integer whose value must be the byte position, measured from the beginning of the XISF data blocks file, of the first byte of the XISF data block pointed to by the block index element.

The block position of a block index element may be zero. In such case the element is called a free block index element. A free block index element does not provide access to any data block, and can be used as a placeholder to access future data blocks that can be stored in the data blocks file.

Block length

An unsigned 64-bit integer whose value must be the length in bytes of the XISF data block pointed to by the block index element.

For free block index elements, the block length must be zero.

Uncompressed block length

An unsigned 64-bit integer. If the data block pointed to by the block index element has been compressed, the value of this item must be the length in bytes of the uncompressed data block.

For free block index elements and index elements pointing to uncompressed blocks, the uncompressed block length must be zero.

Reserved field

Eight contiguous bytes reserved for future use. The values of these bytes shall be zero.

XISF data blocks (optional)

Zero or more XISF data blocks, which may be stored in the XISF data blocks file. Each data block is a sequence of one or more contiguous bytes, whose position and length must be defined by at least one block index element pertaining to the block index of the XISF data blocks file.

Unused space (optional)

Since XISF data blocks can be stored at arbitrary locations in an XISF data blocks file, there may be sequences of zero or more contiguous bytes that don't pertain to any data block. These sequences collectively form the unused space of an XISF data blocks file. All bytes pertaining to the unused space must be zero.

Unused spaces can be useful to optimize I/O buffering and access to data blocks in some devices. They can also be useful as pre-allocated spaces for storage of new data blocks.

Figure [7] shows a graphical representation of an XISF data blocks file. An XISF decoder or a file management application can use the XISB signature (the first eight bytes of the file) to quickly determine if a given file can be a valid XISF data blocks file. Figure [8] represents the structure of a block index structure. Finally, Figure [9] outlines the structure of a block index element.

Figure 7 — Structure of an XISF data blocks file.

Figure 8 — Structure of an XISF block index.

Figure 9 — Structure of an XISF block index element.

The following C++ code snippet shows a practical implementation of a structure to represent the initial components of an XISF data blocks file, including construction and validation.

#include <cstdint>

/*
 * XISF data blocks file header.
 */
struct XISFDataBlocksFileHeader
{
   uint8_t  magic[ 8 ];   // 'XISB0100'
   uint64_t reserved;     // reserved - must be zero

   /*
    * Constructs a default XISF data blocks file header.
    */
   XISFDataBlocksFileHeader()
   {
      magic[0] = 'X'; magic[1] = 'I'; magic[2] = 'S'; magic[3] = 'B';
      magic[4] = '0'; magic[5] = '1'; magic[6] = '0'; magic[7] = '0';
      reserved = 0;
   }

   /*
    * Returns true if this is a valid XISF data blocks file header.
    */
   bool IsValid() const
   {
      return magic[0] == 'X' && magic[1] == 'I' && magic[2] == 'S' && magic[3] == 'B' &&
             magic[4] == '0' && magic[5] == '1' && magic[6] == '0' && magic[7] == '0';
   }
};

In the next code fragment we implement the XISF block index element structure with automatic generation of unique identifiers. The RandomUI64() static member function of XISFBlockIndexElement should be implemented to generate high-quality, pseudo-random uniform deviates as unsigned 64-bit integer values with a good random seeding routine. The XorShift1024* algorithm[51] [52] [55] is recommended.

#include <cstdint>

/*
 * XISF block index element.
 */
struct XISFBlockIndexElement
{
   uint64_t uniqueId;                // unique block index element identifier
   uint64_t blockPosition;           // block byte offset from file beginning
   uint64_t blockLength;             // stored block length in bytes
   uint64_t uncompressedBlockLength; // uncompressed block length in bytes
   uint64_t reserved;                // reserved - must be zero

   /*
    * Constructs a free block index element.
    */
   XISFBlockIndexElement() : uniqueId( RandomUI64() )
   {
      blockPosition = blockLength = uncompressedBlockLength = reserved = 0;
   }

   /*
    * Returns true if this is a free block index element.
    */
   bool IsFree() const
   {
      return blockPosition == 0;
   }

private:

   /*
    * Returns a pseudo-random 64-bit unsigned integer.
    */
   static uint64_t RandomUI64()
   {
      ...
   }
};

9.9 XISF Header

An XISF header shall be a valid XML 1.0 document[11] encoded in UTF-8[16] [17], with strict compliance of the syntax and properties described in this document. The functional roles of an XISF header shall be at least the following:

  • Define a number of elements and properties describing an XISF unit as a whole.
  • Define a number of elements and properties describing every object and data structure contained in an XISF unit.
  • Define the location and length of every XISF data block stored in an XISF unit.

A valid XISF header shall have the following structure:

XML declaration

A valid XISF header must begin with the following XML declaration:

<?xml version="1.0" encoding="UTF-8"?>

Initial comment (recommended)

An XISF header should include a brief comment after the XML declaration. This initial comment should (1) specify that the XML document pertains to an XISF unit by declaring the XISF format and its version, and (2) identify the software which has created the XISF unit. The following example shows the starting lines of an XISF header with the XML declaration followed by a valid initial comment:

<?xml version="1.0" encoding="UTF-8"?>
<!--
Extensible Image Serialization Format - XISF version 1.0
Created with TheMagicImagingSoftware - http://reallymagic.com/
-->

An XISF encoder should replace "TheMagicImagingSoftware" and "http://reallymagic.com/" in the comment above with the name of the software application that has created the XISF unit and a relevant URL, respectively.

An XISF decoder must not rely on the information included in the initial comment of an XISF header to modify its behavior in any way.

XISF root element

The XISF root element is the top-level XML element that defines the entire contents of an XISF unit. A unique instance of the XISF root element shall be present in every XISF header, following the XML declaration and, when it exists, the initial comment. The XISF root element has the following properties:

  • Its name shall be xisf and shall be interpreted as case-sensitive.
  • Must have the version attribute, whose value shall be "1.0".
  • Should have the following namespace[12] and schema[13] attributes:

    xmlns="http://www.pixinsight.com"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.pixinsight.com/xisf http://pixinsight.com/xsd/xisf-1.0.xsd"
  • May have child XML elements describing the objects and data structures contained in an XISF unit.
  • If the XISF unit contains some data structures or objects, then the XISF root element must have the necessary child elements to describe them.
  • Must not have character data contents.

The following example shows the initial lines of a valid XISF header with the start-tag of its XISF root element:

<?xml version="1.0" encoding="UTF-8"?>
<!--
Extensible Image Serialization Format - XISF version 1.0
Created with TheMagicImagingSoftware - http://reallymagic.com/
-->
<xisf version="1.0"
   xmlns="http://www.pixinsight.com"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://www.pixinsight.com/xisf http://pixinsight.com/xsd/xisf-1.0.xsd">

XISF child elements (optional)

XISF child elements are child XML elements of the unique XISF root element in an XISF header. XISF child elements shall describe data structures and objects stored in an XISF unit. A reduced number of XISF child elements have been formalized in this specification to support essential data structures; we call these elements XISF core elements.

XISF is an extensible format mainly because new child elements may be defined as necessary to describe new data structures and objects stored in XISF units, which have not been formalized in this specification.

XML signature (optional)

After the XISF root element, an XML signature[14] may be included to provide data integrity and authentication. The signed data shall be the XISF root element and its entire contents, including the whole XML tree: All child XML elements of the root element and their contents.

If the XISF unit is digitally signed with an XML signature, then the checksum attribute must be included and correctly implemented for every data block not serialized directly in the XISF header, including all blocks with attachment, url, absolute path and relative path block locations. With a digital signature and data checksums correctly implemented, a signed XISF unit effectively seals all of its contents and guarantees data integrity.

The XML signature of an XISF unit must be implemented as a detached signature in the header XML document. The XISF root element, which shall be the object being signed, must be specified through an id attribute in the Reference element of the signature. The following example clarifies these concepts (some parts of the code replaced with ... to shorten the example):

<?xml version="1.0" encoding="UTF-8"?>
<!--
Extensible Image Serialization Format - XISF version 1.0
Created with TheMagicImagingSoftware - http://reallymagic.com/
-->
<xisf version="1.0" id="XISFRootElement"
   xmlns="http://www.pixinsight.com"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://www.pixinsight.com/xisf http://pixinsight.com/xsd/xisf-1.0.xsd">
   ...
   ...
   ...
</xisf>
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
<SignedInfo>
   <CanonicalizationMethod Algorithm="http://www.w3.org/2006/12/xml-c14n11"/>
   <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#dsa-sha1"/>
   <Reference URI="#XISFRootElement">
      <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
      <DigestValue>...</DigestValue>
   </Reference>
</SignedInfo>
<SignatureValue>...</SignatureValue>
<KeyInfo>
   <KeyValue>
      <RSAKeyValue>
         <Modulus>...</Modulus>
         <Exponent>...</Exponent>
      </RSAKeyValue>
   </KeyValue>
   <X509Data>
      <X509SubjectName>...</X509SubjectName>
      <X509IssuerSerial>
         <X509IssuerName>...</X509IssuerName>
         <X509SerialNumber>...</X509SerialNumber>
      </X509IssuerSerial>
      <X509Certificate>...</X509Certificate>
   </X509Data>
</KeyInfo>
</Signature>

In this example, the XISF root element being signed is selected with the "XISFRootElement" identifier through a same-document reference.[6] The root element is signed with a SHA-1 digest[4] and the RSA public key of an X.509 certificate.[9]

9.10 XISF Data Block Location

The location—and, when necessary, also the dimensions—of an XISF data block must be completely defined in the XISF header of the XISF unit where the block is serialized. The location XML attribute defines the location of an XISF data block and a number of related properties. The values of a location attribute can be:

location="inline:encoding"

Defines an inline XISF data block. An inline data block is serialized in the character contents of an XML element, encoded as plain ASCII[18] text. encoding defines the data encoding used, which must be either base64 or hex, corresponding to the Base64 and Base16 (or hexadecimal) encodings,[8] respectively. For data encoded in Base16, the digit symbols 'a', 'b', 'c', 'd', 'e' and 'f' shall be used to represent the nibble values 1010, 1110, 1210, 1310, 1410 and 1510, respectively; in other words, a lowercase hexadecial representation shall be used for Base16 encoding.

Example of inline XISF data block:

<Property id="TestProperty" type="ByteArray" length="34" location="inline:base64">
   VGhpcyBpcyBhIHRlc3QgLSBURVNUIC0gMTIzNDU2Nzg5MA==
</Property>

Note that white space is irrelevant and must be ignored for Base64 and Base16 encoded data.

XML elements serializing inline XISF data blocks shall not have child XML elements.

location="embedded"

Defines an embedded XISF data block. An embedded block shall be serialized in the character contents of a Data child element, encoded as plain ASCII[18] text. The Data child element must have an encoding attribute, with the same meaning and rules as for inline data blocks.

Example of embedded XISF data block:

<Image geometry="6:6:3" sampleFormat="UInt8" colorSpace="RGB" location="embedded">
   <Data encoding="base64">
      AAAAAP8A/wD/AAAAAAAAAP8AAP8AAAAAAAAA/wD/AP8AAAAA/wD//wD/AP8AAP8A/wD//wD//
      wD//wD/AP8AAP8A/wD//wD/AP8AAAAAAAAA/wD/AP8AAAAAAAAAAP8A/wD/AAAAAAAAAP8A
   </Data>
</Image>

Note that white space is irrelevant and must be ignored for Base64 and Base16 encoded data.

The Data child element serializing an embedded data block shall not have child elements or additional character data contents.

Embedded data blocks are necessary for blocks that have to be serialized within XML elements that can have child elements or character data contents. In these cases an inline serialization is forbidden by this specification.

location="attachment:position:size"

Defines an attached XISF data block in a monolithic XISF file. position and size must be plain text representations of unsigned integers. The value of position is the byte position of the first byte of the XISF data block from the beginning of the monolithic XISF file where it is stored. The value of size is the size in bytes of the XISF data block.

Example of attached XISF data block:

<Image geometry="960:540:3" sampleFormat="Float32" bounds="0:1" colorSpace="RGB"
                  location="attachment:4096:6220800" />
location="external:url(URL)"
location="external:index-id:url(URL)"

Defines an external XISF data block in a distributed XISF unit.

URL must be a valid uniform resource locator[6] specification for a local or remote file where the XISF data block should be available.

In the first form, the data block shall start at byte position zero of the specified file, and its size in bytes must be the total size of the specified file, whose value must be retrievable by some means that are beyond the scope of this specification; for example, by reading the properties of a local directory entry, or by fetching the total length of the data downloaded from a remote server.

In the second form, the specified file must be an XISF data blocks file. index-id must be a plain text representation of an unsigned integer, whose value must be the unique identifier of a block index element in the XISF data blocks file. index-id should be represented as an hexadecimal number. The referenced block index element must define the position and size of the XISF data block being located.

Example of external XISF data block stored on a remote server:

<Image geometry="4096:4096:1" sampleFormat="UInt16"
                  location="external:url(http://mysite.example.com/myfile.bin)" />

Example of XISF data block stored in an XISF data blocks file:

<Property id="AHugeOne" type="F32Matrix" rows="100000" columns="25000"
                  location="external:0x7a73526b584c6167:url(file:///data/huge-things.xisb)" />

Parentheses appearing in a URL must be written as percent-encoded octets,[6] so that the resulting location attribute value can be parsed correctly. For example:

<Property id="FooMatrix" type="Matrix" rows="12345" columns="54321"
                  location="external:url(ftp://ftp.example.com/public/example%282015%29.dat)" />

In the above example the file name being referenced is "example(2015).dat".

location="external:path(abs-file-path)"
location="external:index-id:path(abs-file-path)"

Defines an external XISF data block stored in a local file with an absolute file path specification.

abs-file-path is an absolute path[23] to a file located on, or accessible from, the local file system. File path specifications shall follow standard UNIX pathname syntax rules[21] and pathname resolution[22] on all platforms. In particular, only slash '/' characters shall be used as directory separators.

In the first form, the data block shall start at byte position zero of the specified file, and its size in bytes must be the total size of the file, whose value must be retrievable by some means that are beyond the scope of this specification; for example, by reading the properties of a local directory entry.

In the second form, the specified file must be an XISF data blocks file. index-id must be a plain text representation of an unsigned integer, whose value must be the unique identifier of a block index element in the XISF data blocks file. index-id should be represented as an hexadecimal number. The referenced block index element must define the position and size of the XISF data block being located.

In general, absolute paths are not recommended and should be avoided, since they tend to prevent distributed XISF units from being relocatable. If a unit with absolute path names is moved or copied to a different file system or machine, it will cease to be valid unless the original directory tree exists on the target file system.

In some special cases, however, absolute paths can be useful. For example, a virtualized file system can be based on absolute paths to symbolic links[25] pointing to actual directories. In this case a distributed XISF unit with absolute paths to external resources can be made relocatable by emulating the required directory tree structure through symbolic links on the deployment file system.

Parentheses appearing in a path specification must be escaped with a backslash, so that the resulting location attribute value can be parsed correctly. For example:

<Property id="DescriptionDocument" type="String8"
                  location="external:path(/Documents/description\(draft\).txt)" />
location="external:@header_path/rel-file-path"
location="external:index-id:@header_path/rel-file-path"

Defines an external XISF data block stored in a local file with a relative directory path specification.

rel-file-path is a path to a file, relative[24] to the directory where the file containing the XISF header is located on, or accessible from, the local file system. Directory path specifications shall follow standard UNIX pathname syntax rules[21] and pathname resolution[22] on all platforms. In particular, only slash '/' characters shall be used as directory separators.

In the first form, the data block shall start at byte position zero of the specified file, and its size in bytes must be the total size of the file, whose value must be retrievable by some means that are beyond the scope of this specification; for example, by reading the properties of a local directory entry.

In the second form, the specified file must be an XISF data blocks file. index-id must be a plain text representation of an unsigned integer, whose value must be the unique identifier of a block index element in the XISF data blocks file. index-id should be represented as an hexadecimal number. The referenced block index element must define the position and size of the XISF data block being located.

The following Image element shows an example of XISF data block located with a relative directory specification. The sample-screenshots.xisb data blocks file should be on the same directory as the XISF header file being decoded:

<Image geometry="1024:768:3" sampleFormat="UInt8" colorSpace="RGB"
               location="external:0x4d373e33756e480f:@header_path/sample-screenshots.xisb" />

The following example assumes that an astrometry sub-directory exists on the local directory represented by @header_path:

<Property id="AstrometricSolution" type="F64Matrix" rows="428" columns="16"
               location="external:@header_path/astrometry/solution.dat" />

Relative file paths are the best option to build relocatable distributed XISF units. A distributed unit that locates all of its external data blocks through relative paths can be transferred to any location on any file system, including multiple machines, without breaking its validity. This can't be ensured for blocks located by absolute paths.

9.11 XISF Data Block Checksum

XML elements serializing XISF data blocks can include a checksum attribute to enforce data integrity with a cryptographic hashing algorithm.

If the XISF unit is digitally signed with an XML signature, then the checksum attribute must be included and correctly implemented for every data block not serialized directly in the XISF header, including all blocks with attachment, url, absolute path and relative path block locations.

Cryptographic checksums are optional for non-signed units, although their implementation is recommended for all XISF encoders as a user-selectable option available in all cases, irrespective of digital signatures.

When a checksum attribute is defined for an XML element serializing a data block, an XISF decoder must verify it by computing a new digest from the serialized data and comparing it with the value of the corresponding checksum attribute. If the digests differ, a decoder should issue a warning or error message and give the user an option to stop loading the affected data block or the entire XISF unit, as appropriate. This validation process must always occur before making the data block available to the user, either directly or indirectly. Checksum verification may be implemented on an on-demand basis: only data blocks whose contents have to be loaded from the XISF unit need to be verified. For unattented operations such as batch processes, a decoder should not load an XISF unit for which a checksum verification fails.

See also the section on compressed XISF data blocks for important security considerations about checksums and compression.

The syntax of the checksum attribute is as follows:

checksum="algorithm:digest"

where algorithm is a cryptographic hashing algorithm, and digest is the corresponding message digest computed for the entire contents of the XISF data block being serialized.

Message digests shall be encoded in Base16 using the lowercase hexadecimal digits 'a', 'b', 'c', 'd', 'e', and 'f'.

Supported algorithms include:

Table 9
Cryptographic Hashing Algorithms

Algorithm

checksum algorithm value

Alternate algorithm value

SHA-1[4]

sha-1

sha1

SHA-256[10]

sha-256

sha256

SHA-512[10]

sha-512

sha512

SHA3-256 (Keccak)[35] [56]

sha3-256

SHA3-512 (Keccak)[35] [56]

sha3-512

SHA-1 is the recommended general-purpose hashing algorithm for use in XISF units.

Example:

<Image geometry="4096:4096:1" sampleFormat="Float32" bounds="0:1"
       location="attached:134217728:67108864"
       checksum="sha1:97b25345e3bd74bcd6613d24e3ecb47617a31d20" />

9.12 XISF Data Block Compression

An XISF data block may be transformed with a compression codec or algorithm. Both lossless and lossy compression schemes are permitted. XML elements serializing the values of compressed XISF data blocks must include a compression XML attribute with the following syntax:

compression="codec:parameters"

where codec is the name of a compression algorithm, and parameters is a set of algorithm-dependent parameters. The value of the compression attribute must be sufficient to recover meaningful data for the XISF data block being serialized. Note that we specify meaningful instead of the original because lossy compression algorithms are supported. The performance and ability of a compression algorithm to maximize compression ratios and minimize data degradation are beyond the scope of this specification.

For embedded XISF data blocks, the compression attribute shall be an attribute of the Data child element serializing the block's contents, not of the element that serializes the block. For the rest of block location modes, the compression attribute shall be included in the element that serializes the block.

9.12.1 Checksums of Compressed Blocks

When a checksum attribute is included in the XML element serializing a compressed data block, an encoder shall compute the corresponding cryptoghraphic digest for the compressed data, and never from the original, uncompressed data. If a checksum verification fails for a compressed data block, a decoder must not attempt to decompress the altered data, as this might cause a vulnerability that could be exploited.

9.12.2 Zlib Compression

The deflate compression algorithm[2] and the standard zlib format[1] are formally supported by this first XISF specification as the standard XISF compression codec. Future revisions of XISF will include more compression methods.

In particular, the implementation written by Jean-Loup Gailly and Mark Adler, the well-known zlib library,[57] is recommended by this specification.

An XML element serializing the contents of an XISF data block compressed with the zlib format must have a compression attribute with the following syntax:

compression="zlib:uncompressed-size"

where uncompressed-size is a plain text representation of an unsigned integer whose value is the size in bytes of the original uncompressed data.

Example:

<Image geometry="6:6:3" sampleFormat="UInt8" colorSpace="RGB" location="embedded">
   <Data compression="zlib:108" encoding="base64">
      eJxjYGBg+A+GEPCfAYkJFQZSUPZ/KBtTBFMXOuc/AwCjKyPd
   </Data>
</Image>

This example serializes a tiny RGB color image of 6x6 pixels, compressed and embedded within an Image element. The following example:

<Property id="Test" type="ByteArray" length="34" compression="zlib:34" location="inline:base64">
   eNoLycgsVgCiRIWS1OISBV2FENfgECBlaGRsYmpmbmFpAACzWQkd
</Property>

shows a Property element serializing a ByteArray XISF property with zlib compression as an inline data block.

This example shows a zlib-compressed image with a SHA-1 checksum:

<Image geometry="960:540:3" sampleFormat="Float32" bounds="0:1"
       colorSpace="RGB" compression="zlib:6220800" location="attachment:4570:1428362"
       checksum="sha1:1ad9a10249b7574f97c7b4771f13924603b9852d" />

10 XISF Core Elements

[hide]

XISF core elements have the following properties:

  • An XISF core element defines or references a data object serialized in an XISF unit.
  • XISF core elements shall be descendant XML elements of the unique XISF root element of an XISF header.
  • The tag names of all XISF core elements, as well as the names of their attributes, shall be interpreted as case-sensitive.

With the only exception of the Reference element, an XISF core element may have the following XML attribute:

uid="unique-elem-id"

where unique-elem-id is a unique element identifier. A unique element identifier must be a sequence of ASCII characters satisfying the following regular expression:[19]

[_a-zA-Z][_a-zA-Z0-9]*

Unique element identifiers shall be interpreted as case-sensitive.

A unique element identifier must be unique in an XISF unit, i.e., no two XISF core elements can be assigned the same unique element identifier in an XISF header.

A unique element identifier may be used to reference a core element through a Reference element.

Three main core elements have been formalized in this specification to support a number of essential components of XISF units: Property, Metadata, and Image. In addition, a number of core elements have been defined to serialize properties and auxiliary objects associated with images: FITSKeyword, ICCProfile, RGBWS, Resolution, and Thumbnail. Finally, the Reference core element allows associating a single core element with multiple data objects.

10.1 Property Core Element

The Property core element has the following properties:

  • A Property element shall serialize an XISF property.
  • An XISF header may contain multiple Property core elements.
  • A Property element may be a child element of a Metadata core element to serialize a property of the XISF unit.
  • A Property element may be a child element of an Image core element to serialize a property of an XISF image.
  • A Property element may be a child element of the unique XISF root element to serialize a standalone property.
  • A Property element can be associated with Metadata or Image core elements by means of a uid attribute and child Reference elements.
10.1.1 Property Attributes Required for All Properties

Every Property element shall have the following attributes:

id

The value of this attribute shall be the identifier of the XISF Property being serialized.

type

The value of this attribute shall identify the name of an XISF property type, which shall be the data type of the XISF Property being serialized.

10.1.2 Property format Attribute (Optional)

A Property element may have a format attribute. If present, the value of a format attribute must be a property format specifier.

Examples:

<Property id="FocalDistance" type="UInt32" value="2540" format="width:8;align:center" />
<Property id="Magnitude" type="Float32" value="1.25"
                                 format="width:6;precision:2;float:fixed;sign:force" />
<Property id="Flags" type="UInt16" value="0xff39" format="width:8;base:hex;fill:." />
<Property id="Coefficients" type="Vector64" length="1234"
                                 location="attachment:4096:9872" format="width:10;precision:3" />

These examples would generate the following plain text representations:

"  2540  "
" +1.25"
"....ff39"
"      32.1, -1.24e-08"

In the last example, note that the float:auto format token is implicitly being used by default. Only the first two vector components are shown, and in this example we assume that the decoder application is representing vectors as comma-separated lists of components, which is not part of this specification and hence is not mandatory.

XISF decoders supporting property format specifiers should apply them, as appropriate, to generate plain text representations of scalar values. This includes all scalar property types, the components of complex properties, string properties (where only width and align format tokens should be taken into account), the components of vector properties, and the elements of matrix properties.

10.1.3 Serialization of Scalar Properties

A Property element serializing a scalar XISF property shall have a value attribute, whose value shall be a valid plain text serialization of the property value. Typically, in these cases the Property element is an empty XML element.

Examples:

<Property id="HasData" type="Boolean" value="1" />
<Property id="Flags" type="UInt32" value="0x8000FFA0" />
<Property id="Volume" type="Float64" value="1.1234e+04" />
10.1.4 Serialization of Complex Properties

A Property element serializing a Complex32 or Complex64 property shall have a value attribute, whose value shall serialize the value of the XISF property as follows:

value="(real,imag)"

where real and imag are, respectively, plain text representations of the floating point real and imaginary parts of the complex number. Typically, in these cases the Property element is an empty XML element.

Example:

<Property id="PeakElement" type="Complex32" value="(0.123,-0.735e-02)" />
10.1.5 Serialization of String Properties

A Property element serializing a String property shall not have a value attribute, and must serialize the property value either directly in its character data contents, or as an XISF data block.

Examples:

<Property id="XISF:BriefDescription" type="String">Definitely one of my very best images.</Property>
<Property id="ALongString" type="String" location="attached:17600:8192" />
<Property id="AnEncodedString" type="String"
               location="inline:base64">VGhlIHF1aWNrIGJyb3duIGZveA==</Property>
<Property id="AnExternalDocument" type="String"
               location="external:url(http://mysite.example.com/MyTextDocument.txt)" />

When a string is serialized unencoded directly in the character contents of a Property element, keep in mind that all of the white space between the starting and ending tags is significant and a compliant decoder must preserve it. To prevent unwanted leading and trailing spaces, the Property element should be generated as a single text line in these cases.

10.1.6 Serialization of TimePoint Properties

A Property element serializing a TimePoint property shall have a value attribute, whose value shall be the plain text representation of the TimePoint value. Typically, in these cases the Property element is an empty XML element.

Examples:

<Property id="CreationTime" type="TimePoint" value="2014-12-01T18:07:54Z" />
<Property id="Observation:StartTime" type="TimePoint" value="2015-01-23T19:52:31.46Z" />

White space is irrelevant in a TimePoint serialization, so it should not be generated by encoders and must be ignored by decoders.

10.1.7 Serialization of Vector Properties

A Property element serializing a vector property:

  • Shall not have a value attribute.
  • Shall have a length attribute. The value of the length attribute must be a plain text representation of an unsigned integer, whose value must be the vector length, that is, the number of components in the vector.
  • Shall serialize the value of the XISF property as an XISF data block.

Example:

<Property id="TestProperty" type="ByteArray" length="34" location="inline:base64">
   VGhpcyBpcyBhIHRlc3QgLSBURVNUIC0gMTIzNDU2Nzg5MA==
</Property>
10.1.8 Serialization of Matrix Properties

A Property element serializing a matrix property:

  • Shall not have a value attribute.
  • Shall have a rows attribute. The value of the rows attribute must be a plain text representation of an unsigned integer, whose value must be the number of rows in the matrix.
  • Shall have a columns attribute. The value of the columns attribute must be a plain text representation of an unsigned integer, whose value must be the number of columns in the matrix.
  • Shall serialize the value of the XISF property as an XISF data block.

Example:

<Property id="AstrometricSolution" type="F64Matrix" rows="428" columns="16"
                        location="external:@header_path/astrometry/solution.dat" />

10.2 Metadata Core Element

The Metadata core element serializes a set of XISF properties to describe an XISF unit. Metadata properties must be serialized with child Property elements.

The identifiers of all Metadata properties have the 'XISF:' namespace prefix. These properties—including mandatory and optional properties—have reserved identifiers that must not be used for purposes different from the ones described herein.

A unique Metadata element shall be included in every XISF header as a child element of the XISF root element. The Metadata element must contain Property child elements serializing the following XISF properties:

XISF:CreationTime

A TimePoint property. The value of this property shall represent the date and time the XISF header has been created.

Example:

<Property id="XISF:CreationTime" type="TimePoint" value="2014-12-09T12:38:15Z" />

XISF:CreatorApplication

A String property, whose value must be a plain text readable identification of the software application that has created the XISF header. The format and contents of this property are implementation-dependent, but it should contain the name and version of the application in readable form.

Example:

<Property id="XISF:CreatorApplication" type="String">PixInsight 01.08.03.1123</Property>

In addition, the following properties are optional for the Metadata element, but should be serialized as child Property elements when appropriate:

XISF:CreatorModule

A String property. If the software that has created the XISF header (XISF:CreatorApplication) is a modular system where the XISF format has been implemented by a versioned subsystem or module, this property should be defined to provide a plain text readable identification of the subsystem or module that has created the XISF header. The format and contents of this property are implementation-dependent, but it should contain the name and version of the subsystem or module in readable form.

Example:

<Property id="XISF:CreatorModule" type="String">XISF module version 01.00.00.0024</Property>

XISF:CreatorOS

A String property. This property should be defined to provide a plain text readable identification of the operating system where the XISF header has been created. The following values must be used to identify several common operating systems:

Table 10
CreatorOS Property Values

Operating system

CreatorOS property value

FreeBSD

FreeBSD

Linux

Linux

Apple OS X

OS X

Microsoft Windows

Windows

Example:

<Property id="XISF:CreatorOS" type="String">Linux</Property>

XISF:CompressionMethod

A String property. If the XISF unit contains compressed data blocks, this property should be defined to enumerate the applied compression algorithm(s) used. If more than one algorithm has been used, they should be specified as a comma-separated list.

Example:

<Property id="XISF:CompressionMethod" type="String">zlib</Property>

XISF:CompressionLevel

An Int32 property. If the XISF unit contains data blocks compressed with the standard zlib method,[1] this property should be defined to specify the compression level used. Zlib's compression level is an integer parameter in the range from 0 to 9 that controls the speed/compression balance.

Example:

<Property id="XISF:CompressionLevel" type="Int" value="6" />

XISF:Author

An optional String property. The name(s) of the person(s) who have created the XISF unit.

Example:

<Property id="XISF:Author" type="String">James T. Kirk</Property>

XISF:Copyright

An optional String property. Copyright information applicable to the materials included in the XISF unit.

Example:

<Property id="XISF:Copyright" type="String">Copyright (c) 2014 Extragalactic Imaging
Corporation. All Rights Reserved.</Property>

XISF:BriefDescription

An optional String property. A brief description or caption for the materials included in the XISF unit. If this metadata property is specified, it should be a brief text not longer than 256 characters.

Example:

<Property id="XISF:BriefDescription" type="String">Messier 45, The Pleiades Cluster</Property>

XISF:Description

An optional String property. A description of the materials included in the XISF unit.

Example:

<Property id="XISF:Description" type="String" location="attached:2048:8325" />

XISF:Keywords

An optional String property. A comma-separated list of keywords or search index terms applicable to the materials included in the XISF unit, specified in decreasing order of relevance.

Example:

<Property id="XISF:Keywords" type="String">IC 342, spiral galaxy, deep-sky object</Property>

10.3 Image Core Element

The Image core element defines an XISF image serialized in an XISF unit. Image elements shall be child elements of the unique XISF root element of the XISF unit.

The entire pixel data contents of an XISF image shall be serialized as a single XISF data block. An Image element cannot serialize pixel data as an inline XISF data block, this restriction obeying to the fact that Image elements can have child XML elements.

An Image element can have the following attributes:

geometry="dim1:...:dimN:channel-count"

This attribute shall be specified for all Image elements.

Defines the geometry of an N-dimensional image. The number N ≥ 1 of dim items is the dimensionality of the image: N=1 for a one-dimensional image, N=2 for a two-dimensional image, etc. Each dimi item is a plain text representation of an unsigned integer whose value, which must be greater than zero, is the corresponding length of the image on its i-th axis in pixel units. The last channel-count item is a plain text representation of an unsigned integer whose value, which must be greater than zero, is the number of existing image channels or planes.

For a two-dimensional or three-dimensional image, the first dimension corresponds to the X-axis of the image and the second dimension corresponds to its Y-axis. For a three-dimensional image, the third dimension corresponds to the Z-axis.

Example:

<Image geometry="960:540:3" sampleFormat="Float32" bounds="0:1"
       colorSpace="RGB" compression="zlib:6220800" location="attachment:4570:1428362" />

This example defines a two-dimensional image with three channels. Its width is 960 pixels and its height is 540 pixels.

sampleFormat="sample-format"

This attribute shall be specified for all Image elements.

Defines the data type used to represent the pixel samples of the image, also known as the sample format of the image. sample-format must be one of the following literals:

Table 11
sampleFormat Attribute Values

Sample data type

sampleFormat attribute value

Unsigned 8-bit integer

UInt8

Unsigned 16-bit integer

UInt16

Unsigned 32-bit integer

UInt32

Unsigned 64-bit integer

UInt64

Real IEEE 754[26] binary32 floating point

Float32

Real IEEE 754 binary64 floating point

Float64

Complex IEEE 754 binary32 floating point

Complex32

Complex IEEE 754 binary64 floating point

Complex64

bounds="lower:upper"

This attribute shall be specified for all Image elements serializing floating point real pixel data. It is optional for Image elements serializing integer pixel data, and must not be specified for complex images. These restrictions are further described and explained below.

The bounds attribute defines the representable range of a real or integer image in the grayscale or RGB color spaces. lower is a plain text representation of a floating point number, whose value is the lower bound of the representable range, or the black point of the image. upper is a plain text representation of a floating point number, whose value is the upper bound of the representable range, or the white point of the image.

Consider an output device able to generate image representations for pixel sample values in the [v0,v1] range, such that v0 < v1. We call this range the device's output range. Pixel samples outside the output range will generate saturated representations: Any pixel sample value ≤ v0 will be represented as black, while any pixel sample value ≥ v1 will be represented as white. Pixel samples within the output range will generate unsaturated representations as gray levels directly proportional to the sample values. Assuming an image representation device so defined, the representable range of the image shall be applied according to the following algorithm:

r0 ← valueOf( lower )
r1 ← valueOf( upper )
if r1 < r0
exchange( r0, r1 )
end
for each pixel sample f
if r1 = r0
f ← 0
else
if f < r0
f ← r0
else if f > r1
f ← r1
end
f ← (f – r0)/(r1 – r0)
end
output( v0 + (v1 – v0)×f )
end

where the output(x) abstract operation sends a pixel sample x for representation on the output device, as described above. This algorithm is a simple linear mapping of pixel sample values from the representable range of an image to a device's output range.

For example, for an image being deserialized to be represented on a typical 8-bit output device, such as a computer display or a printer, we would have v0 = 0 and v1 = 255. Another example is an image processing application working internally with normalized floating point pixel data in the [0,1] range, which would apply the above algorithm with v0 = 0 and v1 = 1 (note that in this case the algorithm can be simplified).

The bounds attribute is mandatory for floating point real images, that is, for images where the value of the sampleFormat attribute is either Float32 or Float64.

For integer images, where the value of the sampleFormat attribute is UInt8, UInt16, UInt32 or UInt64, the bounds attribute is optional. If the bounds attribute is not specified for an integer image, the representable range shall be [0,2n–1], where n is the number of bits per pixel sample. For integer n-bit images using representable ranges equal to the default [0,2n–1] range, the bounds attribute should not be specified.

For complex images, where the value of the sampleFormat attribute is Complex32 or Complex64, the bounds attribute must not be specified. The reason for this restriction is that a visually representable range, as we have described it with the above algorithm, does not really make sense for data in the frequency domain. The methods used for representation of complex-valued pixel data are left as implementation-defined by this specification.

For images expressed in color spaces different from grayscale or RGB, the representable range defined by the lower and upper bounds shall apply to pixel sample values once converted to the RGB color space.

offset="sample-value"

This attribute is optional for all Image elements.

Defines a positive pixel sample value that has been added to every pixel sample in the image. An offset value (also known as pedestal) is necessary sometimes to ensure positivity of pixel data, mainly because of noise variations.

sample-value must be a plain text representation of a floating point scalar. Its value must be greater than or equal to zero, and can be subtracted from each pixel sample in the image to obtain zero-based pixel sample values. This operation is critical for image calibration and integration processes.

For Image elements without an offset attribute, the default offset value shall be zero.

colorSpace="color-space"

This attribute is optional for all Image elements.

Defines the color model or color space where pixel sample values are represented for the serialized image. color-space must be one of the following literals:

Table 12
colorSpace Attribute Values

Color space

colorSpace attribute value

Grayscale (monochrome)

Gray

RGB

RGB

CIE L*a*b*

CIELab

For descriptions of these color spaces and the corresponding transformation algorithms, see the XISF Image section.

For Image elements without a colorSpace attribute, the default grayscale color space shall be assumed.

pixelStorage="pixel-storage"

This attribute is optional for all Image elements.

Defines the pixel storage model used to organize the stored pixel data of the image. pixel-storage must be one of the following literals:

Table 13
pixelStorage Attribute Values

Pixel storage model

pixelStorage attribute value

Planar pixel storage model

Planar

Normal pixel storage model

Normal

For Image elements without a pixelStorage attribute, the default planar pixel storage model shall be assumed.

imageType="type-spec"

This attribute must be generated by encoders—typically, by image acquisition applications—producing raw frames of the bias, dark, flat field and science/light types (see the table below). For the rest of image types this property is optional, but generating it is strongly recommended when appropriate.

type-spec shall be one of the following literals:

Table 14
imageType Attribute Values

Image type

imageType attribute value

Bias frame

Bias

Dark frame

Dark

Flat field

Flat

Light or science frame

Light

Integrated bias frame

MasterBias

Integrated dark frame

MasterDark

Integrated flat field

MasterFlat

Integrated light or science image

MasterLight

Defect map

DefectMap

Pixel rejection map, high values

RejectionMapHigh

Pixel rejection map, low values

RejectionMapLow

Slope map

SlopeMap

A master frame is the result of the integration of two or more individual frames or images of the same type.

A defect map is an integer or floating point real image where nonzero pixel sample values represent invalid or defective pixels. Defective pixels are typically ignored or replaced with plausible statistical estimates, such as robust averages of neighbor pixels.

Rejection maps are integer or floating point real images where each pixel sample value is proportional to the number of rejected pixels in an integration process at the corresponding pixel coordinates. Low and high rejection maps correspond, respectively, to rejected pixels below and above a central reference value (typically a robust location estimate such as the median of a set of integrated pixels). A rejection map sample value equal to the lower bound of the representable range corresponds to zero rejected pixels, while the upper bound indicates that all integrated pixels have been rejected.

Slope maps are integer or floating point real images where each pixel sample value is proportional to the slope of a straight line fitted to a set of integrated pixels at the corresponding pixel coordinates. A slope map value equal to the lower bound of the representable range corresponds to a horizontal line (or a slope of zero degrees), while the upper bound represents a vertical line (infinite slope).

cfaType="cfa-spec"

This attribute should be specified for Image elements serializing images mosaiced with a Color Filter Array (CFA)[36], such as a Bayer filter. [37] [41] This attribute shall not be specified for images not mosaiced with a CFA.

The supported cfaType attribute values are:

Table 15
cfaType Attribute Values

Color filter array

cfaType attribute value

BGGR Bayer filter

BGGR

GRBG Bayer filter

GRBG

GBRG Bayer filter

GBRG

RGGB Bayer filter

RGGB

CYGM (cyan/yellow/green/magenta) filter

CYGM

Fujifilm X-Trans filter

XTrans

id="image-id"

This attribute is optional for all Image elements. image-id shall be a sequence of ASCII characters satisfying the following regular expression:[19]

[_a-zA-Z][_a-zA-Z0-9]*

This attribute may be used by a decoder to identify images in a software application. image-id must be considered as case-sensitive.

uuid="uuid"

This attribute is optional for all Image elements. It may be generated to uniquely identify (with overwhelming probability) an XISF image object.

uuid must be a plain text representation of a Universally Unique Identifier (UUID)[7] in canonical form. Version 4 UUIDs must be used with a pseudo-random number generator of cryptographic quality. The XorShift1024* algorithm[51] [52] [55] is recommended by this specification.

Example:

<Image uuid="c5c93b6d-9072-4e85-9548-1a5391377683"
       geometry="960:540:3" sampleFormat="Float32" bounds="0:1"
       colorSpace="RGB" compression="zlib:6220800" location="attachment:4570:1428362" />
orientation="rotation"

This attribute is optional for all Image elements.

rotation defines geometric operations that can be applied to change the default orientation of the image. rotation must be one of:

0

The default image orientation should be preserved, so do nothing.

flip

Flip (reflect) the image horizontally.

90

Rotate the image by 90 degrees, counter-clockwise direction.

90;flip

Rotate the image by 90 degrees in the counter-clockwise direction, then flip it horizontally.

-90

Rotate the image by 90 degrees, clockwise direction.

-90;flip

Rotate the image by 90 degrees in the clockwise direction, then flip it horizontally.

180

Rotate the image by 180 degrees.

180;flip

Rotate the image by 180 degrees, then flip it horizontally (equivalent to a vertical reflection).

If the orientation attribute is present for an Image element, a decoder supporting image orientation should apply the appropriate rotation and/or reflection upon loading an image to be represented visually.

However, a decoder must not apply orientation transformations to images loaded for image processing tasks that depend on the physical disposition of pixel data. For example, an image calibration process should not reorient image data, since this might invalidate the application of master calibration frames.

10.3.1 Astronomical Image Properties

In this subsection we formalize a reduced set of Property core elements designed specifically to support astronomical image data. The Property elements described here may be generated by encoders as child XML elements of Image core elements. They may also be associated with images by means of uid attributes and child Reference elements.

The identifiers of these properties use namespaces to organize them into several fundamental categories: Observer, Organization, Observation, Instrument, and Processing. These properties have reserved identifiers that must not be used for purposes different from the ones described herein.

All of these properties are optional. However, all encoders—especially image acquisition applications—should implement them as necessary and appropriate, following the norms and recommendations given in the list below.

Observer:Name

A String property. The name or identification of the person or group that acquired the image or carried out the observation.

Example:

<Property id="Observer:Name" type="String">Nyota Uhura</Property>

Organization:Name

A String property. The name or identification of the organization (such as a company, institution, etc.) responsible for the data acquired during this observation.

Observation:Equinox

A Float64 property. The equinox of equatorial coordinates expressed in years. A value before 1984 is a Besselian epoch; otherwise it is a Julian epoch.

This property applies to the coordinates given by the following properties: Observation:Object:RA, Observation:Object:Dec, Observation:Center:RA, and Observation:Center:Dec.

If this property is not specified, the default value shall be 2000.0 (J2000.0 = JD 2451545.0).

Example:

<Property id="Observation:Equinox" type="Float64" value="1991.25" />

Observation:CelestialReferenceSystem

A String property. Identifies the coordinate reference system to which celestial equatorial coordinates are referred.

This property applies to the coordinates given by the following properties: Observation:Object:RA, Observation:Object:Dec, Observation:Center:RA, and Observation:Center:Dec.

If this property is not specified, the default reference system shall be the International Celestial Reference System (ICRS).[58]

Example:

<Property id="Observation:CelestialReferenceSystem" type="String">FK5</Property>

Observation:GeodeticReferenceSystem

A String property. Identifies the geodetic system or datum to which geographic coordinates are referred.

This property applies to the coordinates given by the following properties: Observation:Location:Longitude, Observation:Location:Latitude, and Observation:Location:Elevation.

If this property is not specified, the default datum shall be the World Geodetic System, WGS 84.[40]

Example:

<Property id="Observation:GeodeticReferenceSystem" type="String">NAD83</Property>

Observation:Center:RA

A Float64 property. The Right Ascension equatorial coordinate of the center of the image, expressed in degrees. This property refers to the coordinates the telescope has been aimed at in the sky, not to an astrometric solution of the image.

Example:

<Property id="Observation:Center:RA" type="Float64" value="195.4997911" />

Observation:Center:Dec

A Float64 property. The Declination equatorial coordinate of the center of the image, expressed in degrees. This property refers to the coordinates the telescope has been aimed at in the sky, not to an astrometric solution of the image.

Example:

<Property id="Observation:Center:Dec" type="Float64" value="47.2661464" />

Observation:Object:Name

A String property. The name of the observed object. The value of this property must follow the IAU recommendations for nomenclature of astronomical objects.[59]

Example:

<Property id="Observation:Object:Name" type="String">NGC 5195</Property>

Observation:Object:RA

A Float64 property. The Right Ascension equatorial coordinate of the observed object, expressed in degrees.

Observation:Object:Dec

A Float64 property. The Declination equatorial coordinate of the observed object, expressed in degrees.

Observation:Location:Name

A String property. The name or identification of the observation location.

Example:

<Property id="Observation:Location:Name"
          type="String">Observatorio de Aras de los Olmos (OAO)</Property>

Observation:Location:Longitude

A Float64 property. The geographic longitude of the observation location in degrees.

Observation:Location:Latitude

A Float64 property. The geographic latitude of the observation location in degrees.

Observation:Location:Elevation

A Float64 property. The geographic elevation of the observation location in meters.

Observation:Time:Start

A TimePoint property. The date and time on which the observation started in the UTC time scale.

<Property id="Observation:Time:Start" type="TimePoint" value="2015-01-23T19:52:31.46Z" />

Observation:Time:End

A TimePoint property. The date and time on which the observation ended in the UTC time scale.

Observation:Title

A String property. A text fragment suitable to be included as a title in tables or graphical representations.

Example:

<Property id="Observation:Title" type="String">CCD Observations of WY Cyg</Property>

Observation:Description

A String property. A brief description of the observation.

Observation:BibliographicReference

A String property. A bibliographic reference associated with the data acquired during this observation.

Instrument:ExposureTime

A Float32 property. Total effective exposure time in seconds.

Instrument:Telescope:Name

A String property. The name of the telescope or main instrument used for this observation.

Instrument:Telescope:Aperture

A Float32 property. The aperture diameter of the optical system expressed in meters.

Instrument:Telescope:FocalLength

A Float32 property. The effective focal length of the optical system expressed in meters.

Instrument:Telescope:CollectingArea

A Float32 property. The effective aperture area of the optical system in square meters. The value of this property must exclude any obstruction present in the optical path, such as central obstructions caused by secondary mirrors.

Instrument:Sensor:XPixelSize

A Float32 property. Length of a sensor pixel on the X axis of the image, in microns. The value of this property refers to the actual region of the sensor that has accumulated light for an image pixel, that is, it must account for pixel binning if it has been used on the X axis (see the Instrument:Camera:XBinning property).

Instrument:Sensor:YPixelSize

A Float32 property. Length of a sensor pixel on the Y axis of the image, in microns. The value of this property refers to the actual region of the sensor that has accumulated light for an image pixel, that is, it must account for pixel binning if it has been used on the Y axis (see the Instrument:Camera:YBinning property).

Instrument:Sensor:Temperature

A Float32 property. Temperature of the sensor during the observation in degrees Celsius.

Instrument:Filter:Name

A String property. The name or identifier of the filter used to acquire the image.

If this property is not specified the image shall be assumed to have been acquired through no filter.

Example:

<Property id="Instrument:Filter:Name" type="String">Sloan r</Property>

Instrument:Camera:Name

A String property. The name of the camera or acquisition system used for this observation.

Example:

<Property id="Instrument:Camera:Name" type="String">Canon EOS 7D</Property>

Instrument:Camera:XBinning

An Int32 property. Pixel binning factor (≥ 1) applied in the horizontal (X-axis) direction.

Instrument:Camera:YBinning

An Int32 property. Pixel binning factor (≥ 1) applied in the vertical (Y-axis) direction.

Instrument:Camera:ISOSpeed

An Int32 property. Sensitivity of the camera in ISO speed units.[34]

Example:

<Property id="Instrument:Camera:ISOSpeed" type="Int" value="3200" />

Processing:History

A String property. The value of this property should be a human-readable description of the image processing algorithms and procedures applied to the image, from the initial raw data to its current state, in chronological order. The role of this property is merely informative; a decoder must not rely on its contents to modify its own behavior or the way it interprets an image.

The format used to generate this property is application-dependent. However, a few guidelines should be observed by encoders to produce useful and reliable information:

  • An application that processes an existing image must preserve the existing processing history information and append its own data at the end of the property value.
  • An application adding new processing history information must identify itself with its name and version, as well as the name(s) and version(s) of the module(s) or subsystem(s) used to process the image, as appropriate.
  • Processing history items, such as processing algorithms, tasks or steps, should be specified as single text lines separated by line feed characters (ASCII code point 1010 = 0A16).
  • When possible, a timestamp should be prepended to each processing history item. Timestamps must adopt the format of TimePoint XISF properties.

Example:

<Property id="Processing:History" type="String">
2015-01-25T23:55:16Z // Acquisition with Pixel-O-Matic Software version 123.4
2015-01-25T23:55:16Z Acquisition.exposure: 600s
2015-01-25T23:55:16Z Acquisition.filter: Clear
2015-01-25T23:55:16Z Acquisition.format: XISF

2015-01-26T11:10:23Z // Calibration with PixInsight 01.08.03.1123
2015-01-26T11:10:23Z // Calibration with ImageCalibration module version 01.03.00.0196
2015-01-26T11:10:23Z // Calibration with ImageCalibration process
2015-01-26T11:10:23Z ImageCalibration.inputHints: fits-keywords normalize raw cfa
   signed-is-physical up-bottom
2015-01-26T11:10:23Z ImageCalibration.outputHints: properties fits-keywords
   no-compress-data block-alignment 4096 max-inline-block-size 3072
   no-embedded-data no-resolution up-bottom
2015-01-26T11:10:23Z ImageCalibration.overscan.enabled: false
2015-01-26T11:10:23Z ImageCalibration.masterBias.enabled: true
2015-01-26T11:10:23Z ImageCalibration.masterBias.fileName: bias-BINNING_1.xisf
2015-01-26T11:10:23Z ImageCalibration.masterBias.calibrate: true
2015-01-26T11:10:23Z ImageCalibration.masterDark.enabled: true
2015-01-26T11:10:23Z ImageCalibration.masterDark.fileName: dark-BINNING_1-EXPTIME_300.xisf
2015-01-26T11:10:23Z ImageCalibration.masterDark.calibrate: true
2015-01-26T11:10:23Z ImageCalibration.masterDark.scalingFactors: 0.464
2015-01-26T11:10:23Z ImageCalibration.masterDark.optimizationThreshold: 0.00000
2015-01-26T11:10:23Z ImageCalibration.masterDark.optimizationWindow: 1024 px
2015-01-26T11:10:23Z ImageCalibration.masterFlat.enabled: true
2015-01-26T11:10:23Z ImageCalibration.masterFlat.fileName: flat-FILTER_Green-BINNING_1.xisf
2015-01-26T11:10:23Z ImageCalibration.masterFlat.calibrate: false
2015-01-26T11:10:23Z ImageCalibration.masterFlat.scalingFactors: 0.538888
2015-01-26T11:10:31Z ImageCalibration.noiseEstimates: 3.062e-04

2015-01-26T11:12:17Z // Cosmetic correction with PixInsight 01.08.03.1123
2015-01-26T11:12:17Z // Cosmetic correction with CosmeticCorrection module 01.02.04.0080
2015-01-26T11:12:17Z CosmeticCorrection: Total corrected pixels: 603

2015-01-26T11:13:05Z // Registration with PixInsight 01.08.03.1123
2015-01-26T11:13:05Z // Registration with ImageRegistration module version 01.10.02.0440
2015-01-26T11:13:05Z // Registration with StarAlignment process
2015-01-26T11:13:05Z StarAlignment.pairMatches: 464
2015-01-26T11:13:05Z StarAlignment.inliers: 1.000
2015-01-26T11:13:05Z StarAlignment.overlapping: 1.000
2015-01-26T11:13:05Z StarAlignment.regularity: 0.981
2015-01-26T11:13:05Z StarAlignment.quality: 0.983
2015-01-26T11:13:05Z StarAlignment.rmsError: 0.097
2015-01-26T11:13:05Z StarAlignment.rmsErrorDev: 0.044
2015-01-26T11:13:05Z StarAlignment.peakErrorX: 0.262
2015-01-26T11:13:05Z StarAlignment.peakErrorY: 0.313
2015-01-26T11:13:05Z StarAlignment.H11: 1.0
2015-01-26T11:13:05Z StarAlignment.H12: 0.0003
2015-01-26T11:13:05Z StarAlignment.H13: -4.4337
2015-01-26T11:13:05Z StarAlignment.H21: -0.0003
2015-01-26T11:13:05Z StarAlignment.H22: 0.9999
2015-01-26T11:13:05Z StarAlignment.H23: 17.6277
2015-01-26T11:13:05Z StarAlignment.H31: 0.0
2015-01-26T11:13:05Z StarAlignment.H32: 0.0
2015-01-26T11:13:05Z StarAlignment.H33: 1.0
</Property>

In this example, the Processing:History property has been generated by several applications and subsystems responsible for image acquisition and some preprocessing steps. Software names and versions have been prefixed with slash characters and processing tasks have been separated with empty lines, which improves readability.

Processing:Description

A String property. A description of the processing techniques and procedures that have been applied to the image.

Example:

<Property id="Processing:Description" type="String">
This image is an example result of the noise reduction techniques we described in the
January 2015 Athens workshop.
...
</Property>

10.4 FITSKeyword Core Element

The FITSKeyword core element defines a FITS header keyword[43] and associates it with an XISF image serialized in an XISF unit.

The contents of a FITSKeyword core element (i.e., the values of its name, value and comment attributes; see below) must comply with the requirements of the FITS format specification[42] for header keywords.

A FITSKeyword core element shall define the following attributes:

name="keyword-name"

keyword-name is the name of a FITS header keyword. Quoted from the FITS standard:[44]

The keyword name shall be a left justified, 8-character, space-filled, ASCII string with no embedded spaces. All digits 0 through 9 (decimal ASCII codes 48 to 57, or hexadecimal 30 to 39) and upper case Latin alphabetic characters ‘A’ through ‘Z’ (decimal 65 to 90 or hexadecimal 41 to 5A) are permitted; lower case characters shall not be used. The underscore ( ‘_’, decimal 95 or hexadecimal 5F) and hyphen (‘-’, decimal 45 or hexadecimal 2D) are also permitted. No other characters are permitted.

value="keyword-value"

keyword-value is the value of a FITS header keyword.[43] The HISTORY and COMMENT keywords don't have a value, so this attribute must be an empty string for these keywords.

comment="keyword-comment"

keyword-comment is the comment of a FITS header keyword.[43]

A FITSKeyword element must be either a child element of an Image core element, or a child element of the unique XISF root element. In the former case, the FITS header keyword shall be associated with the corresponding image. In the latter case, a uid attribute should be defined to associate the FITS header keyword with one or more Image elements by means of child Reference elements.

Multiple instances of the FITSKeyword element, and/or multiple Reference elements referencing existing FITSKeyword elements, can be children of an Image element to define a list of FITS keywords associated with an image.

The FITSKeyword element provides a compatibility layer with image data stored as legacy FITS files. An XISF decoder compatible with FITS must load all existing FITSKeyword elements and give access to them transparently, as if the original data were stored in the FITS format.

Examples (some parts of the code replaced with ... to shorten the examples):

<Image geometry="2048:2048:1" sampleFormat="Float32" bounds="0:1"
                                 colorSpace="Gray" location="attachment:12288:16777216">
...
   <FITSKeyword name="DATE-OBS" value="'2012-03-15T02:55:15'" comment="Observation start time, UT"/>
   <FITSKeyword name="EXPTIME" value="300" comment="Exposure time in seconds"/>
   <FITSKeyword name="XBINNING" value="1" comment="Binning factor, X-axis"/>
   <FITSKeyword name="YBINNING" value="1" comment="Binning factor, Y-axis"/>
   <FITSKeyword name="HISTORY" value="" comment="Processed with magic techniques"/>
...
</Image>

In the next example, a FITSKeyword element has been defined somewhere in the XISF header with uid="KWD001", and has been associated with an image through a Reference element:

<Image ... >
...
   <Reference ref="KWD001" />
...
</Image>
...
<FITSKeyword ... uid="KWD001" ... />

10.5 ICCProfile Core Element

The ICCProfile core element defines an ICC color profile[28] [30] serialized in an XISF unit.

ICC color profiles shall be serialized as XISF data blocks storing ICC profile structures.

An ICCProfile element must be either a child element of an Image core element, or a child element of the unique XISF root element. In the former case, the ICC profile shall be associated with the corresponding image. In the latter case, a uid attribute should be defined to associate the ICC profile with one or more Image elements by means of child Reference elements.

The embedded profile flag (see Section 7.2.11 of the ICC profile specification[28]) of an ICC profile stored in an XISF unit must be set to one to signify that the profile has been embedded in a file.

Note that the ICC profile specification states that all profile data shall be encoded as big-endian (Section 7 of the ICC profile specification[28]). This implies that an XISF data block storing an ICC profile structure is an exception to the native little-endian encoding of XISF.

Examples (some parts of the code replaced with ... to shorten the examples):

<Image geometry="4657:3452:4" sampleFormat="UInt8" colorSpace="RGB"
                                    location="attachment:4096:48227892">
...
   <ICCProfile location="attachment:48234496:28456"/>
...
</Image>

In the above example, the ICC color profile has been stored as an attached data block. The following example:

<Image ... >
...
   <ICCProfile compression="zlib:3024" location="inline:base64">
      eNq1lmdYE1kXx+9MeqMkAemEXoN0Aoj0LtKrjZAECCVASMCCDREVWFFEpCmCLAq44OpSZC2IBQ
      uLgBQVdYMsAsrr4io2VN5B9n10P74f9jzPnfnf35wz994zH+YPAKlPkCLgwQCAZIFIGOTpwoiI
      jGLgBgEOyAEaIAENNic9FXkMIGR4u7sy0pGkv6d/x9vhpdk9plcAgwH+v5Dh8tI5yOvcEC2KQR
      ...
      CFh98YEgFLtyW/BsA/feU/Av5Wh5JHhgWCar+xlBoAWHMIP5DOj/vKXINCGN/1gRnEi+UJeQLk
      qGF8XiZfEIecX8Dli/gpAgZfwPhHm/49v/bNK4t4G0Vf95mSuknIj4sXMbx4mUk8kYgRwOYkso
      VcY4a5qaklSI+1MF/qDBnxyJjfFxZe6wKAKwbgc+HCwnz1wsJn5OyoUQA6xf8FPPPKDg==
   </ICCProfile>
...
</Image>

shows a zlib-compressed, Base64-encoded ICC color profile serialized as an inline data block.

In the next example, an ICCProfile element has been defined somewhere in the XISF header with uid="TheProfile", and has been associated with an image through a Reference element:

<Image ... >
...
   <Reference ref="TheProfile" />
...
</Image>
...
<ICCProfile ... uid="TheProfile" ... />

10.6 RGBWS Core Element

The RGBWS core element defines the parameters of a colorimetrically-defined RGB working color space (RGBWS) serialized in an XISF unit. From the referenced section, recall that the parameters of an RGBWS are:

Reference white

All RGB working space parameters defined in an XISF unit shall be relative to the standard D50 reference white.[31]

Gamma exponent.


Chromaticity coordinates of the RGB primaries, normalized to the [0,1] range.

Luminance coefficients of the RGB primaries, normalized to the [0,1] range.

These parameters shall be serialized by an RGBWS core element with the following mandatory XML attributes:

gamma="gamma-spec"

gamma-spec can be one of:

  • A plain text representation of a floating point number, whose value > 0 is the exponent of the RGBWS.
  • The word sRGB (case-insensitive)—in this case the RGBWS shall use an sRGB[27] gamma function instead of a fixed exponent.
x="x_R:x_G:x_B"

x_R, x_G and x_B are plain text representations of floating point numbers. The values of these attributes are, respectively, the chromaticity coordinates of the red, green and blue primary colors of the RGBWS.

y="y_R:y_G:y_B"

y_R, y_G and y_B are plain text representations of floating point numbers. The values of these attributes are, respectively, the chromaticity coordinates of the red, green and blue primary colors of the RGBWS.

Y="Y_R:Y_G:Y_B"

Y_R, Y_G and Y_B are plain text representations of floating point numbers. The values of these attributes are, respectively, the luminance coefficients of the red, green and blue primary colors of the RGBWS.

The following attribute is optional for an RGBWS element:

name="rgbws-name"

rgbws-name is an arbitrary sequence of Unicode characters that will be assigned as the name of the RGBWS for identification purposes.

An RGBWS element must be either a child element of an Image core element, or a child element of the unique XISF root element. In the former case, the RGBWS shall be associated with the corresponding image. In the latter case, a uid attribute should be defined to associate the RGBWS with one or more Image elements by means of child Reference elements.

When no RGBWS element is specified for an Image core element, the default RGBWS shall be the sRGB space.[27]

Examples (some parts of the code replaced with ... to shorten the examples):

<Image ... >
...
   <RGBWS x="0.648431:0.230154:0.155886"
          y="0.330856:0.701572:0.066044"
          Y="0.311114:0.625662:0.063224" gamma="2.2"
          name="Adobe RGB (1998)" />
...
</Image>
<Image ... >
...
   <RGBWS x="0.648431:0.321152:0.155886"
          y="0.330856:0.597871:0.066044"
          Y="0.222491:0.716888:0.060621" gamma="sRGB"
          name="sRGB IEC61966-2.1" />
...
</Image>
<Image ... >
...
   <RGBWS x="0.648431:0.230154:0.155886"
          y="0.330856:0.701572:0.066044"
          Y="0.333333:0.333333:0.333333" gamma="1"
          name="Uniform Linear with sRGB Primaries" />
...
</Image>

In the next example, an RGBWS element has been defined somewhere in the XISF header with uid="SampleRGBWS", and has been associated with an image through a Reference element:

<Image ... >
...
   <Reference ref="SampleRGBWS" />
...
</Image>
...
<RGBWS ... uid="SampleRGBWS" ... />

10.7 Resolution Core Element

The Resolution core element defines the data necessary to control the resolution of an XISF image stored in an XISF unit. Resolution defines the amount of pixels represented per unit of surface on a display medium, such as a printer or a computer monitor, and can be measured either in pixels per inch or in pixels per centimeter.

A Resolution core element shall define the following mandatory attributes:

horizontal="x-resolution"

x-resolution is a plain text representation of a floating point number, whose value must be greater than zero and shall be the resolution of the image in its X-axis (or horizontal axis), measured in pixels per resolution unit (see the unit attribute below).

vertical="y-resolution"

y-resolution is a plain text representation of a floating point number, whose value must be greater than zero and shall be the resolution of the image in its Y-axis (or vertical axis), measured in pixels per resolution unit (see the unit attribute below).

The following attribute is optional for a Resolution core element:

unit="resolution-unit"

resolution-unit defines the unit of length used to express image resolution values in the X and Y axes. resolution-unit must be one of:

inch

Image resolution shall be measured in pixels per inch.

cm

Image resolution shall be measured in pixels per centimeter.

When no unit attribute is specified, image resolution shall be expressed in pixels per inch by default.

A Resolution element must be either a child element of an Image core element, or a child element of the unique XISF root element. In the former case, the resolution parameters shall be associated with the corresponding image. In the latter case, a uid attribute should be defined to associate the resolution parameters with one or more Image elements by means of child Reference elements.

When no Resolution element is specified for an Image core element, the default image resolution shall be 72.0 pixels per inch.

Examples (some parts of the code replaced with ... to shorten the examples):

<Image ... >
...
   <Resolution horizontal="120" vertical="120" unit="cm" />
...
</Image>

In the next example, a Resolution element has been defined somewhere in the XISF header with uid="PrintingResolution", and has been associated with an image through a Reference element:

<Image ... >
...
   <Reference ref="PrintingResolution" />
...
</Image>
...
<Resolution ... uid="PrintingResolution" ... />

10.8 Thumbnail Core Element

The Thumbnail core element defines a thumbnail image and associates it with an XISF image serialized in an XISF unit. A thumbnail is a two-dimensional image of relatively small dimensions providing a significant representation of a larger image. Typically, thumbnails are loaded by file browser applications to generate quick graphical previews of images stored in disk files.

A thumbnail image shall be serialized as an XISF image with the following properties:

  • Two-dimensional image.
  • UInt8 or UInt16 sample format.
  • RGB or grayscale color space.
  • Its width and height must not be greater than 1024 pixels.
  • Can have a single alpha channel.

Other than its tag name, a Thumbnail core element is identical to an Image core element, with the following restrictions:

  • The bounds attribute must not be defined. This implies that the representable range of a thumbnail image is always [0,2n–1], where n is either 8 or 16 bits.
  • The cfaType attribute must not be defined. This implies that thumbnail images associated with mosaiced images should be already demosaiced versions.
  • A child Thumbnail element shall not be present, that is, nested Thumbnail elements are not permitted.
  • A child Reference element referencing a Thumbnail element shall not be present, for the same reason as above.

A Thumbnail core element must be either a child element of an Image core element, or a child element of the unique XISF root element. In the former case, the thumbnail shall be associated with the corresponding image. In the latter case, a uid attribute should be defined to associate the thumbnail with one or more Image elements by means of child Reference elements.

Examples (some parts of the code replaced with ... to shorten the examples):

<Image ... >
...
   <Thumbnail geometry="400:300:3"
              sampleFormat="UInt8" colorSpace="RGB" location="attachment:8192:360000" />
...
</Image>

In the next example, a Thumbnail element has been defined somewhere in the XISF header with uid="TheThumbnail", and has been associated with an image through a Reference element:

<Image ... >
...
   <Reference ref="TheThumbnail" />
...
</Image>
...
<Thumbnail uid="TheThumbnail" geometry="400:300:3"
           sampleFormat="UInt8" colorSpace="RGB" location="attachment:8192:360000" />

10.9 Reference Core Element

The Reference core element associates an XISF core element with a data object serialized in an XISF unit.

A Reference core element shall define a unique XML attribute, namely:

ref="unique-elem-id"

where unique-elem-id is the unique element identifier of an existing XISF core element, which must be defined in the same XISF unit. unique-elem-id shall be the value of the uid attribute of the referenced XISF core element.

Note that the Reference element is the only XISF core element that cannot have a uid attribute. This obeys to the fact that chained references, that is, references to references, are not permitted in this specification.

Reference core elements should be used, when possible, to prevent duplicate object serializations in XISF units. Reference elements can also be used to reduce the volume of markup required to associate a core element with several objects, and/or to improve structuration of XISF headers. Consider an example where three images have been serialized as follows:

<Image id="IMG7953" geometry="4096:4096:4" sampleFormat="Float32" bounds="0:1"
                              colorSpace="RGB" location="attachment:4096:268435456">
   <RGBWS x="0.648431:0.230154:0.155886"
          y="0.330856:0.701572:0.066044"
          Y="0.311114:0.625662:0.063224" gamma="2.2"
          name="Adobe RGB (1998)" />
</Image>
<Image id="IMG7954" geometry="4096:4096:4" sampleFormat="Float32" bounds="0:1"
                              colorSpace="RGB" location="attachment:268439552:268435456">
   <RGBWS x="0.648431:0.230154:0.155886"
          y="0.330856:0.701572:0.066044"
          Y="0.311114:0.625662:0.063224" gamma="2.2"
          name="Adobe RGB (1998)" />
</Image>
<Image id="IMG7955" geometry="4096:4096:4" sampleFormat="Float32" bounds="0:1"
                              colorSpace="RGB" location="attachment:536875008:268435456">
   <RGBWS x="0.648431:0.230154:0.155886"
          y="0.330856:0.701572:0.066044"
          Y="0.311114:0.625662:0.063224" gamma="2.2"
          name="Adobe RGB (1998)" />
</Image>

As you see, the three images have been associated with identical RGB working spaces. Instead of repeating the same RGBWS element three times, we can simplify the markup considerably with Reference elements:

<Image id="IMG7953" geometry="4096:4096:4" sampleFormat="Float32" bounds="0:1"
                              colorSpace="RGB" location="attachment:4096:268435456">
   <Reference ref="RGBWS001" />
</Image>
<Image id="IMG7954" geometry="4096:4096:4" sampleFormat="Float32" bounds="0:1"
                              colorSpace="RGB" location="attachment:268439552:268435456">
   <Reference ref="RGBWS001" />
</Image>
<Image id="IMG7955" geometry="4096:4096:4" sampleFormat="Float32" bounds="0:1"
                              colorSpace="RGB" location="attachment:536875008:268435456">
   <Reference ref="RGBWS001" />
</Image>
<RGBWS uid="RGBWS001"
       x="0.648431:0.230154:0.155886"
       y="0.330856:0.701572:0.066044"
       Y="0.311114:0.625662:0.063224" gamma="2.2"
       name="Adobe RGB (1998)" />

Let's put a slightly less evident example:

<?xml version="1.0" encoding="UTF-8"?>
<!--
Extensible Image Serialization Format - XISF version 1.0
Created with TheMagicImagingSoftware - http://reallymagic.com/
-->
<xisf version="1.0"
   xmlns="http://www.pixinsight.com"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://www.pixinsight.com/xisf http://pixinsight.com/xsd/xisf-1.0.xsd">

   <Image geometry="4096:4096:4" sampleFormat="Float32" bounds="0:1"
                                 colorSpace="RGB" location="attachment:4096:268435456" />
   <Image geometry="4096:4096:4" sampleFormat="Float32" bounds="0:1"
                                 colorSpace="RGB" location="attachment:4096:268435456" />
   <Image geometry="4096:4096:4" sampleFormat="Float32" bounds="0:1"
                                 colorSpace="RGB" location="attachment:4096:268435456" />
   <Image geometry="4096:4096:4" sampleFormat="Float32" bounds="0:1"
                                 colorSpace="RGB" location="attachment:4096:268435456" />
   ...
   ...
   ...
</xisf>

In the above example we have four images serialized in an XISF header. Note that these images are identical; in fact, all of them have been serialized as the same attached XISF data block (see the location attributes), so fortunately there is no quadruplication of pixel data. However, the markup clearly leaves room for improvement. The same result can be achieved with Reference elements in a much cleaner way:

<?xml version="1.0" encoding="UTF-8"?>
<!--
Extensible Image Serialization Format - XISF version 1.0
Created with TheMagicImagingSoftware - http://reallymagic.com/
-->
<xisf version="1.0"
   xmlns="http://www.pixinsight.com"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://www.pixinsight.com/xisf http://pixinsight.com/xsd/xisf-1.0.xsd">

   <Image uid="foo_bar"
          geometry="4096:4096:4" sampleFormat="Float32" bounds="0:1"
                                 colorSpace="RGB" location="attachment:4096:268435456" />
   <Reference ref="foo_bar" />
   <Reference ref="foo_bar" />
   <Reference ref="foo_bar" />
   ...
   ...
   ...
</xisf>

References

[1] Jean-Loup Gailly, L. Peter Deutsch (1996), RFC 1950: ZLIB Compressed Data Format Specification version 3.3

[2] Jean-Loup Gailly, L. Peter Deutsch (1996), RFC 1951: DEFLATE Compressed Data Format Specification version 1.3

[3] Scott Bradner (1997), RFC 2119: Key words for use in RFCs to Indicate Requirement Levels.

[4] Donald E. Eastlake, 3rd, Paul E. Jones (2001), RFC 3174: US Secure Hash Algorithm 1 (SHA1).

[5] Chris Newman, Graham Klyne (2002), RFC 3339: Date and Time on the Internet: Timestamps

[6] Tim Berners-Lee, Roy T. Fielding, Larry Masinter (2005), RFC 3986: Uniform Resource Identifier (URI): Generic Syntax.

[7] Paul J. Leach, Michael Mealling, Rich Salz (2005), RFC 4122: A Universally Unique IDentifier (UUID) URN Namespace.

[8] Simon Josefsson (2006), RFC 4648: The Base16, Base32, and Base64 Data Encodings.

[9] David Cooper, Stefan Santesson, Stephen Farrell, Sharon Boeyen, Russell Housley, Tim Polk (2008), RFC 5280: Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile.

[10] Donald Eastlake, Tony Hansen (2011), RFC 6234: US Secure Hash Algorithms (SHA and SHA-based HMAC and HKDF).

[11] W3C Recommendation: Extensible Markup Language (XML) 1.0.

[12] W3C Recommendation: Namespaces in XML 1.0.

[13] W3C Recommendation: XML Schema Part 1: Structures.

[14] W3C Recommendation: XML Signature Syntax and Processing.

[15] The Unicode Consortium. The Unicode Standard.

[16] The Unicode Consortium. The Unicode Standard Version 7.0 – Core Specification, § 3.9: Unicode Encoding Forms: UTF-8, UTF-16, UTF-32 conversion and validation.

[17] The Unicode Consortium. The Unicode Standard Version 7.0 – Core Specification, § 3.10: Unicode Encoding Schemes: UTF-8, UTF-16 (BE/LE), UTF-32 (BE/LE) conversion and validation.

[18] The Unicode Consortium. The Unicode Standard Version 7.0 – C0 Controls and Basic Latin

[19] ECMA International. ECMAScript Language Specification 5.1 Edition, § 15.10: RegExp (Regular Expression) Objects.

[20] The Open Group Base Specifications Issue 7. IEEE Std 1003.1, 2013 Edition. printf() function definition.

[21] The Open Group Base Specifications Issue 6. IEEE Std 1003.1, 2004 Edition § 3.266 Pathname.

[22] The Open Group Base Specifications Issue 6. IEEE Std 1003.1, 2004 Edition § 4.11 Pathname Resolution.

[23] The Open Group Base Specifications Issue 6. IEEE Std 1003.1, 2004 Edition § 3.2 Absolute Pathname.

[24] The Open Group Base Specifications Issue 6. IEEE Std 1003.1, 2004 Edition § 3.319 Relative Pathname.

[25] The Open Group Base Specifications Issue 6. IEEE Std 1003.1, 2004 Edition § 3.372 Symbolic Link.

[26] IEEE Computer Society (2008), IEEE Standard for Floating-Point Arithmetic

[27] International Electrotechnical Commission, IEC 61966-2-1:1999 – Multimedia systems and equipment – Colour measurement and management – Part 2-1: Colour management – Default RGB colour space – sRGB

[28] International Color Consortium, Specification ICC.1:2010 (Profile version 4.3.0.0). Image technology colour management — Architecture, profile format, and data structure.

[29] International Color Consortium, Specification ICC.1:2010 (Profile version 4.3.0.0), Annex E.

[30] International Organization for Standardization, ISO 15076-1:2010 – Image technology colour management – Architecture, profile format and data structure – Part 1: Based on ICC.1:2010.

[31] International Organization for Standardization, ISO 3664:2009 – Graphic technology and photography – Viewing conditions

[32] International Organization for Standardization, ISO 8601:2004 – Data elements and interchange formats – Information interchange – Representation of dates and times

[33] International Organization for Standardization, ISO/IEC 8859-1:1998 – Information technology – 8-bit single-byte coded graphic character sets – Part 1: Latin alphabet No. 1

[34] International Organization for Standardization, ISO 12232:2006 – Photography – Digital still cameras – Determination of exposure index, ISO speed ratings, standard output sensitivity, and recommended exposure index

[35] National Institute of Standards and Technology (2014) Draft FIPS 202, SHA-3 Standard: Permutation-Based Hash and Extendable-Output Functions.

[36] Wikipedia article: Color filter array

[37] Wikipedia article: Bayer filter

[38] Wikipedia article: Color model

[39] Wikipedia article: Coordinated Universal Time

[40] Wikipedia article: World Geodetic System

[41] Bayer, B. E., Color Image Array. U.S. Patent 3971065, July 1976.

[42] Definition of the Flexible Image Transport System (FITS). The FITS Standard version 3.0.

[43] Definition of the Flexible Image Transport System (FITS). The FITS Standard version 3.0. § 4.1. Keyword records

[44] Definition of the Flexible Image Transport System (FITS). The FITS Standard version 3.0. § 4.1.2.1. Keyword name

[45] National Institute of Standards and Technology. Dictionary of Algorithms and Data Structures (2014): Array

[46] National Institute of Standards and Technology. Dictionary of Algorithms and Data Structures (2014): Linked list

[47] Robert Sedgewick, Kevin Wayne (2011), Algorithms, 4th Ed., Addison-Wesley Professional, § 1.3, pp. 142–155

[48] Thomas H. Cormen et al. (2009), Introduction to Algorithms, 3rd Ed., MIT Press, § 10.2, pp. 236–241

[49] R. C. González and R.E. Woods (2008), Digital Image Processing, Third Edition, Pearson / Prentice Hall, § 6.2, pp. 401–414

[50] Marko Tkalcic and Jurij F. Tasic (2003), Colour spaces: perceptual, historical and applicational background, EUROCON 2003. Computer as a Tool. The IEEE Region 8. Vol. 1, pp. 304–308

[51] Sebastiano Vigna (2014), An experimental exploration of Marsaglia's xorshift generators, scrambled (arXiv:1402.6246)

[52] Sebastiano Vigna (2014), Further scramblings of Marsaglia's xorshift generators (arXiv:1404.0390)

[53] Thomas Porter and Tom Duff (1984), Compositing Digital Images, Computer Graphics, Vol. 18, No. 3, pp 253–259.

[54] Bruce Justin Lindbloom, Bruce Lindbloom's Web Site

[55] Sebastiano Vigna, xorshift*/xorshift+ generators and the PRNG shootout (XorShift website)

[56] Guido Bertoni, Joan Daemen, Michaël Peeters and Gilles Van Assche, The Keccak sponge function family (Keccak Website)

[57] Greg Roelofs, Mark Adler, Zlib Home Site

[58] International Earth Rotation and Reference Systems Service, The International Celestial Reference System (ICRS)

[59] Strasbourg Astronomical Data Center, IAU Recommendations for Nomenclature