Author Topic: Problem integrating images that are compressed xisf  (Read 1398 times)

Offline slang

  • Member
  • *
  • Posts: 60
    • View Profile
Problem integrating images that are compressed xisf
« on: 2015 February 12 01:02:06 »
Hey.

I have a fast machine, some fast local cache disk, lotsa ram but majority of storage is network based NAS, so it's a bit slow.
To help compensate, and the fact that I've generally got oodles of CPU, i'm testing the compression of .xisf images. I currently have it set to 6.

I have a stack of 32 images, which are read into the local disk cache. When I try to integrate these (Image Integration), I get the following error message;

Code: [Select]
Integration of 31 images:
Pixel combination ......... average
Output normalization ...... additive + scaling
Weighting mode ............ noise evaluation
Scale estimator ........... iterative k-sigma / BWMV
Range rejection ........... range_low=0.000000
Pixel rejection ........... Winsorized sigma clipping
Rejection normalization ... scale + zero offset
Rejection clippings ....... low=yes high=yes
Rejection parameters ...... sigma_low=4.000 sigma_high=2.000

* Using 665 concurrent pixel stack(s) = 1022.98 MB

* Integrating channel 1 of 3:
Integrating pixel rows:     0 ->   976:   0%
*** Error: Cannot perform sequential/random disk read operations on a compressed data block.
<* failed *>

I have tried purging the file cache (manually deleting all files in the cache disks, as well as unselecting 'use file cache' as well, but I still get this error message whenever I try to integrate these (compressed .xisf) files.

My machine is an Intel i950, 24Gbyte RAM, 3xRaptor local cache, Linux 12.04, PI 01.08.03.1123.

Any suggestions?
« Last Edit: 2015 February 12 01:08:31 by slang »
--
Mounts: Orion Atlas 10 eq-g, Explore Scientific G11-PMC8
Scopes: GSO RC8, Astrophysics CCDT67, ES FCD100-80, TSFLAT2
Guiding: ST80/QHY OAG/QHY5L-II-M
Cameras: Canon EOS 450D (IR Mod), QHY8L, QHY163m/QHYFW2-US/Astronomik LRGBHaSiiOii

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7018
    • View Profile
    • http://pixinsight.com/
Re: Problem integrating images that are compressed xisf
« Reply #1 on: 2015 February 12 01:31:56 »
Quote
*** Error: Cannot perform sequential/random disk read operations on a compressed data block.

Unfortunately, the ImageIntegration tool has no way to know in advance that compressed XISF files cannot be used for incremental file read operations (as happens, for example, with DSLR raw or JPEG files). This is not actually a bug, but a limitation of the current platform that will be overcome in the next version of PixInsight. For now, you need uncompressed XISF images for ImageIntegration.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline slang

  • Member
  • *
  • Posts: 60
    • View Profile
Re: Problem integrating images that are compressed xisf
« Reply #2 on: 2015 February 12 01:38:51 »
Thanx for the prompt response, greatly appreciated.

Look forward to the next version - I had wondered if it was something like that. Guess it's time to load in, and re-save all those files ;-)

Cheers -
--
Mounts: Orion Atlas 10 eq-g, Explore Scientific G11-PMC8
Scopes: GSO RC8, Astrophysics CCDT67, ES FCD100-80, TSFLAT2
Guiding: ST80/QHY OAG/QHY5L-II-M
Cameras: Canon EOS 450D (IR Mod), QHY8L, QHY163m/QHYFW2-US/Astronomik LRGBHaSiiOii

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7018
    • View Profile
    • http://pixinsight.com/
Re: Problem integrating images that are compressed xisf
« Reply #3 on: 2015 February 12 02:01:43 »
You can use the BatchFormatConversion script. In the output format hints field enter "no-compress-data" (without quotes). Enter ".xisf" in output extension. Select your files, select an output directory, and click OK. This will save all your files without compression, irrespective of your current XISF preferences.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline slang

  • Member
  • *
  • Posts: 60
    • View Profile
Re: Problem integrating images that are compressed xisf
« Reply #4 on: 2015 February 12 02:21:59 »
Awesome, thanx.
--
Mounts: Orion Atlas 10 eq-g, Explore Scientific G11-PMC8
Scopes: GSO RC8, Astrophysics CCDT67, ES FCD100-80, TSFLAT2
Guiding: ST80/QHY OAG/QHY5L-II-M
Cameras: Canon EOS 450D (IR Mod), QHY8L, QHY163m/QHYFW2-US/Astronomik LRGBHaSiiOii