Debayering problem after format conversion

[bulrichl]

...when starting with PI some  month ago I had an "out of memory error" while stacking big RAW files
I got the recommendation to do the conversion:
.....ImageIntegration doesn't need to load all the images at the same time when the image format allows to read them by rows. In this case PI can process really big datasets with "normal" amounts of memory.
FITS and XISF allow this. The RAW formats don't. But I don't know TIFFs are read. This is why I suggested to change the format of the files.

Aha, I see. However this only applies to the generation of the Master calibration files when you have to apply an Image Integration to raw files, doesn't it? So I would only convert files that have to be integrated directly, i.e. bias and dark frames, whereas a conversion of light and flat frames would NOT be necessary.

Again:
so I let the input file format hints empty? or CFA or no-CFA?

With 'Pure Raw' set in DSLR_RAW or RAW the input file hints can be left empty.

---

OK, I guess that I have found a fundamental difference between the two cases:
1) Load a IIQ raw file and save as XISF file and
2) Use batch format conversion for the conversion of the IIQ raw file to a XISF file.

In case 1) the console output is:

Reading 1 image(s):
H:/CF001564.xisf
Loading image: w=11608 h=8708 n=1 Gray UInt16
8 image properties
9 FITS keyword(s) extracted.

in case 2) the console output is:

Reading 1 image(s):
H:/CF001564_1.xisf
Loading image: w=11608 h=8708 n=1 Gray UInt16
9 FITS keyword(s) extracted.

I.e. the image properties are missing! These are the image properties in case 1):

<Property id="Instrument:Camera:ISOSpeed" type="Int32" value="50"/>
<Property id="Instrument:Camera:Name" type="String">Phase One IQ3 100MP</Property>
<Property id="Instrument:ExposureTime" type="Float32" value="0.004"/>
<Property id="Instrument:Telescope:Aperture" type="Float32" value="0.01607143"/>
<Property id="Instrument:Telescope:FocalLength" type="Float32" value="0.045"/>
<Property id="Observation:Description" type="String">Lens Position: 28</Property>
<Property id="Observation:Time:Start" type="TimePoint"value="2017-08-06T21:07:44.000Z"/>
<Property id="PCL:CFASourcePattern" type="String">RGGB</Property>

the important one being the last property.

That's it. Indeed, you seem to have found a bug in the script Batch format conversion.

Bernd

[msmythers]

Peter

Just a question, did I read correctly that you are doing the batch conversion because your computer 'had' a low amount of ram but you have since increased the ram? If you increased the ram do you need to do the batch conversion now? Please excuse the question if I am missing something again.


Mike

[bulrichl]

No, I am not Peter, but - yes, this was the thread:

https://pixinsight.com/forum/index.php?topic=11122

Bernd

[Peter S]

thanks for all replies!

I started with 16 GB and installed additional 16 GB RAM = 32 GB.
Integration: I increased buffer to 1024 an stack size to 12500.
I remember that even with that amount of 32 GB RAM I had problems integrating IIQ RAWs (20 - 50 à 120 MB) so the batch conversion to XISF helped because of reading rows and not trying to load the big files.

> If you increased the ram do you need to do the batch conversion now?

I  think IIQ aren`t accepted in some of the processes - to be checked later.

 To make it weird: the camera support wrote me that the CFA pattern is RGBGRGBG.

Peter