DBE Error: "Variant::ToDouble(): Invalid conversion" with Gigapixel-Image

[SilvioM]

Hi,

when I use Dynamic Background Extraction on a really large image, DBE throws a messagebox:
"Variant::ToDouble(): Invalid conversion from invalid type" while computing view property: MAD.

Details:
- PixInsight Core 01.08.03.1123 Ripley (x64)
- Windows 7 64bit
- Image: FITS: 16-bit integers, 3 channel(s), 64433x38460 pixels
- reproduction:
   - open the image
   - open Dynamic Background Extraction, leave all options at default
   - under "Sample Generation", klick on "Generate"
   --> PI first reserves 90Gigabytes of RAM, computes view property Median, then tries to compute view property MAD, then throws the error:
   "Variant::ToDouble(): Invalid conversion from invalid type"

This seems to be some kind of type overflow. It is fairly easy reproducable with that image. The image is about 14.5Gigabytes in size, so it would be a real challenge to upload it for reproduction.

If I crop the image down to 54140x19062 pixels, DBE works fine.

I know I'm really pushing the envelope here, but PixInsight has some beautiful tools, that HAVE to be used!

Hope you can help.


Kind Regards,

Silvio.

[Juan Conejero]

Hi Silvio,

How much RAM do you have in your machine? I wouldn't try this with less than 48 GB. The problem with MAD is that it has to compute two medians: first the median of the whole image, then the median of the absolute differences from the image median. The first step requires a duplicate of the image, since the algorithm is destructive (we use a selection algorithm to compute the median, which alters the order of data elements). The second step reuses the same duplicate, but this is a big amount of memory.

I'll investigate the origin of this error. Anyway, I have been thinking on a faster and less memory consuming implementation of the STF AutoStretch feature, basically one that does not require medians, but this is not easy.

[SilvioM]

Hi Juan,
thanks for looking into this.

I'm trying this on a i7 with 32GB of RAM. The Swapdrive is a ssd. Sure, it's slow, but that would be ok - if it would work in the end.
Hope you can find a solution for the error.

What has the STF AutoStretch feature todo with DBE? Of course, STF is slow too, but it's no real neccessary tool for my workflow. DBE is.

Kind Regards,

Silvio.