Photometric Color Calibration problem with large image

[samlising]

Hi.

I have been trying to test Photometric Color Calibration with extremely large images (33580 x 25360) and I always get the same error:

*** Error: Failure to process image file 'D:/Users/Sam/AppData/Local/Temp/PCC_R_DYY0RBYCVD0N.xisf': Error: Image.getSamples(): error setting array element

However, I've been able to get Photometric Color Calibration to work with the same images by scaling them down to a smaller size.

The results of PCC are excellent with my scaled down images and it would be great if they could be used with larger images.

Here's the complete log:

Code: [Select]

PhotometricColorCalibration: Processing view: rosette_RGB300_master_linear_mure0_0_x10
Writing swap files...
1315.971 MiB/s

run -x -p="phot_apertureShape,1" -p="phot_minimumAperture,30" -p="phot_apertureSteps,1" -p="phot_apertureStepSize,1" -p="phot_bkgWindowMode,1" -p="phot_bkgModel,1" -p="phot_backgroundSigmaLow,5" -p="phot_backgroundSigmaHigh,2.7" -p="phot_bkgAperture1,45" -p="phot_bkgAperture2,60" -p="phot_bkgMMTlayers,8" -p="phot_minSNR,4" -p="phot_showFoundStars,false" -p="phot_maxMagnitude,15" -p="phot_defaultFocal,847.32" -p="phot_gain,1" -p="phot_saturationThreshold,0.95" -p="phot_manualFilter,true" -p="phot_filter,Johnson V" -p="phot_filterKeyword,FILTER" -p="phot_margin,15" -p="phot_catalogName,APASS" -p="phot_catalogFilter,Vmag" -p="phot_vizierServer,http://cdsarc.u-strasbg.fr/" -p="phot_useActive,false" -p="phot_extractMode,0" -p="phot_manualObjects,[]" -p="phot_outputDir,D:/Users/Sam/AppData/Local/Temp" -p="phot_outputPrefix,PCC_" -p="phot_generateFileTable,true" -p="phot_generateFluxTable,true" -p="phot_generatePSFFluxTable,true" -p="phot_generateBackgTable,false" -p="phot_generateSNRTable,false" -p="phot_generateFlagTable,false" -p="phot_saveDiagnostic,false" -p="phot_autoSolve,false" -p="phot_forceSolve,false" -p="phot_solverUseImageMetadata,false" -p="phot_saveSolve,false" -p="phot_solveSuffix,_WCS" -p="phot_generateErrorLog,false" -p="phot_showBackgroundModel,false" -p="phot_files,D:/Users/Sam/AppData/Local/Temp/PCC_R_DYY0RBYCVD0N.xisf|D:/Users/Sam/AppData/Local/Temp/PCC_G_TXEQDJUCQGLK.xisf|D:/Users/Sam/AppData/Local/Temp/PCC_B_XPCRYED6K8D0.xisf" -p="non_interactive,true" -p="isGlobalTarget,true" -p="isViewTarget,false" "C:/Program Files/PixInsight/src/scripts/AdP/AperturePhotometry.js"

Processing script file: C:/Program Files/PixInsight/src/scripts/AdP/AperturePhotometry.js

*******************************
Processing image D:/Users/Sam/AppData/Local/Temp/PCC_R_DYY0RBYCVD0N.xisf

Reading image:
D:/Users/Sam/AppData/Local/Temp/PCC_R_DYY0RBYCVD0N.xisf
Loading image: w=33580 h=25360 n=1 Gray UInt16
20 FITS keyword(s) extracted.
Reference image: PCC_R_DYY0RBYCVD0N

Downloading Vizier data:
http://cdsarc.u-strasbg.fr/viz-bin/asu-tsv?-source=II/336/apass9&-c=98.001354 5.009906&-c.r=1.130979&-c.u=deg&-out.form=|&-out.add=_RAJ,_DEJ&-out=B-V&-out=Vmag&-out=Bmag&-out=g'mag&-out=r'mag&-out=i'mag&Vmag=0..15
250240 bytes transferred in 1.83 s @ 133.68 KiB/s
Removed 78 duplicated objects
fov:1.130979 actual:1.130463
Catalog APASS size: 1131 objects inside the image

Catalog stars: 3814
1 view(s)
1129 star(s)
365 PSF fittings
Found 26 stars with invalid position
Reference stars: 365
Validating reference stars
Stars with problems: 26
*** Error: Failure to process image file 'D:/Users/Sam/AppData/Local/Temp/PCC_R_DYY0RBYCVD0N.xisf': Error: Image.getSamples(): error setting array element

*******************************
Processing image D:/Users/Sam/AppData/Local/Temp/PCC_G_TXEQDJUCQGLK.xisf

Reading image:
D:/Users/Sam/AppData/Local/Temp/PCC_G_TXEQDJUCQGLK.xisf
Loading image: w=33580 h=25360 n=1 Gray UInt16
20 FITS keyword(s) extracted.
Reference image: PCC_G_TXEQDJUCQGLK
1 view(s)
1129 star(s)
429 PSF fittings
Found 30 stars with invalid position
Reference stars: 429
Validating reference stars
Stars with problems: 30
*** Error: Failure to process image file 'D:/Users/Sam/AppData/Local/Temp/PCC_G_TXEQDJUCQGLK.xisf': Error: Image.getSamples(): error setting array element

*******************************
Processing image D:/Users/Sam/AppData/Local/Temp/PCC_B_XPCRYED6K8D0.xisf

Reading image:
D:/Users/Sam/AppData/Local/Temp/PCC_B_XPCRYED6K8D0.xisf
Loading image: w=33580 h=25360 n=1 Gray UInt16
20 FITS keyword(s) extracted.
Reference image: PCC_B_XPCRYED6K8D0
1 view(s)
1129 star(s)
373 PSF fittings
Found 7 stars with invalid position
Reference stars: 373
Validating reference stars
Stars with problems: 7
*** Error: Failure to process image file 'D:/Users/Sam/AppData/Local/Temp/PCC_B_XPCRYED6K8D0.xisf': Error: Image.getSamples(): error setting array element

*******************************
Photometry process finished:
   0 of 3 images processed successfully.
   3 images with errors.
   3 errors and warnings.

*** Error: Failure to calculate photometry: rosette_RGB300_master_linear_mure0_0_x10
Reading swap files...
2527.951 MiB/s
<* failed *>

[Juan Conejero]

Hi,

This is not a bug, but a limitation of the JavaScript engine (the PCC tool is a hybrid C++/JavaScript implementation). The problem is that JavaScript Array objects cannot address more than 4 GiB of array data. I think we can modify PCC's supporting scripts to circumvent this limitation, but for now, you can easily apply PCC's calibration factors to the large image with PixelMath. Do the following:

- Apply PCC to a reduced version of the image, as you already have done. Obviously, you have to redefine image scale parameters accordingly.

- Write down the values of the following parameters shown on the console at the end of PCC: W_R, W_G, W_B. These are the computed scaling factors for the red, green and blue channels. One of the factors should be 1.0, so you really need just two numbers.

- Open the large image and the PixelMath tool. For each channel (uncheck the use a single... option), enter the following expression: $T*W, where W is the value of the corresponding W_x scaling factor. See the attached screenshot for a real example, using symbols with values directly copied from the console.

- Apply the PixelMath process to the large image.

- Open the BackgroundNeutralization tool and use it to equalize the background, as usual.

You'll get the same results with this procedure and applying PCC directly to the image. Let me know hot it goes.

[samlising]

Hi Juan.

Thanks for your reply.

That worked perfectly! The large scale and small scale images are exactly the same now.

Thanks.
Sam