.CR2 RAW files show 2 or 3 rows that are not from sensor data

[georg.viehoever]

Hi,

when I load .CR2 RAW images (from Canon EOS40D) into PI, they always show 2 or 3 lines of data at the bottom that is not from sensor data. This is especially visible in dark images, and independent of the RAW debayer mode (all 4 shown in screenshot). I don't know if this is a dcraw or a PI problem. But those rows dont exist when I load into IrfanView (also uses dcraw http://www.cybercom.net/~dcoffin/dcraw/) or DigitalPhotoProfessional (Canon proprietary).

Georg

PS: Resolution accordining to PI: 3908x2602, according to DPP: 3888x2592, which is also what you can read in other sources, e.g. http://en.wikipedia.org/wiki/Canon_EOS_40D.

[pauljv]

Try capturing your significant data closer to the centre of the frame, it won't be so noticeable then.

Regards Paul

[georg.viehoever]

Hi Paul,

Try capturing your significant data closer to the centre of the frame, it won't be so noticeable then.
...

the only noticed it because I am currently "debugging" my calibration results. Obviously, such rows distort the statistics of bias/dark/flat/light. And heaven knows what ImageCalibration is doing with this.

A workaround is not difficult, just crop the image. But it is an additional step in the calibration flow consuming time and disk space, so I would prefer if PI did read .CR2 correctly.

Georg

[NKV]

Georg,
Iris and MaximDL doing same 3908x2602. But only PixInsight can handle Overscan area during calibration.
Unfortunately, I never seen anything definitive about Canon CMOS sensors.

[georg.viehoever]

Hi Nikolay,

...
Iris and MaximDL doing same 3908x2602. But only PixInsight can handle Overscan area during calibration.
...

Overscan areas are rows/columns of a sensor that are kept dark. So they should show values similar to those of pixels in dark images. But they dont. They are clearly visible as bright lines in my dark frames too. So something else is going wrong here?

If you have MaximDL: Could you compare what you see in the lower couple of rows in PI and MaximDL after loading a CR2 file?

Georg

[NKV]

Could you compare what you see in the lower couple of rows in PI and MaximDL after loading a CR2 file?

I see bright pixels in PixInsight/MaximDL/Iris and Google Picasa.

What you think about Lleft/Top/Right additional row of pixels? It's usable or also only good for garbage collector?
I ask because AIP4WIN read 3944x2622 pixels !!! Look attachment. It's corners (thank's aberration-spotter.js) from Canon40D.cr2 decoded in AIP4Win.

Geord, did you see overscan area now?

I will happy if PI will decode 3944x2622. 3908x2602 not enough for me.

[georg.viehoever]

Hi NKV,

I can see those bright lines in your screenshot both above and below the actual data. In the meantime, I have also been able to produce them Inrfan 4.23, see screenshot below. In Irfan, you have to deselect "Options/Properties+Settings/PlugIns/Load Half Size Image", which by default is on.... Given that Irfan, MaximDL and PI all see those bright lines, it appears that this is not a PI issue, more likely something in dcraw. Lucky Juan...

What I definitely dont see in PI or Irfan are those dark areas at the borders of you screenshot. If this is real sensor data, only without light, then these would be overscan areas that would be interesting for image calibration. Does anyone have information on that? NKV: Does it look like real sensor data to you (i.e. with noise, with a few hot pixels, with the same "color" as the main sensor area in dark frames)?

Georg

[NKV]

Does it look like real sensor data to you (i.e. with noise, with a few hot pixels, with the same "color" as the main sensor area in dark frames)?

Yes, it's look like real sensor data. More information about .CR2 at Understanding What is stored in a Canon RAW .CR2 file, How and Why

According ExifTool:

Code: [Select]

ExifTool Version Number         : 8.34
File Name                       : _MG_9674.CR2
Camera Model Name               : Canon EOS 40D

Sensor Width                    : 3944
Sensor Height                   : 2622

Sensor Left Border              : 40
Sensor Top Border               : 23
Sensor Right Border             : 3927
Sensor Bottom Border            : 2614

Image Size                      : 3888x2592
And look at AIP4Win PrintsScreen:

[georg.viehoever]

Does it look like real sensor data to you (i.e. with noise, with a few hot pixels, with the same "color" as the main sensor area in dark frames)?

Yes, it's look like real sensor data. More information about .CR2 at Understanding What is stored in a Canon RAW .CR2 file, ...

Wow, this document provides a wealth of information. Apparently, for most Canons the sensor is a bit larger than the effective image. It would be useful to have a detailed look at these sensor areas and see if they can be used as an overscan area. But unfortunately, I seem to be unable to read them with PI.61.

So in effect this bug report now has an two points:
1. Get rid of the 2 lines that are not image data
2. Optionally, allow to read in the whole sensor area (which your screenshots suggest is possible with AIP4WIN).

Georg

[NKV]

1. Get rid of the 2 lines that are not image data
2. Optionally, allow to read in the whole sensor area (which your screenshots suggest is possible with AIP4WIN).

As fast solution for read 3944x2622 pixels:
Look in dcraw.c http://www.cybercom.net/~dcoffin/dcraw/

Code: [Select]

 } else if (is_canon && raw_width == 3944) {
    height = 2602;
    width  = 3908;
    top_margin  = 18;
    left_margin = 30;Replace:

Code: [Select]

 } else if (is_canon && raw_width == 3944) {
    height = 2622;
    width  = 3944;
    top_margin  = 0;
    left_margin = 0;
 8)

[NKV]

One more way to access to overscan area:
Download from http://www.libraw.org/download
& use unprocessed_raw executable (which in /bin directory of sources)
See Usage Examples

Code: [Select]

E:\2\unprocessed_raw.exe -N E:\2\_MG_7675.CR2
Processing file E:\2\_MG_7675.CR2
Image size: 3908x2602
Raw size: 3944x2622
Margins: top=18, left=30, right=6, bottom=2
Unpacked....
Writing file E:\2\_MG_7675.CR2.tiff