Noob with a problem

[drmorbius]

Hi everyone,

I'm new to AP and PixInsight, so I'm a bit overwhelmed at the moment  .  I'm following some of the great tutorials, but the results I get are... different.  Searches of the InterWeb haven't turned up anything useful so I thought I'd ask the forum.

I have taken 4 widefield photos using a Canon 450D, so I have a number of CR2 files that I have converted from RAW to FIT.  No darks or flats out this stage...

When I view a single FIT (NB - RAW is the same) and use ScreenTransferFunction to look at it, I get a greenish picture with a few faint stars in the background (see attached).  An auto-stretch produces nothing but a uniform pale green. When I attempt to stack the photos using ImageIntegration, the end result is a sickly green, mottled photo.

Does anyone know what I'm doing wrong   

Thanks -- randall

[Philip de Louraille]

I am not a Canon expert but I am pretty sure you need to debayer (ColorSpaces | Debayer) your shots after you convert them from raws.
You may even be able to go straight from RAW to RGB and saving them as fits by going to the Debayer process first.

Details on debayering are here. http://www.stark-labs.com/craig/articles/assets/Debayering_API.pdf
Basically, it is an encoding process and you need to extract your RGB channels from that RAW.

[Cleon_Wells]

Randall
Check this link on DSLR conversion for image Calibration by Vicent.

http://www.astrofoto.es/Adler/Documentation/Calibration/Calibration.xpsm

I have a Canon T1i/Hap mod, I convert my CR2 files to raw fits with the Brightness set to 4.0 in the (Format Explore/DSLR_RAW) path, by double clicking the DSLR_RAW heading. This change gives me a fits file with a clip pixel level of 1.00 compared to 0.25 when you leave Brightness set to 1.0, Vicent pointed out that the T!i creates a 14bit file and the DSLR_RAW  conversion expects a 16bit file.
Cleon

[drmorbius]

Cheers guys.  I sort of understand the theory of what my problem is now, but I'm not sure what to do with the information. 

So I need a separate program to convert the RAWs before I can use them in PI?  If so, any recommendations for Mac OS?

Alternatively, I'm wondering if one or more of the tutorials I followed have stuffed up my settings somewhere.

Cleon, what settings do you use in the DSLR_RAW Preferences (i.e. - what boxes are ticked, etc)?

Thanks again

[Cleon_Wells]

Randall,  use the BatchFormatConversion Utility  to convert the 14bit CR2  files to the raw  16bit .fit files, (---at the top of the screen click on Script/Utilities/BatchFormatConversion--) then Add the CR2 files, Select an Output  Folder an change the extension from .jpg to .fit. I create folders for all the converted files like M78_fit, M78_Cal, M78_DeBayer_Cal, M78_DeBayer_Cal_Reg, yes I create all these folders to keep track of all the processes need to Integrate  all the subs.  Don't forget to change The DSLR_RAW Preferences before you open up the BatchFormatConversion Utility, (-- uncheck the Use Camera White Point, check the Create Raw Bayer Image, check No Black Point, I have Bilinear  checked for Interpolation--)
Cleon

[drmorbius]

Thanks Cleon, I've tried your settings but the result is exactly the same... sigh...

Just to make sure, I've attached a snapshot of my preference settings... are they right?

Not sure where to go from here, but I appreciate your help.

[Hans Pleijsier]

Check out what the user Astropixel compiled on this subject in this forum.

For instance, you could read:

///

It is not necessary, or desirable, to convert RAW files to .fit format before processing - doing so produces an inferior result. A more complete explanation is provided here.

http://pixinsight.com/forum/index.php?topic=2118.msg13737#msg13737

The DSLR_RAW workflow calibrates RAW files with ImageCalibration against master calibration files (bias, dark and flat). The calibrated files are debayered with the BatchDebayer script, then registered with StarAlignment and integrated with ImageIntegration, to produce a debayered .fit file ready for further processing.

///

[zerro1]

and how does one perform image integration, with a RAW file

[Cleon_Wells]

Randall, yes those setting are correct. To help you get to the bottom of your problem I would like you follow me in some basic PI steps,((sort of like the blind leading the blind..LOL))  open up the HistogramTransformation process and the ScreenTransformation process and one of your CR2 files also Debayer this CR2 file then use BatchFormatConversion to get a .fit file  and Debayer the .fit file of this same CR2 file. Now Calibrate the CR2 file and then the .fit file Debayer these files. Now you have 8 files to examine with the HistogramTransformation window, select a file in the Histogram display, file selector box and then adjust the STF as you look at the screen display, then drag the Triangle/New Instance icon in the lower left part of the STF window to the bottom of the HT window and see how the histogram changes. You should be able to see the image with the correct RGB balance.
A point on Hans CR2 file Calibration point, when I Cal a CR2 file I get an error window with --FileFormat implementation: Extra Keywords() must be reimplemented in descendant class-- error, it does seem to Calibrate the file.
Philip gave use a good link on the bayer matrix..
Zerro1 is correct, you can't integrate a CR2 file.

  Cleon

[zerro1]

so how do you create your "calibration" masters, without converting RAW to fit?

[Hans Pleijsier]

Ok ... I assumed you already had the masterdark and -flatfiles.

Try this:

1.
Produce the masterdark and -flat with imageintegration and the tutorial of Vicent Peris.
As you pointed out, this cannot be done with CR2 so you have to use batchformatconversion-script from cr2 to FITS.
Before you do this you have to check the DSLR-RAW-settings: the setting that were mentioned earlier in this post, should work.

2.
Now you can callibrate your RAW CR2 Lights with these masterdark and -flatFITS.  You should use it on the CR2-Lights-files that came straight from your camera.

3.
Now you want to debayer with the BatchDebayer-script.

4.
Proceed with ImageRegistration/Alignment

5.
Finalize with ImageIntegration

6.
Now it is time for some further image-processing: STF/DynCrop/DBE/ColorCallibration/HistTrans/Noisereduction ... etc.
 

[drmorbius]

I really appreciate this guys...

Ok... image calibration is a new step in the process... but given I didn't take darks or flats on the night (doh!)... does that mean I won't be able to calibrate my images?

[Philip de Louraille]

Not really. You can make your own darks anytime: just make sure no light enters the CCD chamber when you take them. Take them at the same temperature and length of time than you took your light exposures. You can even take them with longer exposures and PI can downscale them for shorter exposures (but that requires at least one more checkbox to, erh, check!)

Same with the flats, as long as you use the exact same set-up, of course and you did not realign the optics or changed anything in the optical path you used to take your lights.

But, for simplicity, and sake of this first work flow for you, I'd skip that step for now. See what your light frames have. If they look good, then you can consider putting the work involved for calibration and get better results.

[drmorbius]

Thanks Phillip,

Skipping the calibration step, I tried BatchDebayer and it says it wants a monochrome image...

Sigh... this is starting to get a bit frustrating... but I do appreciate your efforts.

[Philip de Louraille]

Juan wrote this a while back. See start of ======
Looks like the ImageCalibration step is needed but you might be able to go through it by not checking Master Bias/Dark/Flat fields.
Don't really know... I never used a DSLR for taking astronomical data.

=====
There are two points that you must consider before starting to use DSLR raw images with PixInsight:
- You must calibrate your DSLR raw images with ImageCalibration directly. You don't have to convert them to the FITS format.
- Before using ImageCalibration, you must adjust the default decoding parameters of the DSLR_RAW module. This is the step-by-step procedure:
  1. Open the Format Explorer window.
  2. Double click on the DSLR_RAW item (left column).
  3. On the RAW Format Preferences dialog, select the following options:
- Color Adjustment: all three values equal to one.
- White balance: disable both options.
- Select 'Create RAW Bayer image'.
- Select 'No black point correction'.
- Click Ok.

Now PixInsight will load true raw data that can be calibrated with the ImageCalibration tool.

After ImageCalibration, you must de-Bayer the images. This can be done with the excellent BatchDeBayer script by Ken Pendlebury which invokes the also excellent Debayer tool written by Sander Pool. Once you have de-Bayered the images, you can register them with StarAlignment and integrate the registered images with ImageIntegration.