My first PI experience -- I'm a little bummed.

[blave549]

I just enabled the 45 day evaluation period, and after looking at Warren Keller's free PI videos, I thought I'd give the Batch Preprocessing Script a try with some recently acquired lights/darks/flats of M31 taken with my Canon 450D (using Nebulosity for camera control; saving as "raw" FITS files). I left pretty much everything at the default settings, imported my subs, and pressed the button. After 10 or 15 minutes (i5-based Win7 laptop with 4GB memory) I looked at the resulting stacked master, and there's no data in it (histogram shows a very fine line on the far left side when I look at the FITS file in Nebulosity).

As I'm typing this in, I just looked at several of the calibrated/registered/master darks/flats/lights that PI generated -- all of them look the same -- totally black and no data shown in the histogram.

My previous experience is almost entirely with Nebulosity, so I have at least a passing familiarity with the general pre-processing workflow. Other than not getting anything useful at the end, I was blown away with how much processing time is required by PI as compared to Neb. Now, of course I realize this is completely different toolset and philosophy, but the upshot is that after less than 5 minutes in Neb with the same data, I get a calibrated, stacked, useful result.

This is not intended as a slam of PI, just an honest "first light" experience with this tool/environment. It's entirely possible/probable that I made some dorky noob error... Any thoughts as to what that might be would be welcomed  .

Finally: one thing I noticed as I watched the logged script activity go by is that it always reported the data as "1 channel" -- I was expecting to see 3 channels.

I have attached the log information from running the script.


cheers,

Dave B.

[Carlos Milovic]

First of all, open some of the raw fits files generated by nebulosity in PI, to make sure tht they are readed propperly. There has being some reports of incompatibilities between those files and PI. Take a look here in the forum (use the search engine).
Then, I strongly suggest to use bias frames. Calibrating a flat with a 602s dark is not the way to go.
Also, you may check in the script the "CFA" option. DSLRs use a bayer matrix to encode color information, and this is a grayscale image. See "debayerization" or "demosaizing".
Finally, take a look at the fits format preferences, in the format explorer. I would use the top left corner as the coordinae origin instead of the default value (also, check that option on the script).

Anyway, i would not use fits format to store this raw data... What is wrong with cr2 files? I use APT for capture and save everything as cr2, and never had a problem.

[chris_todd]

OK, possibly silly question:
Did you stretch the master (using the ScreenTransferFunction or HistogramTransformation tools)?

Otherwise, Carlos' suggestion of checking your DSLR_RAW format preferences, and making sure you have CFA checked in the BPP, are both critical options. 

Chris

[blave549]

thanks for the replies guys. I didn't have CFA enabled, but I'm still getting "dead" histograms for everything after checking that box. Opening the pre-PI-touched flats, for instance, looks OK but the pre-processed ones are black black black (I know about the transform function).

I'm retreating to a point of starting with just "lights" and trying to get some stacked and such. I'm also working through Harry's excellent videos today and have already learned a lot about the, er, unconventional user interface.

I'll keep you posted.  But again the intent of my post was just to illustrate someone from the unwashed masses giving PI a go without too much research (which I already knew would be necessary; I just wanted to try something that seemed pretty straightforward).

cheers,

Dave.

[chris_todd]

Don't give up on PI based on your initial experiences.  I've been using it about a year, and I still feel like a newbie, only because it is so powerful and there are so many different tools.  I learn something new every time I come on this forum.  PI seems to be one of those pieces of software that rewards you for diligence and effort.  You *will* achieve superior results if you're willing to put in the effort to learn, and there are many excellent and continuously improving educational resources out there (the product docs, Harry's tutorials, Warren Keller's, RAB's unofficial reference guide, and this forum are several I've used).

Check out FAQ #1.11 (http://www.pixinsight.com/faq/index.html) for an idea of the PI philosophy; I think that will give you a better idea of where PI is coming from, which (for me, at least), will help you understand why it is the way it is. 

Be sure to come back here with your questions, this is a friendly and helpful group.

Links to the pages I mention above:
http://www.harrysastroshed.com/pixuser/pixuserhome.html
http://www.ip4ap.com/pixinsight.htm
http://blog.deepskycolors.com/PixInsight/

[Juan Conejero]

These lines tell you where the problem is:

Calibrating target frame 1 of 10
Loading target frame:
C:/Users/Dave/Documents/astro 2013/2013-02-10 m31/Series1_001.fit
Reading FITS: 32-bit floating point, 1 channel(s), 4290x2856 pixels: done
Rescaling sample values to [0.00000000e+000,1.00000000e+000]: done

Since the data have been rescaled from an unknown input range, they have lost their physical meaning and we can't treat them as raw data anymore.

In general, floating point images generated by other applications (what we call alien images) cannot be calibrated in PixInsight. This has been discussed many times on this forum. The main problem is: The FITS standard does not provide a way to know the range of numeric values to which floating point data are referred. In other words, FITS does not provide a way to know which values correspond to the black and white points in the image. Without knowing these points, we simply cannot interpret the data.

Each application uses its own floating point range. Actually, this is not a problem for PixInsight, since our tools let you specify a floating point input range. The problem is that, in general, we don't know the range of alien floating point images because the applications don't publish them (please correct me if I'm wrong). Some applications also seem to use different ranges for each image, in apparently arbitrary ways. PixInsight always writes floating point images in the normalized [0,1] range, where 0=black and 1=white.

Storing raw CCD or CMOS data in a floating point format is an error, both technically and conceptually. Digital cameras generate 16-bit unsigned integer data, so this is the only valid format that can be used to store raw data during acquisition. Your data won't be better or 'more precise' if you store them as 32-bit floating point numbers; they will just occupy twice the required space on disk and will generate incompatibility problems, that's all.

So if you write your raw data as FITS files in 16-bit unsigned integer format, there should be no problems to preprocess them in PixInsight. On a side note, I see that you don't use bias frames. Without bias frames you cannot calibrate the data correctly (i.e., dark scaling won't work at all, and flat fielding will be inaccurate).

I hope this helps.

[Josh Lake]

I have no new info to add to Juan's post, Dave, but I'll echo the sentiment from higher in the thread -- stick with it, it IS worth it. I was a die-hard Nebulosity user before finding PI, and I went through many of the same difficulties early on (with manual pre-processing at the time). Once you get over this FITS file hurdle and get used to the interface, I think you'll find that the power of its tools is liberating. I think you'll also find that the UI really starts making a lot of sense once you know what's what.