Incompatiabilty FIT files between Nebulosity and PixInsight?

[topboxman]

I capture images with Nebulosity and process them with PixInsight. I always save in 16 or 32 bits FIT format with PixInsight. But when I use Nebulosity to open PixInsight processed FIT files, I get an error message saying "Unsupported type or read error loading FITS file". If PixInsight can read Nebulosity generated FIT files (16 or 32 bits) but not vice versa, is PixInsight using their own proprietary FIT file formats that Nebulosity can't understand?

Thanks,
Peter

[Nocturnal]

Hi Peter,

I'm pretty sure Craig has a PI license. It might be worth sending him an email to ask. I've found that posting in the stark labs yahoo group is pretty hopeless if you have a true Nebulosity issue. If you attach a tiny FITS file that shows the behavior it might help him debug the issue. Near as I can tell both apps use the cfitsio library so I'm surprised there is this compatibility issue.

Did you use 32b integer or float?

[Philip de Louraille]

I believe Nebulosity expects to see a BZERO and BSCALE in the FITS headers.
To avoid problems between Neb and PI, save your files as *unsigned* FITS.
I think that by default, PI writes signed integer FITS images.

I have had that conversation between Craig and Juan in 2009. The consensus is that both programs are supporting the FITS standard but the standard, itself, is loose. This is because the FITS standard doesn't support unsigned integer data natively.

[Nocturnal]

Yes, it's probably something like that. In any event, I've never had the urge to load a PI file in Nebulosity. I use it only to capture

[topboxman]

I tried saving as unsigned FITS and Nebulosity still can't open it.

For example, after stacking the darks with PI, I wanted to create Bad Pixel Map from the stacked dark with Nebulosity but Nebulosity couldn't open the file.

Peter

[Philip de Louraille]

16-bit unsigned. I just played with Neb 2.5 (Haven't upgraded to 3.x yet) and if you save the files as 16-bit unsigned, Nebulosity is happy.

[topboxman]

Already tried that.

I always calibrate and deBayer with Nebulosity and save calibrated and deBayered files as 32 bit float. From then on it's PixInsight starting with Star Alignment/Integration all the way to post processing. All files always saved in 32 bit float by PixInsight. If I save 32 bit float as 16 bit unsigned integer with PixInsight, Nebulosity is still not happy.

I've haven't tried saving Nebulosity files in 16 bit unsigned format before passing to PixInsight and then back to Nebulosity. Maybe it's the conversion from 32 bit float to 16 bit unsigned integers by PixInsight may be the source of the problem. I am at work right now so I can't test it yet.

This is not really important. Sometimes I want to take a quick look at some FIT images and if PixInsight isn't currently running, I start Nebulosity because it's much quicker to start Nebulosity than PixInisight.

Peter

[Nocturnal]

Hi Peter,

it should work of course and hopefully there is a format that's usable for your workflow. When you get your faster PC you won't care about PI startup times anymore. Actually I've always found it to start very quickly even on my old 1.6G laptop. It's possible that the auto update feature takes most of the startup time. I have found start up times to be quite variable. Sometimes it just pops right into view, other times it takes 10-20 seconds. I have disabled auto update but don't know yet if that makes a difference.

In any event you'll probably start using PI for just about everything really soon. Its debayer routine is much better. Heck I wrote it Seriously, the version I wrote was faster because it's multi threaded. Even on your HT P4 you should see close to a 2x improvement. The bilinear interpolation I implemented was primitive but effective. VNG was added recently by Zbynek.

[topboxman]

Hi Sander,

When I first started PixInsight almost a year ago I had trouble deBayering and was not outputting correct colors for all Bayer matrix settings. Just last week, I decided to give another try and it's working. The Bayer Matrix for my SXVR-M25C camera was the very last setting (I can't remember which one since I'm at work). So, I will do all processing with PI including calibration. I noticed three different deBayer methods. Which is is the best? Does that depend on the captured images?

Great job on writing deBayering code.

Thanks,
Peter

[Nocturnal]

Hi Peter,

it's unfortunate we didn't discuss those debayer issues back then because really there is no magic. There are only 4 ways to debayer an image and all 4 are supported. It doesn't even depend on the camera or sensor even though my QHY8 has the same sensor as your M25C. As you can see the debayer process does not include color weighting so the colors may look 'odd' but would still be correct. Color correction happens early in post processing.

Technically VNG is the most complicated. Like bilinear it creates an image that is the same size as the original. Superpixel creates a half sized image. I calibrate an stack with DeepSkyStacker so I haven't really investigated the differences between VNG and Bilinear. I recommend you try both and decide which one you like best. I wrote the debayer module because I wanted to be able to examine individual subs in PI rather than Nebulosity or Fitswork

[topboxman]

Hi Sander,

Yeah, I wish I had discussed with you earlier regarding to deBayering issues.

I always use Nebulosity's Bad Pixel Mapping instead of dark subtraction thanks to very low noise from our cameras' CCD sensor. Does PixInsight have anything like BPM? If so, how do you do it? I think it's called Defect Map but I have been unsuccessful.

Thanks,
Peter

[Nocturnal]

Hi Peter,

I wrote my own bad pixel and bad column fixer a few years ago because Neb's implementation was rather poor. Not surprisingly I still use my own utility It's command line driven so if that's not your cup of tea then it won't help. It is possible that there is a process or script in PI that works as well as what I wrote but I don't know. Someone else may comment.

http://www.tungstentech.com/Software/FixFITS/tabid/78/Default.aspx

I have a more recent version I can email you. I added triple column fixing. When it was really cold recently I had 3 bad columns next to each other so I had to figure out a way to patch those. You probably have no need for that. Since I have an OSC camera the tool defaults to that behavior.

Definitely do not use dark subtraction with your camera.

[topboxman]

Hi Sander,

I am a firmware/software/embedded engineer so I use command line all the time.

I find Nebulosity's BPM pretty decent. It seems to remove just about all bad pixels because I don't see bad color pixels after deBayering. With dark subtraction, it never removes all bad pixels and I see several different bad color pixels and they were driving me nuts.

Thanks for your tool. So far I don't have bad columns. Sorry that you camera is beginning to get them. Are you looking for another camera? I have been thinking about getting mono camera with filters for a long time but I can't find one with similar size CCD sensor as M25C. I really like having large CCD and works well with my EdgeHD. The closest mono camera I found that has reasonably sized CCD is KAI-4022 but it's a little pricey. I like QSI 540wsg.

Peter

[Nocturnal]

Hi Peter,

the problem with Neb's tool is that it's not automatic. I'm the lazy sort. Would rather spend a few hours writing code than 5 minutes doing tedious work With my tool all you do is point it at a dark file and the rest is taken care of. You can adjust the selection criteria but it's generally not needed if the dark is relevant (ie. within 5 minutes of the actual duration and within 20C of the actual temperature).

My '8 has had bad columns for years. My tool defaults to 4 of them. Here's an image from almost 4 years ago that shows how subtle the problem is. That's why I needed to write a tool to detect the columns in raw images. I was spending too much time pixel peeping.

http://gallery.tungstentech.com/main.php?g2_itemId=763

I've also considered going mono with the increased LP in my town. I'm considering an QHY9M with filter wheel. It's a little smaller than the 453 sensor but not by much. I wish someone ripped the bayer matrix of a 453 and stuck it back into the camera. The new mono SXV with smallish Sony chip looks good. Not sure I could go back to a sensor that size though.