Unable to perform Batchpreprocessing

[kwalker]

First off ,I've been a DSLR Guy for 5 years or so and PixInsight has always been the Platform to process my images.
Recently I purchased a TPO8RC and dressed it up with my first CCD...an STF 8300c OSC.
Learning this will take time and the first mistake was saving the data as uncompressed(sbig format) ,I later saved the files in ccdops as TIF
Getting to the point ,PI says my flats don't exist. OK  8) ,my darks don't deserve the interrogation I chose and claimed the 11 darks where less than 8.
After struggling awhile I used deep sky stacker for the first time in forever and that seemed to work.
I included screen shots.
Left image PixInsight ,Right image deep sky stacker.
So If I'm having a dumb guy moment go ahead and let me have it (I can take it)
If not then I think the DSLR will be my camera and CCD in the trash 
For reference I used the same basic settings I would use for my 60da

*** EDITED BY FORUM ADMINISTRATOR ***
Image deleted as it does not comply with forum's image posting rules.

This is suppose to be Running Man
NGC 1977 First Light

*** EDITED BY FORUM ADMINISTRATOR ***
Image deleted as it does not comply with forum's image posting rules.

[Paul Deeter]

I'm no expert, but after some trail and error I was able to get BPP to work for me.  Have you read this: http://www.lightvortexastronomy.com/2015/07/tutorial-pixinsight-pre-processing.html ?  Make sure to check CFA images for OSC. 

[kwalker]

I applied the same principle as if it where my Canon 60da
It has to be something to do with ccdops file format during acquisition on my run last night fighting the cold and moon.
hopefully I can get out for a few hours tomorrow night to try and set things straight.
I didn't let the camera do its dark frame subtraction and treated the ccd like a DSLR for darks ,flats ,lights ,and bias.
I'm missing something.

[pfile]

color looks off - debayer settings could be wrong? if the acquisition program is saving as fits, it might be "upside-down" relative to how PI's fits reader is set up. you can either change the debayer setting to the opposite one (for instance if it's supposed to be RGGB then set BGGR), or open up the format explorer tab on the left, double-click FITS and change the reader direction to the opposite of what it's set to...

rob

[kwalker]

Color is off
I was starting to wonder if my original blunder in saving the data in SBIG format uncompressed could have caused the problem during the conversion to FITS.
PI can not read SBIG format.
converting from one format to another may be skewing the color channel alignment.
That's what I get for reading the manual (recommends saving in SBIG format either compressed or uncompressed)
I haven't tried different debayer formats ,that does make a lot of sense.
If the skies cooperate this evening I will try another quick run ,and save as FITS.
I was hoping STF 8300c users would chime in.
either way ,the artifacts in the final stack in PI is alarming.
I wore myself out yesterday trying to wrap my head around the error in Batch Preprocessing ,after being up all evening with my new toys.

[kwalker]

I see the powers to be edited my original post and removed the images because I used dss for stacking.
The image was entirely processed in PixInsight.
not having the batch preprocessing and dss stack to compare makes it difficult to see the problem.
cant get help that way.
oh well

[kwalker]

 Just so we have some point of reference I will upload screen shots in compliance to rules.


after Batch Preprocessing LRGB channels split





After many hours I was able to gain with Image registration set to Mitchell-Netravali Filter for light frames  but can not reject noise.
 flats seem to be working darks and bias minimal if any with all integration settings.

[speach]

Let me first say that I'm just a starter at PI, but batch pre-processing does not seem to work as well as going through the pre-processing steps  individually. At least that is what I've found.   

[kwalker]

I've always had very good results with BPP (DSLR) and use PI for every step of the processing procedure.
This was a big shock to me and I'm sure there is an answer, just haven't been able to image since the first light acquisition a few nights back.
Now I get to enjoy the New Mexico cold ,rain ,and snow for a week or two 
Another thread was brought to my attention which shared similar errors I experienced   https://pixinsight.com/forum/index.php?topic=8228.0
the thread kind of died out so I just have to be patient until the photons come my way again.
the color from the deleted picture is most likely the inability to calibrate.
So changing settings at this time is rather a moot point.
I'll report back when I can.

[kwalker]

Update
I brought the camera inside ,updated the firmware, and re shot the dark frames.
Now I'm a first time CCD imager and long time DSLR so BIG mistake on the dark frames.
I was using 10 minute darks and 5 minute lights.
I can get away with that with a DSLR but rarely mismatch files because data is so hard to get.
I know I should only try one step at a time to address an issue so as to come to the right conclusion but for the time being lets say the firmware update had little to do with the issue.
The Screen shots (ALL PixInsight)and screen transfer with background neutralization ,no processing
Left image is fleeing light flats.
The right image is the tee t-shirt trick.
I need to clean a sensor for one thing and go back to school on flat frames.
maybe an EL panel and the right ADU settings for flats(haven't a clue)
Maybe someone can give me a tip or two on what's going on with my flat frames with BPP
I refuse to use DSS when PI does it all.
The rest I got.
EDIT  After doing a little research the duration of the dark frames should not have caused the issue.
http://pixinsight.com/forum/index.php?topic=6797.msg46103#msg46103
That being said I'm still investigating the root cause which may be file corruption and not BPP at all.








The next screen shot shows an error that is random in its timing (no correlation between master dark and target frames)
I'm more curious than anything about this issue.

[Juan Conejero]

* Subtracting pedestal keyword 'PEDESTAL': -100.0000 DN

This is the problem, or at least one of them. You are probably using frames that have already been calibrated with other application, since PEDESTAL keywords written by PixInsight have always zero or positive values. Note that -100 is being subtracted from these frames prior to calibration, which is obviously an error.

In general, images calibrated with other applications cannot be used with PixInsight. You should perform the whole preprocessing work (calibration, registration and integration) in PixInsight to get coherent results.

[kwalker]

Thanks Juan
The files are direct fits downloads form the STF 8300c.
except for the light frames which were converted from SBIG uncompressed to Fits.
there must be a file corruption somewhere either before the firmware update or during acquisition such as usb hubs or the repeater which I tossed after this first light run.
I did go over all the data to make sure every fits file was the raw data acquired. (doesn't mean I didn't catch everything)
I'll re evaluate the data.
I was only able to complete the BPP process after re taking the darks.
the only thing left is addressing the flats.

[kwalker]

I was able to collect a few frozen photons last night ,Humidity was near 90% and cold with frost on everything within an hour.
blasting the dew heaters ,fighting the city lights to the north and the moon I fought through 6 hours of imaging.
not the best data one would wish for but the object was to find the cause of my initial problem.
I collected everything in FITS., Flats ,Bias ,Darks ,and lights.
Now data corruption (not sure why) may have been the cause ,or the firmware update fixed the issue.
Still getting correlation warnings (darks to target image during flat calibration)but none failed.
need to work on a few things but I did make progress.
Debayer was set to BGGR.
Data was at best.... POOR