raw format APT/Fits format Sgpro not compatible?

[Cafo78]

Hello everybody,
hope someone can help me. I shot light and dark frames in FITS format with Sgpro and flat frames in raw format with APT.
The problem is getting them work together...I tried all possible combination in BatchPreprocessing but the problem is always the same: flat frames are discarded or the process gets stuck with warning problems. Bottom line, the flat frames are not used to calibrate the final image.

Is there something that I don't know or I have to do before using those flats? Or maybe it's better to take them in fits format with Sgpro? Or select raw format in Sgpro instead of fits   

Thanx to anyone is able to give me a "hope" or at least an answer.

Ohh by the way, happy 2020!!

Mirco

[dave_galera]

You should ALWAYS use the same camera and software for ALL images, i.e. Lights, Darks and Flats

[Cafo78]

Yeah... I suspected that. But taking flats with Sgpro it’s quite a pain..it takes soo long to download a single frame. That’s why I prefer APT, much faster to take flats.
Nevermind, thanx for the hint!

Mirco

[John_Gill]

Hi,

You could try and make “synthetic flats” so at least you can use the Light frames.

space is not black
John

[Cafo78]

Ok, I will give it a try. But the problem remains for the next time... I can’t understand why, maybe it’s related to the fact the raw files from APT and fits file from SGPRO don’t have the same CFA pattern?

[bulrichl]

Hi Mirco,

this is again a case of missing information. Please tell us which camera make and model you are using.

Bernd

[stevek]

Yeah... I suspected that. But taking flats with Sgpro it’s quite a pain..it takes soo long to download a single frame. That’s why I prefer APT, much faster to take flats.
Nevermind, thanx for the hint!

Mirco

Why so?  Given the same ADU the longest amount of time in taking the flats is the download time and that is a function of the size of the frame and the download speed of the camera/computer combination.  The capture SW shouldn't make any difference.

[John_Gill]

Flats are used to fix optical errors, so using a different camera changes the optical system. Flats are normally only 1 to 2 seconds exposure, so will only take a couple of minute to complete.  ... no short cuts or you will end up with a mess.....

John

[dave_galera]

Flats are used to fix optical errors, so using a different camera changes the optical system. Flats are normally only 1 to 2 seconds exposure, so will only take a couple of minute to complete.  ... no short cuts or you will end up with a mess.....

John

+1......There has been several threads that have highlighted this problem in recent months

[pfile]

Yeah... I suspected that. But taking flats with Sgpro it’s quite a pain..it takes soo long to download a single frame. That’s why I prefer APT, much faster to take flats.
Nevermind, thanx for the hint!

Mirco

Why so?  Given the same ADU the longest amount of time in taking the flats is the download time and that is a function of the size of the frame and the download speed of the camera/computer combination.  The capture SW shouldn't make any difference.

unfortunately it does make a difference - for certain cameras there are both an ASCOM driver and a driver which can be compiled into the capture application itself. some of these CMOS cameras have no RAM buffer and the image is read right off the chip and over the USB bus; the configuration of the compiled-in driver and the default configuration of the ASCOM driver are almost certainly different, as some people report exactly what the OP is saying - SGP is super slow (ASCOM-based) and other tools are fast (built-in driver based.) one of the camera settings is how much USB bandwidth to utilize, which as you can imagine impacts the download speed. it's not clear to me that the ASCOM and built-in drivers can be configured equivalently.

one problem with these cameras is that they have various readout-related amp glows, and so the readout speed changes the data coming off the camera; slower downloads exhibiting more amp glows and other artifacts.

for this reason it's really important to use the same capture software *and* driver type (if that is an option in your software of choice) for all of your frames - lights, darks, bias and flats. otherwise the data just won't match up.

rob

[Cafo78]

My camera is a modified canon550d, connected via usb to the computer.
SGPRO is much slower in downloading a single image. It takes about 12/15 seconds whereas apt is capable of a fast download in about 2 seconds. That’s why I preferred taking flats with apt.

I tried to take flats with sgpro (fits format) and seems like Pixinsight is working now... maybe the issue was related to the file format. Really don’t know...

[pfile]

i'm not disagreeing that one is faster than the other... but you never said what kind of camera it was so i was pointing out sort of the worst case situation.

one thing to consider is that if you save DSLR files as fits in SGP then SGP rescales the 14-bit data to 16-bits. i don't know if APT does the same thing, and even if it does, if it doesn't do it using the same algorithm then that would account for the difference between frames.

you could always take bias frames in APT just to calibrate the flats. once a flat has been calibrated it should be compatible with any lights, and any rescaling that migh have taken place in the capture program should not matter as flats are divided into the lights.

rob

[dave_galera]

Thanks for the concise explanation(s) and clarification Rob

[guest13093]

I've had the same problem taking lights with ASIair and calibration frames with SGPro or vice-versa.

If I pre-process in another well known software package (that contains the letters A and P) there are no problems; lights and all calibration frames are recognised and used without any problems.

If I use PixInsight the flats are always rejected unless they come from the same imaging package. I have found the problem to be with the way ASIair and SGPro record the details of the filter in the fits header. In the past I have used a well known fits header editor to make the filter details the same. However, in the end i decided to get calibration frames using both the ASIair and SGPro and use the appropriate flats as the need arises.

PixInsight is fussy about the exact details of the fits header; if lights and calibration frames differ they are rejected by PI. The other package doesn't seem to give a damm - it just gets on and calibrates the lights.

HTH

Adrian

[pfile]

that's a different issue - you're talking about metadata in the flats that BPP or WBPP needs to match flats to lights.

i'm talking about the actual data that's been recorded in the file. note that calibration is essentially a "blind" process - just subtraction and division - and no tool can really know if the calibration masters are appropriate or not. usually the only way you know your calibration was wrong is that the flats have overcorrected or undercorrected the light.

so just because another package calibrated the images doesn't mean too much about the result. it's always "garbage in, garbage out" and unless you really understand what the different capture programs are doing and how they are saving the data, you're taking a risk by mixing capture programs.

rob