PixInsight Forum

Software Development => New Scripts and Modules => Topic started by: NKV on 2010 June 19 03:53:20

Title: Bias Normalization script
Post by: NKV on 2010 June 19 03:53:20
Hi everybody.

Unfortunately, Bias of my camera (FLI ProLine16803) is varying from ambient temperature.  >:D
CCD temperature is fixed and stable, but Median(Bias) in low speed (low noise mode of reading from CCD) vary from 1900 to 2100 ADU. In winter Median(bias) lower.
Readout noise from my camera about 6.3 ADU, so I can't close my eyes on the +/-100ADU. And of course I can't go via standard ImageCalibration route, because excellent DarkOptimization route some time report about very strange coefficient of dark scaling.  I must pre process all images before.

I got solution to avoid the strange Bias problem:
1) I always take one Bias for every one Light frame.
File01_Light + File01_Bias,
File02_Light + File02_Bias,
File03_Light + File03_Bias, ...
Same for Dark and Flat.

2) I make MasterBias from all Bias files
3) I normalise every Light/Dark/Flat via PixelMath: FileXX_new = FileXX + Median(MasterBias) - Median(FileXX_Bias)
4) ImageCalibration without problem.

OK, in attachment very simple script. Only for doing point 3) in batch mode: Bias Normalization.
I understand, the script only for very specific purpose, anyway I want share it. Maybe other one ProLine16803 have similar problem.
To use it, you must select MasterBias, Light (or other) frames which you like normalise. Set Suffix for Light and Bias (like you set it in MaximDL>AutoSave) and of course you should have Bias for every one Light in same folder. Script will create new file with "_n".

Best regards,
Nikolay.
Title: Re: Bias Normalization script
Post by: Niall Saunders on 2010 June 19 07:48:39
Because I have 'grown up' with the Meade DSI cameras,

I have learned (the HARD way) that you simply cannot (or, at least, 'should not') rely on any kind of 'library data' for your calibration files.

I now take fresh calibration images every session - it doesn't really have any time penalties (at least not for the small image size and rapid download speed of the tiny DSI cameras), although that cannot obviously be said for the acquisition of Darks. However, with the luxury of an observatory, I just leave the system acquiring Darks once I have finished for the session - they will all be 'fresh and ready' by the following morning / evening.

All I know is that I have NEVER been able to trust that the Bias level will be the same session after session - I don't know why it changes, and I haven't even bothered to see what the exact changes are, or how they might correspond with the environment. I just know that they 'can' change, and my imaging time is too precious for that possibility to add to the huge list of candidates for image-vandalism >:(

So, I take Bias frames because I can. But I don't use them because I take dedicated FlatDarks for dedicated Flats, and dedicated Darks for Lights.

And even then, my images are rubbish :yell:

Cheers,
Title: Re: Bias Normalization script
Post by: NKV on 2010 June 20 22:03:23
Hello Niall,
my situation bit different.
'library data'
I collect library for 2 years, and some time I compare old data with new. Except some unstable pixels old and new data is same. So to use or not use 'library data' it's depend on camera model. For me 'library data' work very well.

Quote
I now take fresh calibration images every session
I take fresh to, if I have time. But pedestal level in all frames (Light/Dark/Flat/Bias) rapidly grow with ambient temperature. So all my frame have a bit different pedestal during one session. One solution to take Dark mediately after Light, but I prefer take Bias (especially with exposure of Light more than 10min).

Quote
it doesn't really have any time penalties
I alway take Light and Dark for Light with RBI enabled in low noise mode. To take 1 frame I spend minimum 2.5min ( 5x (RBI flashing + reading) ). To take Bias or Flat I spend 30sec ( CCD reading time in low noise mode).

Please understand, I doing Bias Normalization because all frames (one night) have different pedestal. I do it not because I do not have fresh calibration frames.
Title: Re: Bias Normalization script
Post by: mmirot on 2010 June 21 09:26:17
It is interesting script.
I have the same camera.
I will have to check my bias stabilty.

I have to ask if it worth all the extra effort especially with RBI turned on. What is the difference in S/N verses just get a single offset value determined from average value from a bias at the start and end of the run?

Max
Title: Re: Bias Normalization script
Post by: mmirot on 2010 June 21 09:49:04
I just realized. You are saved from the extra work!

Your camera has a overscan feature!!

This allows you get the change in bias offset as part of each exposure.

You can do this easily set this up with MaxIM DL. This is the light protected pixels which can be read out with each image. It comes out as a dark edge tab on each image which you can trim off later. The ave value of these pixel provide the correct offset.
You do not need to shoot a bias with each frame. A master from the library is still used for the nose structure.

BTW (Sbig recently camera out with a driver to read the overscan part of the chip.)

We need to use the PI integration process work with this.
I saw that overscan is part of the of the integration module.
Perhap Juan or Vincent can tell us how to use this?

I suspect your scipt may turn out to be a orphan.

Max

Title: Re: Bias Normalization script
Post by: NKV on 2010 June 21 20:44:32
Hi Max.
Your camera has a overscan feature!!

This allows you get the change in bias offset as part of each exposure.

You can do this easily set this up with MaxIM DL.
Sounds good. But how? I don't see any overscan features in MaxIM DL for FLI ProlLine16803. May be I use wrong plugin?

BR,
Nikolay.
Title: Re: Bias Normalization script
Post by: mmirot on 2010 June 21 21:02:20
Yes you need the right MaxIM DL  plug in. 
 I think you find it under the exposure,  options tab.
You can check the the Yahoo FLI group for more details.

I took me a little while to find the settings in MaxIM

If can find it try downloading the latest driver and plug in

Max
Title: Re: Bias Normalization script
Post by: mmirot on 2010 June 21 21:03:15
 I Started and new thread on overscan calibration. 

 http://pixinsight.com/forum/index.php?topic=2015.0
Title: Re: Bias Normalization script
Post by: NKV on 2010 June 21 23:20:38
You can check the the Yahoo FLI group for more details.
Max, thanks for pushing me to right way. I hope you speak about this driver: http://tech.groups.yahoo.com/group/FLI_Imaging_Systems/files/Software/
Title: Re: Bias Normalization script
Post by: mmirot on 2010 June 23 08:12:27
If you are using the RBI driver you have it already.
Go to where the RBI flash flood settings are found.
It is just to right of it.
Check enble overscan and try some numbers such as 25 verticle 25 horizontal.

Take a light frame. Look at the boarders .
There should be dark boarders on the image now( overscan pixels).

Max
Title: Re: Bias Normalization script
Post by: NKV on 2010 June 23 09:08:41
Hello Max.
Thank you for help with my problem.
Ok, I installed latest FLI driver and MaxImDL PlugIn ( from FLI official site). Now I can enable Overscan in MaximDL > Camera Control > Camera Settings > Overscan.
But, what about size of overscan area? I ask you because I got strange size:
Normal frame size 4096x4096.
With Overscan W16, H16 frame size 4148x4133
How to understand ?
What about of your frames size and overscan parameters ?

Also, if I change Overscan Width/Height without reconnecting (via Maxim menu), I see error:
"An invalid parameter was passed to the plug-in driver. (-3)"
Do you have same error?

Best regards,
Nikolay.
Title: Re: Bias Normalization script
Post by: mmirot on 2010 June 23 13:36:14
I am still trying to figure that out myself. I have tried 25 X 25. I am just starting overscan now that I understand how to use it in PI.

Max
Title: Re: Bias Normalization script
Post by: NKV on 2010 June 23 14:04:39
I am still trying to figure that out myself. I have tried 25 X 25.
And you got 4096x4096 or (4096+25)x(4096+25) or ?x?
Size of your image with overscan 25x25?
Title: Re: Bias Normalization script
Post by: enzosantin on 2012 May 03 16:35:13
Hi everybody!

I have the same problem with my camera (FLI ProLine16803): varying +/- 100ADU depending on ambient temperature.

I want to ask to Nikolay and Max what size of overscan area I must set in MaximDL? 25x25? or ?x? What area is "real" overscan?

And how I must set overscan parameters in PI integration process?

Many thanks!

Enzo
Title: Re: Bias Normalization script
Post by: mmirot on 2012 May 03 20:37:08
You just reminded me to start using overscan. I forget the settings. So you might ask the fli group too
Max
Title: Re: Bias Normalization script
Post by: NKV on 2012 May 04 00:36:31
I want to ask to Nikolay and Max what size of overscan area I must set in MaximDL? 25x25? or ?x? What area is "real" overscan?

And how I must set overscan parameters in PI integration process?
I can't check MaximDL right now, also it's depend on FLI-MaximDL driver version. Now I use driver "FLI-New-Betta" and getting images 4157x4142 pixels. After ImageCalibration(see screen with settings) a getting 4098x4098(real light sensitive area).
Title: Re: Bias Normalization script
Post by: enzosantin on 2012 May 04 03:20:22
Max, NIkolay, many thanks! If you remember maxim settings, please let me know!
@ Nikolay, I don't see your ImageCalibration screen with settings... perhaps you have forgotten to link it? Thanks again

Enzo
Title: Re: Bias Normalization script
Post by: NKV on 2012 May 05 00:09:24
Nikolay, I don't see your ImageCalibration screen with settings... perhaps you have forgotten to link it?
Sorry.
Title: Re: Bias Normalization script
Post by: enzosantin on 2012 May 05 03:13:16
Thank you, Nikolay! Sorry, I have some other questios: have you solved the drifting bias problem whith overscan? In your opinion you get better results? And which is your calibration workflow whith this? Thanks
Enzo
Title: Re: Bias Normalization script
Post by: NKV on 2012 May 05 03:24:19
Overscan in ImageCalibration module solve the drifting bias problem.
Title: Re: Bias Normalization script
Post by: vicent_peris on 2012 May 05 09:26:59
Hi,

The what overscan areas are intended for. Now we have a very simple implementation of overscan correction, but in the future we'll add more capabilities:

- Multiplicative or additive correction.

- Correction with splines, instead of simply calculating a pedestal.


Regards,
Vicent.
Title: Re: Bias Normalization script
Post by: mmirot on 2012 May 05 16:20:06
We need to add this function to the preprocessing batch script.  Tell Juan to add it to a V 1.1 release

Max