PixInsight Forum (historical)

PixInsight => Release Information => Topic started by: Juan Conejero on 2014 July 05 13:23:33

Title: New Tool: Superbias
Post by: Juan Conejero on 2014 July 05 13:23:33: Hi all,

I have just released an update to the ImageCalibration module that includes a new tool: Superbias. This update is now available on all supported platforms: FreeBSD, Linux, Mac OS X, and Windows.

The Superbias method was created by Vicent Peris, and initially implemented (http://pixinsight.com/forum/index.php?topic=3269.0) as an unofficial module by Carlos Milovic, back in 2011. Today we have released a completely revised implementation as a new standard PixInsight tool.

The Superbias algorithm generates a noise-free master bias frame, equivalent to a master bias integrated from thousands of individual bias frames, by extracting pixel-to-pixel differences with vertical (or horizontal) orientation and large-scale structures from a real master bias. The better the source master bias you use, the better the generated superbias will be, but excellent results can be obtained from a master integrating a reduced number of frames, say just 20 or 40 bias frames. You can use a superbias for image calibration purposes, just as a regular master bias frame. By calibrating your data with a superbias, you save the noise introduced by subtraction of a real master bias frame, and your images benefit from a more accurate modelling of the bias signal.

The new implementation uses the multiscale median transform (MMT) to perform the necessary multiscale analysis. Previously the starlet transform (AKA à trous wavelet transform) was used, which is now also available as an option. The MMT, despite being computationally expensive, is now used by default because it is much better at isolating structures in the multiscale representation. The new tool also implements a robust trimmed mean algorithm to compute the mean value of each column (or row), which replaces a previous implementation using more traditional statistical methods (one-step rejection based on the standard deviation), and an option to remove large-scale gradients from the internal working images used for calculation of mean column (row) values. Most of these features have been suggested recently by Mr. Peris.

To use Superbias, open a master bias frame and apply the Superbias tool to it. It's just that simple. A new image window will be generated with the superbias frame, which you can save to a FITS file for later use. To inspect a superbias, you should use the ScreenTransferFunction tool (STF), or simply press Ctrl(Cmd)+A to apply STF AutoStretch. Using 24-bit STF LUTs is necessary in most cases.

See the following screenshot for an example of superbias generation. In this case the source master bias has been integrated from 40 bias frames.

(http://forum-images.pixinsight.com/20140706/SB/superbias-tn.jpg) (http://forum-images.pixinsight.com/20140706/SB/superbias.jpg)
Click for full-size image (http://forum-images.pixinsight.com/20140706/SB/superbias.jpg)

I hope you'll find this new tool useful. Thank you for your continued support.
Title: Re: New Tool: Superbias
Post by: Andres.Pozo on 2014 July 06 04:44:00: I don't know if I am doing something wrong, but the SuperBias process doesn't seem to work with my bias images. I have a QSI540 with a KAI-04022 sensor. This sensor produces quite "dirty" raw images but the calibration fixes them very well.
I have applied SuperBias to a 200 frame bias and the result is worse than the original. In the original there are partially lighter columns that in the SuperBias go fully from top to bottom. I have attached a cropped comparison of the same area.

I don't know whether this process can be used with my camera or I am doing something wrong.
Title: Re: New Tool: Superbias
Post by: vicent_peris on 2014 July 06 05:09:50: Hi,

The Superbias tool is designed to reproduce the column structure of the bias pattern, so it doesn't work with bias frames with partial columns. But I think we can think on a solution for this and publish an update in the future.

Thanks,
Vicent.
Title: Re: New Tool: Superbias
Post by: btracy on 2014 July 06 07:56:38: I got the same sort of results as Andres. The master bias was built from 20 frames from my ST-8300. Could be that I am at (or below) the minimum needed for the new process to work.

Bob
Title: Re: New Tool: Superbias
Post by: Carlos Milovic on 2014 July 06 09:05:00: Bob, are you looking at the data with the 24bits STF enabled?
Title: Re: New Tool: Superbias
Post by: btracy on 2014 July 06 09:09:41: Yes.
Title: Re: New Tool: Superbias
Post by: vicent_peris on 2014 July 06 09:37:50: Bob, enable the track view of the STF to see the STF settings of the regular bias. Then, drag and drop the triangle over the superbias image to apply the same STF settings to both images.

V.
Title: Re: New Tool: Superbias
Post by: btracy on 2014 July 06 10:22:52: Thanks Vincent,

I didn't think about that at the time, too much of a newbie. Looks better when you apply the same stretch. I really appreciate the help.

Bob
Title: Re: New Tool: Superbias
Post by: Harry page on 2014 July 06 10:48:51: Hi
I also have this problem !!!

Harry

Quote from: Andres.Pozo on 2014 July 06 04:44:00
I don't know if I am doing something wrong, but the SuperBias process doesn't seem to work with my bias images. I have a QSI540 with a KAI-04022 sensor. This sensor produces quite "dirty" raw images but the calibration fixes them very well.
I have applied SuperBias to a 200 frame bias and the result is worse than the original. In the original there are partially lighter columns that in the SuperBias go fully from top to bottom. I have attached a cropped comparison of the same area.

I don't know whether this process can be used with my camera or I am doing something wrong.
Title: Re: New Tool: Superbias
Post by: ftherrmann on 2014 July 06 15:10:33: Here's a 24 bit display. Doesn't look right to me. STT-8300. The partial column went full length and it generated a type of moire pattern. I can't see this working better than the original bias.

Fred
Title: Re: New Tool: Superbias
Post by: Juan Conejero on 2014 July 06 16:00:15: Bias frames with partial columns are still not supported by the Superbias tool. Sorry for the inconvenience. We'll release a new version to overcome this limitation, but probably not before September.

Quote
...it generated a type of moire pattern.

Superbias images definitely require 24-bit STF for proper visualization. I see some screenshots posted here where normal 16-bit STFs are still being used.
Title: Re: Superbias and MaximDL
Post by: brew on 2014 July 06 17:19:21: I created some superbias frames, then tried to use them in MaximDL. Maxim does not recognize the frames as Bias frames in the Calibration Setup tab.

After experimenting, I found that the IMAGTYPE keyword in the Fits header needed to be changed. PI is using "Master Bias", while Maxim is looking for "BIAS". When I changed this keyword, the frames worked as expected.

I am another ACP user, so I need to use Maxim for image collection. Calibration is very convenient, since the images are calibrated as they are collected. I do not have to do anything, the calibrated images are ready the next morning. I use PI for alignment and everything after that.
Title: Re: New Tool: Superbias
Post by: sedonapete on 2014 July 06 17:36:18: Works a treat for me.

Thanks for all the work.
Title: Re: New Tool: Superbias
Post by: vpcirc on 2014 July 06 18:18:09: Worked great with my 16803 chip and 40 bias frames, now if only it could work on darks!
Title: Re: New Tool: Superbias
Post by: ftherrmann on 2014 July 06 18:21:01: Juan,

Yes I was aware of the column issue. I am however more concerned with the pattern that the SuperBias is producing. This same pattern is also seen in other images posted. I did select 24 bit from PI. Is there a 24 bit setting somewhere else that I need to set? I know of only one place and that's at the bottom of the PI screen.

Thanks,

Fred
Title: Re: New Tool: Superbias
Post by: ftherrmann on 2014 July 06 18:23:41: Quote from: Juan Conejero on 2014 July 06 16:00:15
Bias frames with partial columns are still not supported by the Superbias tool. Sorry for the inconvenience. We'll release a new version to overcome this limitation, but probably not before September.

Quote
...it generated a type of moire pattern.

Superbias images definitely require 24-bit STF for proper visualization. I see some screenshots posted here where normal 16-bit STFs are still being used.

Is the image I posted being displayed in 16 bit? I selected 24 bit from the bottom of the PI screen. I couldn't find any bit settings within the STF itself. Could you please be a little more specific?

Thanks,

Fred
Title: Re: New Tool: Superbias
Post by: Alejandro Tombolini on 2014 July 06 19:26:48: Hi Fred,

Note the second icon in the Screen Transfer Function (http://pixinsight.com.ar/en/docs/142/pixinsight-tool-bar--screen-transfer-function.html) tool bar. That is the one you have to select.

Saludos. Alejandro.
Title: Re: New Tool: Superbias
Post by: vpcirc on 2014 July 06 19:58:43: I noticed if I tried it on a master bias that was created with BPP I got an odd pattern. I made a new master bias with image intragration with the recommended settings and then it worked perfect.
Title: Re: New Tool: Superbias
Post by: TobiasLindemann on 2014 July 06 21:23:14: Works great with my DSLR-Biases. Thank you Vicent and Juan :-)
left: masterbias with 270 frams at ISO 800
right: superbias
Title: Re: New Tool: Superbias
Post by: naavis on 2014 July 07 01:17:35: Quote from: ftherrmann on 2014 July 06 18:21:01
Juan,

Yes I was aware of the column issue. I am however more concerned with the pattern that the SuperBias is producing. This same pattern is also seen in other images posted. I did select 24 bit from PI. Is there a 24 bit setting somewhere else that I need to set? I know of only one place and that's at the bottom of the PI screen.

Thanks,

Fred

I think the bottom setting only sets the bits the probe uses to show the pixel values under your cursor. The 24-bit STF setting is on the top toolbar row next to the auto STF buttons.
Title: Re: New Tool: Superbias
Post by: MikeOates on 2014 July 07 03:52:12: No one has mentioned this yet, but how do you use the vertical and horizontal option. Setting this produces two superbias frames. Do you combine them in some way?

Mike
Title: Re: New Tool: Superbias
Post by: Juan Conejero on 2014 July 07 04:23:58: No, you use just one orientation, the one that fits your bias frames. This is normally the vertical (columns) orientation. The horizontal orientation is available in case you have to work with rotated bias frames for some reason, and also for other analysis tasks, different from generation of superbias frames. For example, you can use both orientations to discover "hidden" oriented structures in your images.
Title: Re: New Tool: Superbias
Post by: ftherrmann on 2014 July 07 08:31:14: Thanks for the 24bit STF button info. Seems like this should be included in the tool also, but I've never used it until now so not really sure it would do much other than to clutter up the control. Maybe a tab or menu access from the STF function box?

I enabled the 24 bit and while the moire type pattern does get better it's still there. Is the a function of not using enough bias files?

Thanks,

Fred
Title: Re: New Tool: Superbias
Post by: Juan Conejero on 2014 July 07 09:17:08: The 24-bit STF feature was released in November 2013. See this tutorial (http://pixinsight.com/tutorials/24-bit-stf/) for a complete description.

There can't be moiré patterns (a special type of aliasing effect) in a superbias. However, if you look at a superbias frame at a reduced scale on the screen, especially with reduction factors of 1:3 or smaller, there can be visible aliasing patterns as a result of fast/inaccurate screen renditions of the fine bias column pattern. For a correct visualization, take a look at the superbias frame with 24-bit STF enabled at its actual size with a zoom ratio of 1:1, or at half its size with a ratio of 1:2.
Title: Re: New Tool: Superbias
Post by: pscammp on 2014 July 07 10:12:36: Hi all,
and you think your newbies LOL

So here i have a master Bias from 21 frames aquired with a modded
Canon 1000D DSLR and the master was produced by PI (if i even got
that right)

Not sure if i fully understand what this 24bit SFT thing is but let
me tell you tell you what i did and you can tell me if i did wrong.

1. Opened my Master Bias

2. Normal SFT applied

3. Clicked the little icon at the top of the screen which has got '24' on it (image changed a bit)

4. dropped the Superbias triangle on the image & got a black image as a result

5. Tried dropping the STF's triangle on the new image but the image stayed black and the STF's
3 bars just reset back to default in the center of it's view (Bit Confused Here)

6. Then i just applied standard STF to the Superbias image (looked real rough)

7. Clicked on the '24' button again with the superbias version selected and the result is on the
right in the attached pic.

Both pics in the attached image have the '24' button applied to it and the '24' was applied to the
origonal bias master before applying the Superbias to it.

So the big question is:

Is this what i want or is it horribly wrong ????

Many Thanks All
Paul
Title: Re: New Tool: Superbias
Post by: TobiasLindemann on 2014 July 07 14:40:44: Hi Paul,

the picture you posted looks like a normal bias. So everything you have done is correct.
Btw.: you don´t have to use the STF at all. It is only for better visibility, you know.

To point 5: if you activate the "track-view"- button, the STF will be reset by the new picture.

Greetings
Tobias
Title: Re: New Tool: Superbias
Post by: pscammp on 2014 July 08 08:52:23: Cheers Tobias,

I almost got something right Wooo Hoooo

So i assume then that this new 'Superbias' image i have here is used in place of my origonal 'Master Bias'
for the calibration/integration proccess etc

If so, what settings would i use for the .fits file when saving it, It defaults to a 32bit IEEE 754 Floating Point,
would it need to be a different one like 64bit for instance ?????

Thanks
Paul
Title: Re: New Tool: Superbias
Post by: Mark Manner on 2014 July 08 10:00:16: I can report it works very well for a 16803 chip camera [SBIG STX16803]
Thanks
Mark
Title: Re: New Tool: Superbias
Post by: vicent_peris on 2014 July 08 10:23:12: Hi,

I already have in mind what should be done to replicate the partial columns, so now we only need to find the time. :P But expect a new version of the tool, maybe after summer.

Best regards,
Vicent.
Title: Re: Superbias and MaximDL
Post by: Juan Conejero on 2014 July 08 11:29:13: Quote from: eridanibrew@gmail.com on 2014 July 06 17:19:21
After experimenting, I found that the IMAGTYPE keyword in the Fits header needed to be changed. PI is using "Master Bias", while Maxim is looking for "BIAS". When I changed this keyword, the frames worked as expected.

I am another ACP user, so I need to use Maxim for image collection. Calibration is very convenient, since the images are calibrated as they are collected. I do not have to do anything, the calibrated images are ready the next morning. I use PI for alignment and everything after that.

Make sure that the other application is able to support the normalized [0,1] range that PixInsight uses to write floating point images, or the superbias frame won't work at all.
Title: Re: New Tool: Superbias
Post by: TobiasLindemann on 2014 July 08 15:00:51: Paul
Quote from: pscammp on 2014 July 08 08:52:23
So i assume then that this new 'Superbias' image i have here is used in place of my origonal 'Master Bias'
for the calibration/integration proccess etc
yes
Quote from: pscammp on 2014 July 08 08:52:23
If so, what settings would i use for the .fits file when saving it, It defaults to a 32bit IEEE 754 Floating Point,
would it need to be a different one like 64bit for instance ?????
I think 32bin floating point is enough.
Title: Re: New Tool: Superbias
Post by: mfulmer on 2014 July 12 09:59:25: I'm probably doing something stupid... but I can't find the Superbias tool. I'm running 01.08.02.1098 Ripley on OS X Mavericks 10.9.4.

Note - I am able to make it work by manually installing the module from http://www.astrophoto.cl/Research.html.

Mike
Title: Re: New Tool: Superbias
Post by: pscammp on 2014 July 13 03:30:33: mfulmer,
Did you search for the tool in 'Resources/Update/Check For Updates' from inside PI, that's where I
found it, the one from your link is probably not the full release version the guy's have released.

You should find it in the 'Image Calibration' sub folder if it's already there, if not then 'Check for updates'
will show it up for you.
Title: Re: New Tool: Superbias
Post by: mfulmer on 2014 July 13 15:25:13: I did. There was nothing under the Image Calibration sub folder (I'm assuming you mean in the process explorer). So I checked for updates and nothing was there. I had performed the update earlier, so it does show that the update for Superbias was applied. That's when I tried the module from astrophoto.cl - just to make sure it would work. Then, super bias showed up under Image Calibration. Should I re-install PixInsight?
Title: Re: New Tool: Superbias
Post by: mfulmer on 2014 July 13 15:37:40: Actually - here's what the Manage Modules tool says about Image Calibration:

Module Id ............ ImageCalibration
Version .............. 01.01.06.0175
Language ............. eng: English
Development status ... <* Unspecified *> (release)
Release date ......... 2014/6/23
Company .............. Pleiades Astrophoto
Author ............... ImageCalibration: Juan Conejero, PTeam / DefectMap: Carlos Milovic F., PTeam
Copyright ............ Copyright (c) 2009-2014, Pleiades Astrophoto
Trade marks .......... PixInsight
Original file name ... ImageCalibration-pxm.dylib
API version .......... 147
Module Description:
PixInsight Standard ImageCalibration Process Module

Installed Updates has this to say about ImageCalibration:

Package type : Module
File name : 20140705-macosx-x86_64-module.tar.gz
Release date : 2014-07-05 00:00:00
Install date : 2014-07-12 16:18:22
This update installs a new version of the standard ImageCalibration module for PixInsight 1.8.2. Version 1.2.0 is an update release:
* New Superbias tool. The Superbias method was created by Vicent Peris and initially implemented by Carlos Milovic in 2011. This update installs an improved version of this tool, completely revised and reimplemented as an official PixInsight tool on all platforms. For more information on the Superbias algorithm, see the following thread on PixInsight Forum:
http://pixinsight.com/forum/index.php?topic=3269.0
Copyright (c) 2014 Pleiades Astrophoto S.L.

That looks like a problem (01.01.06.0175 vs 1.2.0) - thoughts?

And let me know if this post is trending in a direction where I should post this in another area of the forum.

Mike
Title: Re: New Tool: Superbias
Post by: mfulmer on 2014 July 13 15:44:09: Ok, never mind - I've restarted everything again and now it's showing up. :tongue:

Thanks for the help, pscammp.
Title: Re: New Tool: Superbias
Post by: Astrocava on 2014 August 26 05:10:27: Is this tool available in 32-bit version?
Title: Re: New Tool: Superbias
Post by: georg.viehoever on 2014 August 26 08:17:54: No. 32bit versions of PI are no longer supported.
Title: Re: New Tool: Superbias
Post by: Astrocava on 2014 August 26 09:19:17: Quote from: georg.viehoever on 2014 August 26 08:17:54
No. 32bit versions of PI are no longer supported.
Thanks for confirming, Georg. Computer change is coming...