Author Topic: DefectMap process (pre-release)  (Read 10832 times)

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
DefectMap process (pre-release)
« on: 2010 May 21 21:31:19 »
Here is a pre-release of the DefectMap process, which will be included in the next PI Core distribution, as part of the ImageCalibration module. As the ImageCalibration process, DefectMap is released as an open source project, subject to the GPL.

Brief decription: This process allows the user to use a reference image (called "Defect Map") to indicate bad pixels on a target image. The Defect Map may be a grayscale or color image, where 0 values means "bad pixel" (gray values are used as scaling factor in convolution modes, see below). So, for "graphical reference" it needs to look like a inverted dark frame. Also, it is assumed that the Defect Map needs no further processing, it is ready to use. If you are using a dark frame, or other calibration frame as defect map, you may use any other process to modify it.
Besides from the Defect Map image selection, the user may choose between five operation methods: Mean, Gaussian Blur, Median, Minimum and Maximum. Those operations needs a structuring element, defined by the user following any of these geometries: Square, Circular, Row or Column. The size of the operation is automatically calculated for every bad pixel. Of course, no bad pixels are included in any of this calculations, regardless of the structuring element selected. Pixel values above 0 on the Defect Map act as a weight factor for the Mean and Gaussian blur operators.
Bayered images are not directly supported. Anyway, It is possible to modify/build the Defect Map in such a way that it uses pixels that corresponds to a certain color only.

Download links:

Source code:   http://pteam.pixinsight.com/pcldev/defectmap/DefectMap-src-20100504.zip
Windows 32bits: http://pteam.pixinsight.com/pcldev/defectmap/DefectMap-pxm-20100504_win_32.zip
Windows 64bits: http://pteam.pixinsight.com/pcldev/defectmap/DefectMap-pxm-20100504_win_64.zip
Linux 64bits: http://pteam.pixinsight.com/pcldev/defectmap/DefectMap-pxm-20100504_x11_64.tar.gz
(Will include 32bits releases if asked).


Feedback is appreciated :)
« Last Edit: 2010 May 27 09:18:20 by Carlos Milovic »
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: DefectMap process (pre-release)
« Reply #1 on: 2010 May 21 21:37:51 »
I suppose I should port my fixfits utility to PCL. Oy vay, the pain! :)

It finds the hot/cold pixels in a dark frame using a simple sigma limit. It also finds bad columns automatically and fixes both artifacts in raw mono and bayer images. As Juan would say "it works extremely well"  >:D

I haven't ported it to PCL because frankly it's a lot of work for not very much benefit. It's easy to just run the tool from the command line and generate fixed FITS files that can then be calibrated and stacked. Of course usage in PCL has benefits too but not for me :)

It is not open source but it's free. http://www.tungstentech.com/Software/FixFITS/tabid/78/Default.aspx
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: DefectMap process (pre-release)
« Reply #2 on: 2010 May 21 21:39:38 »
I may stole some ideas from there ;)
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: DefectMap process (pre-release)
« Reply #3 on: 2010 May 21 21:41:13 »
Maybe what I should do is incorporate my code in yours. That's probably faster than porting my code to PCL and hooking up all the variables and instances and callbacks and all that.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: DefectMap process (pre-release)
« Reply #4 on: 2010 May 21 21:42:57 »
Which actually begs the question how we do source control on open source parts of PI? Do we have an SVN or CVS server? Should we start a sourceforge project for that?
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: DefectMap process (pre-release)
« Reply #5 on: 2010 May 21 21:51:58 »
Not sure what you mean by source control...
About "sharing" and updating the code, I was thinking on just attaching the changed files to a post in this thread, and then me, Juan, or other PTeam member, will upload it to the server and update the main links at the first post (at least, those with moderator capabilities).

You are highly welcome to jump in :) btw, I'm a big fan of the "divide and conquer" paradigm, so IMHO some features of your program may be best suited in other ad-hoc processes (like bad columns/pixel detection, for example... which is something I found very clever and usefull).
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: DefectMap process (pre-release)
« Reply #6 on: 2010 May 21 21:53:56 »
Ok, you're developing code but you don't know what source control is? That's dangerous :) How do you keep track of code revisions? Don't you use CVS or something like that in your work/play? If you and I were to work together on this module wouldn't you agree it's a good idea we know who made what change? That's what source control allows you to do.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: DefectMap process (pre-release)
« Reply #7 on: 2010 May 21 21:58:58 »
I didn't say I was a good habits programmer :D

You are right. With the exception of Juan (who somehow understands my chaotic code ;) ) I've never worked with other people. We need a system to keep track to all of this... I think I'll have to educate myself into this topic :D
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: DefectMap process (pre-release)
« Reply #8 on: 2010 May 21 22:00:32 »
I'm a professional. I'll whip you into shape :)

At least tell me you keep backups in safe locations, please? Your code is gold my friend. Protect it.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: DefectMap process (pre-release)
« Reply #9 on: 2010 May 21 22:12:32 »
Yes, of course. I keep zipped versions of the code for active projects on a specific pen drive. Also, I have a huge backup in one of my hard drives, with every code I ever wrote with the PCL.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: DefectMap process (pre-release)
« Reply #10 on: 2010 May 21 22:14:21 »
My "control system" is the date suffix :P
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline rom67

  • Newcomer
  • Posts: 1
Re: DefectMap process (pre-release)
« Reply #11 on: 2010 May 22 03:37:38 »
Windows and Linux 64bits are not available

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: DefectMap process (pre-release)
« Reply #12 on: 2010 May 22 09:07:34 »
Sorry!!!! I fixed both links
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: DefectMap process (pre-release)
« Reply #13 on: 2010 May 27 09:18:44 »
Added Windows 32bits compilation.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com