Author Topic: Debayerization needed  (Read 17224 times)

Offline vicent_peris

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 988
    • http://www.astrofoto.es/
Debayerization needed
« on: 2009 April 19 03:48:42 »
Hi folks,

after reading the message from ManoloL in the v1.5 thread, I have seen that, at this point, we are going to have a big problem with DSLR users.

PixInsight doesn't have any routine to debayerize images, apart from the ones in the DCRaw when reading RAW camera files.

The problem is that, when you save calibrated frames in DSS, the images are saved with Bayer matrix! This is a big handicap for the DSLR users, as PixInsight won't debayerize the calibrated frames. So this means that DSLR will be forced to use the registration and integration routines in DSS. In other words, they won't be able to use the registration and integration modules of PI.

This would be solved if we have a module that simply interpolates an RGGB matrix in a TIFF file. I asked Juan to make such a module... but, for the moment, he's very busy with the 1.5 version...

So I ask here for the user community. Could we, as users, begin this small project?

I'm the first interested, as I make some times use of my DSLR. I don't know anything about programming, but my eyes will do a good betatesting work for you. ;)


Best regards,
Vicent.

Offline ManoloL

  • PixInsight Addict
  • ***
  • Posts: 220
Debayerization needed
« Reply #1 on: 2009 April 19 10:24:15 »
Hola Vicent:
En primer lugar lamento no poderte contestar en inglés, por mis limitaciones :oops:  :oops:  :oops:
pero como se que tu me entiendes lo hago en español :lol:  :lol:  :lol:
Tu veras si es preciso realizar alguna breve explicación para los que no sean capaces de entender lo que digo.
Creo, para mi alegría, que no llevas razón. Dss puede dejar grabadas las imágenes calibradas, al menos en formato FITS, que es el que he probado, y supongo que también en TIF, debayerizado:
He realizado los ajustes en los archivos intermedios que recoge esta imagen:



y he pedido que me calibre, registre y apile mi colección de imagenes del triplete de Leo que obtuve el mes pasado ( y que estoy deseoso de poder aplicar en ellas el ajuste de color).
Me he olvidado de la imagen final obtenida, ya la tenia, pero me he puesto a mirar las imágenes calibradas que me ha generado.
Una de ellas era la IMG_9464.cal.fts
Cuando la he intentado abrir con Pixi (v1.4) me ha dado un error de formato desconocido. Pero me acordado que hace unos días en el foro, aunque no soy capaz de escribir si leo el inglés, alguien planteaba este problema, que se resuelve renombrando el archivo para que solo tenga un punto antes de la extensión.
Le he dejado con el nombre IMG_9464cal.fts y ya he podido abrirla.
He manipulado la imagen para ver los píxeles sobre una de la galaxias y me ha salido esto:



La imagen tiene pinta de estar debayerizada, o al menos eso me parece a mi. Eso si ruido tiene a mansalva......
Si realmente es la imagen calibrada ya la podré utilizar, junto con las otras 14, para realizar el registro, apilado y demás con el Pixi.

Saludos.
Saludos.

Manolo L.

Offline vicent_peris

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 988
    • http://www.astrofoto.es/
Debayerization needed
« Reply #2 on: 2009 April 19 10:59:00 »
Hola ManoloL,

¡Muchas gracias!  :D Esto me da una alegría. La verdad es que hace bastante tiempo que no uso el DSS, y creo que cuando yo lo usaba no estaba esa opción.

Entonces ya no hay ningún problema, al menos de momento... ya veremos cuando nos toque poner mano a la calibración.  :wink:

En cuanto a tu imagen, es un tanto extraño el resultado... ¿qué exposición hiciste por toma? Tiene muchísimo ruido, como si hubieras hecho sólo 30 segundos de exposición. Quizás algo no está funcionando bien en la calibración...

Saludos,
Vicent.


=============================


Hello all,

as ManoloL says, seems that latest versions of DSS have the option of saving calibrated AND debayerized images; obviously, I was not aware of that! :)

This solves the problem for the moment, although we need anyway the debayerization routine for attempting any DSLR image calibration.

Regards,
Vicent.

Offline ManoloL

  • PixInsight Addict
  • ***
  • Posts: 220
Debayerization needed
« Reply #3 on: 2009 April 19 11:32:54 »
Quote from: "vicent_peris"

En cuanto a tu imagen, es un tanto extraño el resultado... ¿qué exposición hiciste por toma? Tiene muchísimo ruido, como si hubieras hecho sólo 30 segundos de exposición. Quizás algo no está funcionando bien en la calibración...



Hola Vicent:
La exposición fue de 300 segundos a ISO 400.

El resultado, pendiente de un nuevo procesado incluyendo un tratamiento del ruido mas "fino" y el ajuste de color :lol:  :lol: , está aquí:

http://fotografiaastronomica.com/files/posted_images/user_41_triplleocc2r2.jpg

Saludos
Saludos.

Manolo L.

Offline vicent_peris

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 988
    • http://www.astrofoto.es/
Debayerization needed
« Reply #4 on: 2009 April 19 12:00:45 »
Ese resultado, ¿son sólo 5 minutos de exposición?

A pesar de hacer el balance a mano, te quedó bastante bien. ;)


Vicent.

Offline ManoloL

  • PixInsight Addict
  • ***
  • Posts: 220
Debayerization needed
« Reply #5 on: 2009 April 19 12:18:21 »
Quote from: "vicent_peris"
Ese resultado, ¿son sólo 5 minutos de exposición?

A pesar de hacer el balance a mano, te quedó bastante bien. ;)


Vicent.


Hola de nuevo:
No soy un mago, fueron 15 exposiciones de 5 minutos, con una buena tandas de darks, flats, darkflats, bias......
Lo mas llamativo son unas zonas oscuras alrededor de las galaxias y el aspecto algodonoso que me ha quedado después de un tratamiento del ruido propio de un principiante.

Saludos.
Saludos.

Manolo L.

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Debayerization needed
« Reply #6 on: 2009 April 20 05:37:03 »
Hi Guys,

Because this thread has gone 'all Spanish' I am unable to follow it.

However, Juan is already aware of my efforts to write a script that will deBayer the CMYG colour-filter array found in Meade DSI imagers.

Once I get the bugs (and my knowledge shortfalls) ironed out, I 'could' transform it such that it would handle the RGBG or RGGB (or even the more current RGBL) CFA's

Right now, with over 90% of the script written, I cannot even get the script to perform a seemingly 'mundane' task of 'opening' a file on the HDD - so, unless I can get help solving THAT problem, there is not much chance of me looking at the DSLR issues !!

Cheers,
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Debayerization needed
« Reply #7 on: 2009 April 20 10:22:48 »
Hi,

I've been trying to get a simple PCL module to load in PI but no luck. I intended to try my hand at a debayer module but clearly if I can't even get a modified statistics module to load I'm pretty much stuck before I can get started. I sent Juan emails with error messages and code but haven't heard yet. I'll start a new thread and see if someone here can help me.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Debayerization needed
« Reply #8 on: 2009 April 20 10:31:20 »
Hi Sander,

Well I am a LONG way from PCL modules - and I am having enough hurdles to overcome in getting to grips with the JSR scripts (how I HATE 'OOP' !!!)

Juan has been able to set aside a little time from the release of v1.5 to have a look at my deBayer script, but I am trying NOT to 'get in the way' of that release.

I am sure that once v1.5 is out, then things will settle down a bit - and all of us 'third-partyers' can crack-on with our applets - which, in all honesty, are probably best retained for v1.6 anyway (or, at least, v1.5.1)

Cheers,
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline vicent_peris

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 988
    • http://www.astrofoto.es/
Debayerization needed
« Reply #9 on: 2009 April 20 10:39:14 »
Quote from: "Niall Saunders"
Hi Guys,

Because this thread has gone 'all Spanish' I am unable to follow it.

However, Juan is already aware of my efforts to write a script that will deBayer the CMYG colour-filter array found in Meade DSI imagers.

Once I get the bugs (and my knowledge shortfalls) ironed out, I 'could' transform it such that it would handle the RGBG or RGGB (or even the more current RGBL) CFA's

Right now, with over 90% of the script written, I cannot even get the script to perform a seemingly 'mundane' task of 'opening' a file on the HDD - so, unless I can get help solving THAT problem, there is not much chance of me looking at the DSLR issues !!

Cheers,


Hi Niall,

well, the only think to know is that ManoloL said that DSS has the option to save calibrated AND debayrized images.

If your script can do the work with different matrix, this would be great! Once you have the script, I think there will be guys capable of porting it to the PCL. :)

The only parameter that the user must know is which type of matrix has its camera, isn't it?


Regards,
Vicent.

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Debayerization needed
« Reply #10 on: 2009 April 20 10:39:18 »
Hi Niall,

scripts and modules are mostly release agnostic as long as PCL changes are backwards compatible. So they don't require inclusion in any particular release, we can provide them whenever. A very cool characteristic of PI for sure :)
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Debayerization needed
« Reply #11 on: 2009 April 20 10:44:57 »
Yes, DSS can create registered (and thus debayered) and fully calibrated versions of the light frames. Of course by the the time you're that far you might as well have DSS stack the images as well :)

I think the benefit for a debayer module is for examining single raw OSC frames and for providing a stacking solutions for Linux and Mac. For Windows there's no need, IMO. DSS works great.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Astrocava

  • PixInsight Addict
  • ***
  • Posts: 180
    • Astrocava.com
Debayerization needed
« Reply #12 on: 2009 April 20 12:23:34 »
Hola ManoloL y Vicent!

Acerca de la posibilidad de que el DSS guarde las imágenes calibradas ya debayerizadas creo que no nos soluciona de todo el problema.

Estoy bastante procupado por los píxeles "calientes" que no son fijos y que no son eliminados por la calibración. También es similar lo que nos pasa con el ruido de las CANON. Nuestra última esperanza entonces es la integración para que con un poco de suerte no caigan en la misma posición demasiadas veces y con un algoritmo de rechazo se nos larguen. El problema de realizar la integración con las tomas ya debayerizadas es que esos píxeles calientes se conviernten en general en una mancha (roja, azul o verde) que ocupa ya varios píxeles y el ruido aparece como un granulado cromático y el rechazo encuentra más dificultades.

No sé exactamente cómo trabaja el DSS, pero me gustaría que la integración sólo recurriera a la debayerización al final de todo el proceso.

¿Qué opináis?

Sergio

---
Trying to translate :-)

I think the DSS' option about saving debayered calibrated sub-images don't solve all the problem.

I'm a bit worried with the "hot pixels" that not appear in all subframes and are not suppressed by calibration process. This applies to noise of CANON cameras in a similar way. Our last hope is to see that with integration process that those pixels don't fall in the same position many times and evaporate. The problem of integrating with debayered images is that those pixels evolute to a bigger spot (red, blue or green) and noise appears as a chromatic granulated and rejecting algorithm works worse.

I don't know exactly how DSS works, but I'd like that integration do the debayer in the last stage.
Moonfish ED80 over a Meade LX200GPS 8"

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Debayerization needed
« Reply #13 on: 2009 April 20 12:33:19 »
Hi,

DSS does debayering as late as possible in the process. So it first dark subtracts, then flattens and then finally debayers before stacking. This is simplified as DSS does so much more but that's the essence of it.

Google translated it as follows:

Quote
   
DSS debayering hace lo más tarde posible en el proceso. Por lo que resta oscuro primero, y luego se aplana y finalmente debayers antes de apilarlas. Esto se simplifica como DSS hace mucho más, pero esa es la esencia de la misma.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Astrocava

  • PixInsight Addict
  • ***
  • Posts: 180
    • Astrocava.com
Debayerization needed
« Reply #14 on: 2009 April 20 12:37:32 »
Quote from: "Nocturnal"
Hi,

DSS does debayering as late as possible in the process. So it first dark subtracts, then flattens and then finally debayers before stacking. This is simplified as DSS does so much more but that's the essence of it.



Is it possible to make the debayering only after rejecting pixels? :twisted:
Moonfish ED80 over a Meade LX200GPS 8"