ALHAMBRA Survey, field 7

[vicent_peris]

Hi all,

I have just uploaded a set of images of the ALHAMBRA Survey. This is a cosmological survey containing about 600.000 galaxies down to mag 26.5 in four square degrees of the sky, acquired with the 3.5 meter Zeiss telescope at Calar Alto Observatory. The data set is composed of 23 images which have been registered and flux equalized. 20 out of the 23 are midband image acquired with 30 nm wide filters; these images have been acquired with LAICA camera. This camera has four, 16 megapixel sensors located at the prime (f3.9) focus, and these images correspond to the ones acquired by one of these four sensors. The filter system is described in page 5 of this PDF article. The other three images have been acquired with Omega2000, a 4 megapixel infrared camera, with the standard wideband J, H and K filters.

The challenge here is to combine a RGB image. Good luck!
Vicent.

[Josh Lake]

I'm very interested in trying to work with this kind of data. Are the files on the PixInsight files server? I still haven't figured out how sharing is done with this system... do I have to ask you to share me in or can you post it somewhere for anyone to grab it?

[Alejandro Tombolini]

Nice challenge Vicent, I have no idea how to start but I'll make a try. The data is very interesting.

Saludos. Alejandro. 

[Juan Conejero]

Hi Josh,

Are the files on the PixInsight files server? I still haven't figured out how sharing is done with this system

They are on the Forum Shared Files (FSF) repository.  To share your files, just create a new folder on FSF and upload them to it. Everybody has read/write permissions on FSF (with the exception of the folders where the original data have been stored, which I've protected by removing the write permission).

I am still trying to configure ajaxplorer to improve the way we can use the PixInsight Files service. It has severe limitations to share files with all users, which I've confirmed with the author. Despite these limitations, ajaxplorer is the best online file management package I've tested. I think we can use it with FSF for now.

Something that I am investigating now is the possibility to allow all PI users to use PixInsight Files automatically, without having to email us. This requires writing an ajaxplorer plugin to provide external user authentication. There are some technical problems but I am working on it. Once I have this sorted out I'll announce PixInsight Files officially to all PI users, and it will be available from the main website.

[vicent_peris]

Hi Josh,

Are the files on the PixInsight files server? I still haven't figured out how sharing is done with this system

They are on the Forum Shared Files (FSF) repository.  To share your files, just create a new folder on FSF and upload them to it. Everybody has read/write permissions on FSF (with the exception of the folders where the original data have been stored, which I've protected by removing the write permission).

I am still trying to configure ajaxplorer to improve the way we can use the PixInsight Files service. It has severe limitations to share files with all users, which I've confirmed with the author. Despite these limitations, ajaxplorer is the best online file management package I've tested. I think we can use it with FSF for now.

Something that I am investigating now is the possibility to allow all PI users to use PixInsight Files automatically, without having to email us. This requires writing an ajaxplorer plugin to provide external user authentication. There are some technical problems but I am working on it. Once I have this sorted out I'll announce PixInsight Files officially to all PI users, and it will be available from the main website.

It would be very nice to access the files from the navigator inside PixInsight and drag and drop the files to the workspace. 


V.

[vicent_peris]

Nice challenge Vicent, I have no idea how to start but I'll make a try. The data is very interesting.

Saludos. Alejandro.

The key is in PixelMath. Build your own RGB palette, it's only a matter of multiplying and adding images. A lot of different palettes can be constructed from different documentary arguments.

V.

[oldwexi]

Vincent!
Thanks for uploading such great data to which i never would have access to.
Great idea.

A question:
I let the 20 narrowband images just run through PI-animation and got the naive impression
a few images are not 100% calibrated. on some images the borders of the 4 cameras can be seen
and for example the image with the 536nm to 567nm filter looks like flat fielding
did not succeed or is missing? Do you have the flatfields per filter also available?
Or, am i expecting too much cleanness from such deep data?

Thanks for giving me the possibility to play with this data!

Aloha
Gerald

[vicent_peris]

Vincent!
Thanks for uploading such great data to which i never would have access to.
Great idea.

A question:
I let the 20 narrowband images just run through PI-animation and got the naive impression
a few images are not 100% calibrated. on some images the borders of the 4 cameras can be seen
and for example the image with the 536nm to 567nm filter looks like flat fielding
did not succeed or is missing? Do you have the flatfields per filter also available?
Or, am i expecting too much cleanness from such deep data?

Thanks for giving me the possibility to play with this data!

Aloha
Gerald

Hi Gerald,

Yes, that's true. LAICA is not a nice camera, and it has problems with flatfielding. All the images have been calibrated by the Survey staff. At the end, when you combine the 20 filters, these problem are greatly reduced, although you'll get a very noisy and patchy background.

Regards,
Vicent.

[Alejandro Tombolini]

At the end, when you combine the 20 filters, these problem are greatly reduced, although you'll get a very noisy and patchy background.
Regards,
Vicent.

Thats right. This is being an extremely interesting challenge...

[Alejandro Tombolini]

Hi,

The image 644 has some background information that does not seem to be a pattern of the camera and I assumed that they are not clouds, perhaps a wrong assumption. If so, the following is not all valid. 

What I did was a combination, only the sum, of:

B = 365 +396 +427 +458 +489
G = 520 +551 +582 +613
R = 644 +675 +706 +737 +768 +799 +830 +861 +892 +923 +954 + H + K + J

For the combined image:
ABE
DBE
HT (STF)
ACDNR to Crominance
SCNR to green

After that I separated the stars and created a mask from the background image 644.
With the mask protecting the background in the main image applied InterChannelCurves with target in CIE L and  CIE C and the reference channel in green.
After that, in the same image without stars:
LHE
CurvesTransformation
CurvesTransformation to stars with Lmask.
PixelMath to add stars again
Erosion (a little bit)
SCNR

The result was too noise...


Crop 1:1


Then I built a luminance with the integration of all images. No rejection
MMT
HT (STF)
LHE
And combine with the image.
ADCNR to Crominance
 
I tried to "repair" the defects in the bottom left corner with masks made over the ones that had the defects (520 and 635) and applying HT

The final:


Crop 1:1


Saludos. Alejandro.

[Josh Lake]

Well done, Alejandro! I have been waiting to get through my work projects so that I could play with this data over the weekend. You have set a nice, high bar to start.   

[Joe DePasquale]

Hi all,

 What a great data set to practice with! Thanks Vicent!

 For my combination, I wanted to get a good color balance.  Technically speaking, this should be a very red image given the filters used, but rather than constrain the filters to their associated colors, I spread them across the RGB channels evenly to give a more natural color balance to the image.  I did end up using PixelMath for this with the following parameters:

R=830+861+892+923+954+J+H+K
G=613+644+675+706+737+768+799
B=365+396+427+458+489+520+551+582

 After the RGB combination, I did some background neutralization as well as color calibration using a selection of spiral galaxies (and the preview aggregator script) as my white reference.  After a round of ABE subtraction, I used Histogram Transformation to de-linearize the image.  I then tried the old trick of using the luminance channel in LRGB for chromatic noise reduction, and color saturation followed by SCNR to remove the green cast.  A slight curves adjustment got me to a point where I wasn't satisfied with how the faint sources were showing up.  I then extracted a new lum channel and overlaid as an inverted mask, to protect the cores of the bright stars and did another round of Histogram Transformation to bring up the faint stuff.  This last step increased the noise a good bit, so a subtle use of ATrousWaveletTransform helped with that.  I also tried to reduce some background variation with another round of ABE to less than desired effect (I still see some variations).  One final curves tweak and here it is. 



Thanks,
Joe
 

[Josh Lake]

Ah, a wild imaging expert appears! 

Beautiful job, Joe, and sound logic for the combination in PixelMath. I am going to take a crack at this with similar parameters. What was your rationale for picking the specific galaxies for white reference that you chose?

[Joe DePasquale]

Ah, a wild imaging expert appears! 

Beautiful job, Joe, and sound logic for the combination in PixelMath. I am going to take a crack at this with similar parameters. What was your rationale for picking the specific galaxies for white reference that you chose?

Thanks, Josh!  To take this one step further, I would've brought it into Photoshop to clean up those artifacts in the bright stars, but I wanted to keep it 100% PixInsight. 

The white reference galaxies were just chosen quickly by eye, I was looking mainly for spirals that appeared almost white in the auto stf.  There's a large spiral, almost edge on in the top third of the image, right of center that was a good addition to the references. 

[Juan Conejero]

Very nice result, Joe!

I would've brought it into Photoshop to clean up those artifacts in the bright stars, but I wanted to keep it 100% PixInsight.

Actually, you can achieve both things---I mean cleaning up and 100% PixInsight Do you know PixInsight's CloneStamp tool?