Crowd Sourced Astro Images (Crowd Imaging or CI)

[MortenBalling]

@alvinjamur

Thanks

I live in a very light polluted city center, with a lot of clouds and bright summer nights, so I have to find alternative ways of collecting data. Meanwhile I keep a single project running on the telescope (currently HH555) shot through both an UHC and a red kodak visual filter. Not optimal  Gotta have those Ha, SII and OIII filters soon.

Here's a link to M45:

http://www.astrobin.com/120897/

I deliberately didn't stretch the image, so the best way of viewing it is to follow the download description, load it into PixInsight, stretch and zoom.

cs

Morten

[marekc]

I have to admit, I'm really quite amazed by this approach to imaging! The results that MortenBalling has posted look really good to me.

When I looked at the M31 image, I checked the credits to see if my image was in there... it wasn't, IIRC. That's neither here nor there, but it made me think about something: Images like these may have a bit of a `Google Earth effect' for some viewers. When people first use Google Earth, they immediately try to go find their house. When people go up in an airplane, they say "I can see my house from here!" When a crowd-sourced image gets posted, people who've posted their own individual images will probably check the credits, to see if their work is in there. Interesting!

This has also made me think about licenses and copyright more than I had before. I'll have to go check the default CC license setting on my (exceedingly modest) little imaging blog. Maybe I didn't allow people to modify the images, or something like that. I never thought much about that until I saw this thread.

What a neat technique! I don't know if there enough M33 images out there to make a good M33, but I'd love to see that. It's a personal favorite of mine.

It also occurs to me that crowd-sourced images are kind of like mineral deposits or oil fields. All of the easy stuff will be gone through fairly quickly. The supergene deposits and big surface anticlines were all discovered long ago. In some cases, those mining districts and oil fields still yield commodities today. That's kind of like if someone decided to apply really advanced PI processing to the crowd-sourced M31 data, to try and squeeze a slightly better result out of it. The rarer deep-sky objects will be like the more recently-discovered resources. Yes, they're there, and we can find them and extract them, but it's much harder to do so. Like trying to make a crowd-sourced image of some obscure Sharpless object that has very few publicly posted images. It could, technically, be done, but it would be very hard to do. In the case of minerals and oil, sufficient demand/scarcity/high price can make a `tough' deposit worth going after - there's a profit motive, at least at certain times. But with crowd-sourced imaging, the lack of a profit motive might, I'd guess, provide a fairly strong dis-incentive against trying to do the tough stuff via publicly-posted images. At a certain point, it's easier to just buy one's own gear, go on a road trip to a dark site, and shoot that sucker oneself.

It's like a little economics lesson! What a fascinating thing - I'm glad someone did this!

- Marek

[MortenBalling]

@Marekc

Interesting points.

I'm sorry I didn't use your images. As mentioned earlier, I started doing these images as an experiment, mainly to test the concept, using whatever images I could find, so I probably did anyhow. I ended up with something I called Andromeda500 with about 550 copyrighted images, in 200 Mpix, and went "WOW!". I love zooming into large images and finding details. With Andromeda, I also just gazed into the field trying to see if I could see the Big Bang (not really).

Anyways, I was astonished by the Crowd Imaging potential, and sad that I couldn't share it with anyone. So much that I started looking for loopholes in the copyright laws (they are actually there, if the result is research related). Then I realized that the quality of the separate images didn't seem to matter that much to the final result. The main thing is lots of data. Of course I got better results with sharp, deep highres images, but as I mentioned, more than 50 random images produce very similar results. That was the time I started to look for Creative Commons images, and considered starting all over.

I picked some of the most popular objects (M31, M42, M45 etc.) as we've all tried imaging those. Therefore there were many images, and also the results are easier to evaluate for anyone. Recently I was looking at SkySafari trying to figure out what to shoot one night, and saw The Draco Dwarf Galaxy. That framed perfectly with my rig, so I shot that three nights in a row. On the final stack I could see a fuzzy, but to be honest it was probably noise and a non-flat field. So I went to the god old interweb, and found all the images of Draco Dwarf that I could find. The result was the most detailed image of the very faint galaxy I've ever seen. Again I had this sad feeling that more people should see it, but I'm afraid I'll have to settle with the fact, that at least I've seen something nobody else has. Pretty mindbogling

Btw. There are a lot of M33s out there, so knock yourself out and try it. I've done one, and it's a beautiful galaxy! I might also do one with CC images in the future, but I'd rather try some more exotic objects next.

In the long run I hope this idea catches on. Georg's link shows, that I'm not the only one who thought about it, even though I started before those guys. The JNER1 collaboration I participated in, was an idea I had, because once again I had spent many nights shooting that with no result, before I realized that I had to work with others, to capture enough photons. I later found out, that others had done collaborations as well. The Crowd Imaging was just an extrapolation of collaborations.

What i really hope is that people start sharing their data. Just 100 amateur astronomers with newtons, refractors, RCs and SCTs spending one night on the same object would be awesome, but very hard to organize. Searching the web for useable images is much more achievable. Also a dedicated place to upload data, like Astrobin, with easier search routines would be cool. There are a lot to be found out there, but searching for images is pretty time consuming.

Finally, the method leaves a lot of room for improvement. So far I weigh all images equally. Both because it's "easy", but also to try to be fair to the people spending their nights catching faint light. They all work equally hard. But to be honest, some pictures are less noisy, sharp etc. than others. Fair enough. I know how hard this is. A better way of evaluating and weighing images would still be a major improvement. I've tried, but PI (or me?) gets pretty confused, and I haven't tried making scripts (yet).

Cs

Morten

[georg.viehoever]

...
What i really hope is that people start sharing their data. Just 100 amateur astronomers with newtons, refractors, RCs and SCTs spending one night on the same object would be awesome, but very hard to organize. Searching the web for useable images is much more achievable. Also a dedicated place to upload data, like Astrobin, with easier search routines would be cool. There are a lot to be found out there, but searching for images is pretty time consuming.
...

That also sounds like an interesting idea. Especially if people would share their RAW, or at least their linear (i.e. calibrated+stacked) data. This could more easily be stacked than already processed images. I wonder if platforms such as astrobin.com would provide the necessary functionality - I never looked into this.
Georg

[MortenBalling]

@Georg

I think Salvatore is already up to his neck in files. It's an amazing site, and a huge job that he does (for free). Also Fits file sets are a lot heavier on the server than jpg's, but Moore's Law works for us.

Linear data would be very nice, but nonlinear are actually not that bad for this technique. At least for the visual side of things. I'm trying to figure out how to analyze the jpg files, to evaluate the amount of useable information stored in each image, and hopefully it suddenly pops up in my head. Things normally do.

In the meantime I would again like to urge people to use Creative Commons licenses for their astro images if they don't plan on selling them.

Morten 

[pfile]

well astrobin can and does host fits, for a fee. you can publish them publicly or keep them private…

rob

[MortenBalling]

@Marek

Here's a M33 for you:

http://www.astrobin.com/121906/

Morten

[marekc]

Thanks, Morten!

That's a nice-looking image    It's got a lot of resolved stars, plus HII regions and the faint outer portions of the galaxy. Good `ol M33!

- Marek

[MortenBalling]

A little more info.

The images used for this technique doesn't have to be perfect. To illustrate that I've recently made a CI of The Rosette Nebula.

Here is a Youtube clip with the different images used:

http://youtu.be/KbYGnDrj43Q

And here is a detail from the final image:



Image Credits (Thanks for sharing!):

Adam Evans, aknotwot, Andolfato, Andrea Pistocchini, Andreas Fink, Asaf Braverman, AstroGG, ASTROIDF, Ben Browning, Carsten Frenzl, Chris Lasley, Chris Madson, Claustonberry, Fernando Fogel, Francescodib, FranckIM06, Fred Locklear, Jan Curtis, Janos Barabas, Javier R., John Lanoue, john.purvis, Luc Jamet, Luis Argerich, Luis Martinez Martin, Marcelo Domingues, Mauro Narduzzi, Michael Karrer, Mike Markiw, Mike Markiw, milosz, neptun, nicoairbus, NicolasP, pascvale13, Pat Gaines, Paul T., petersdome-observer, pfile, Phil Hosey, Phil Hosey, Ralph W, Ram Viswanathan, Rhett Herring, Richie Jarvis, s58y, Salvatore Iovene, Salvopa, Serge, Shekhar Phatak, Stan McQueen, Stephen Rahn, Steve Yan, stevebryson, stseiya, Tim.

Full version: http://www.astrobin.com/122809/0/

Image @100%: http://www.astrobin.com/full/122809/0/?real=&mod=

Another thing I've been wanting test the method for is Deep Field. Because the final image stack has such high SNR, it can be stretched enormously. A negative luminance of NGC3628 shows QSOs out to (at least) 10-12 billion lightyears away (Z=2,4):



Image Credits (Thanks for sharing!

Adam Evans, Alvinillo, Angel Requena, Anton, Armelle & Eric, AstroGabe, AstroGG, Astroluc63, Ben Browning, Bob Familiar, budman1961, Cano Vääri, Carsten Frenzl, chripell, Cobbler, Cody Garges, Creedence, David L Milligan, Eduardo Mariño, Eric Gorski, Ferran Ginebrosa, Flavastro, fragro, FranckIM06, Fred Locklear, Fredrik Ödling, Fryns, geco71, Gerardo Blanco, harbinjer, Heiko Günther, Hewholooks, jdiwnab, John Bowles, Jorge A. Loffler, jpstanley, Juan Lozano, Jürgen Kemmerer, Jussi Kantola, Ken_Lord, Ljubinko Jovanovic, Luca Argalia, M.W.Hoy, Marc Van Norden, Marcelo Domingues, Matthew, Maxvlt, Miodrag Sekulic, Morten Balling, NicolasP, PaulHutchinson, Pavel (sypai) Syrin, pbkwee, pbkwee, Pete Collins, Peter Williamson, pfile, Phillip Seeber, Ram Viswanathan, Rhett Herring, Richie Jarvis, RIKY, Rob Glover, Roberto Ferrero, s58y, Salvatore Iovene, Salvopa, samuele, Sendell, Serge, Shane Poage, Stephen Rahn, Steve Elliott, Steve Yan, stevebryson, Surfus_1980, theilr, Tim, Tom Harrison, Vincent Bhm, Wayne Young, zemt-fr.

[Josh Lake]

Morten, I'm a bit late to the party here but I am positively blown away! You've got to take this technique on the road to astronomy conferences!

I strongly believe that astrophotgraphers these days are far more willing to share, especially if their data sets are being credited and used in something excellent like this.

I'd like to get involved, I'd like to spread the word, and I'd love to process some of these beautiful data sets. I'm going to go back and read your article and try it for myself, but are you also willing to post some of the merged data sets?

I'm a moderator over at Reddit's /r/astrophotography forum and we tried something like this on M42 with bad results. But with your technique, the sky's the limit. Would you allow me to link to your post here and your work on Astrobin? Or would you like to post something yourself? With a community of 10's of thousands of imagers, some with really excellent data sets, I think there might be a gold mine in the making.

Thanks so much for posting, I'm bursting with the excitement and possibilities!

[MortenBalling]

Hi Josh

Thanks

First of all: You are more than welcome to spread the word. That was my idea about this thread. I've tried sending some images to APOD and to Phil Plait (Bad Astronomy), not to brag, but to try to spread the idea, but I don't think the really understand the concept (yet). The more people getting involved the merrier! (The sky is big).

You can link to anything I do, and use the images as you like. I think crediting the participants are important with this (currently looking at incorporating credits in the bottom of the images). All my CI images are made with Creative Commons files, and so they are automatically also CC. Also I encourage everyone interested to try the technique themselves. The biggest problem is finding enough images, but for testing and practising, you can use copyrighted material as well. Just don't publish those. I personally use that a lot, and I've seen things you people wouldn't believe

If I can assist you in any way please let me know. If you send me a PM with you email, I can send you some of the raw stacks, so that you can try to process them.

I've been trying to spread the word about the idea myself. In Denmark (where I live), I've made a few lectures to some of the astronomical societies, and people are starting to, at lest, do collaborations.

The idea came to life after I suggested a collaboration of the planetary nebula JNER1 here in Denmark. I didn't know about collaborations back then, but the idea seemed pretty straight forward. To bring out details and a deep field, you need lots of exposure time, and parallax is not a problem with so distant objects as we image. At the same time I noticed that we're all shooting the same Messier objects over and over, so I started thinking about how much exposure time that added up to. A lot!!!

The first tests I did looked very promising but because the images were so differently framed, I had problems with the edges of the field, unless I cropped a lot. That brought out the idea of making a sort of flat frame with the coverage of each image, and using that to make a simple flat compensation (by division).

Another thing that I really like about CI is that it's kind of an opposite to something I've seen a lot of in the astro photo community. Everybody wants to be the best. Better than Nasa, better than Robert Gendler, best at Astrobin and so forth. I think that's a shame. When I started out with a DSLR on a tripod, I was amazed at what you could do. Once you get better you start to make challenges for yourself, and the natural reaction is to be proud if you succeed and wanting to show it to everyone. Problem is that I think most of us forgot that this is mainly something we do for fun (and searching for QSOs ofcurz ). It is great if somebody else can have an experience with what we create with the telescopes/cameras, but once you've seen 100+ M42's they start to look the same. If you add them up they suddenly become something special, and everyone participating can have a feeling of having been involved in the final result. Together we're strong, and all that...

Best Regards

Morten

[pfile]

I personally use that a lot, and I've seen things you people wouldn't believe

like attack ships on fire off the shoulder of orion?

rob

[MortenBalling]

Yep! That and much more...