Author Topic: Stripes in background  (Read 16962 times)

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: Stripes in background
« Reply #15 on: 2013 May 07 12:40:41 »
varying the pause between subexposures does not help that much, unless you pause for a very long time. canon cameras sample the temperature at the beginning of the exposure, so your temperature will seem to be low but in fact the sensor heats up again pretty quickly once the shutter is open.

ian, you should shoot a bias frame at the end of your dark and see what the temperature is. the true temperture of your dark will be somewhere between what the dark EXIF says and the bias EXIF.

Offline TobiasLindemann

  • PixInsight Addict
  • ***
  • Posts: 126
    • Trackingstation
Re: Stripes in background
« Reply #16 on: 2013 May 07 12:54:35 »
ahh, ok than my light frames are between 18 an 21 C and the difference between them are not that much I thought.

Offline NGC7789

  • PixInsight Old Hand
  • ****
  • Posts: 391
Re: Stripes in background
« Reply #17 on: 2013 May 07 13:26:13 »
For what it's worth, to generate my dark library on my Canon 50D i just set it going in the garage on a cloudy night shooting say 10 minute darks. I then have a script that uses exiftool to sort the files by temperature into different folders. Eventually over the course of several nights I can accumulate enough at each temperature range (say 5 degree increments) as the ambient changes with the weather. I repeat this each season to cover the range I need and to replace "stale" darks.

Offline IanL

  • PixInsight Addict
  • ***
  • Posts: 116
    • The Imaging Toolbox
Re: Stripes in background
« Reply #18 on: 2013 May 07 14:43:37 »
varying the pause between subexposures does not help that much, unless you pause for a very long time. canon cameras sample the temperature at the beginning of the exposure, so your temperature will seem to be low but in fact the sensor heats up again pretty quickly once the shutter is open.

ian, you should shoot a bias frame at the end of your dark and see what the temperature is. the true temperture of your dark will be somewhere between what the dark EXIF says and the bias EXIF.

Yes I understand that the camera heats up over the duration of the exposure and that the EXIF temperature is only a sample taken at the start of the exposure.  I don't need to re-sample the temperature again at the end of the exposure, since I am starting at the same temperature each time (9C in this case), the exposure length is the same (10 minutes) and the environmental temperature is constant (fairly close to 0C once the camera and fridge have been pre-cooled for a couple of hours).  Since all these factors are equal, the temperature gain from start to end of the exposure is going to be equivalent each time.

You do indeed need to insert a reasonably long pause between exposures.  In my case a 2 hour initial cool down, followed by a repeating cycle of one 10 minute dark followed by a 20 minute pause produces a constant EXIF temperature of 9C at the start of each exposure.  After about 8 or 9 hours, the temperature starts to rise as the outer layer of the ice packs melt (they are solid in the core still, but not as effective at cooling).  This is not an issue since I just set up APT with the relevant plan (bias exposure with a 2 hour pause, then a large number of 10 minute exposures with 20 minute pauses) and walk away.

Of course this is not as satisfactory as having a set point cooled camera, as the rate of dark current accumulation will increase as the exposure goes on, but this mimics the behaviour of light frames which have the same problem.  Ultimately the dark scaling process tries to match the dark frame and the light frame by analysing the noise in each, so to my mind (perhaps erroneously) the best compromise is to start with a master dark frame that is created from a large number of matching frames rather than a bunch of darks that have been taken under different conditions.

The other wrinkle is the on-camera processing over which we have no control.  The behaviour seems to be that in the first minute or two of exposure, the dark current decreases (when it should increase) but the dark current noise increases.  Thereafter the behaviour seems more normal as the dark current and noise both increase fairly linearly with time.  This is why I aim for 10 minute exposures rather than very short ones, in the hope that the dark scaling process has a better chance of succeeding.

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: Stripes in background
« Reply #19 on: 2013 May 07 14:52:53 »
i'm not saying that your temperatures do not match... they do. what i am getting at is that your end temperature may be much higher than it seems, and so the benefit of adding pauses may not be as great as you think. what if the end temperature is 20C? what if it was at 20C for 90% of that 10 minute exposure and 9C only for the beginning? what if the camera only reached 22C under continuous 10 minute exposures?

you really have to trade off only capturing photons for 10 out of every 30 minute period against the perceived temperature benefit. you're taking a huge hit in potential SNR increase so if it were me i'd want to prove that the result is superior.



Offline NGC7789

  • PixInsight Old Hand
  • ****
  • Posts: 391
Re: Stripes in background
« Reply #20 on: 2013 May 07 19:21:24 »
It would seem to me that the best temperature match would be to shoot the darks with the same duration and interval as the lights. That way the temperature recorded by the camera at the beginning would be the same for both.

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: Stripes in background
« Reply #21 on: 2013 May 07 20:43:57 »
It would seem to me that the best temperature match would be to shoot the darks with the same duration and interval as the lights. That way the temperature recorded by the camera at the beginning would be the same for both.

yes, ideally you shoot the darks under the same exact conditions as the lights, same ambient temperature, same camera cooling (if any) and same duration and interval between shots.

Offline Jan

  • PixInsight Enthusiast
  • **
  • Posts: 82
    • Astrofotky.cz
Re: Stripes in background
« Reply #22 on: 2013 May 08 01:12:10 »
Interresting to see how this topic did grow within not even one day. Anyway I have to say, that:

- However we shoot dark frames with our DSLR ´s, it will never be precise enough to allow totally perfect calibration. There are too many inputs into it like change of temperature during the night etc. As DSLR users, WE HAVE TO LIVE WITH IT unless we invest money into colled DSLR.. Myself I do darks after lightframes, when outside air temperature is stabilized and also the camera is thermally stabilised, I do not see anything more I could do without unacceptable hassles, results are ok but never perfect.

- To solve your problem Tobias, use ditherring as I described in my first reaction. If you can buy device which is able to do that, you can just SLIGHTLY move your telescope by hand after every light acquisition, any direction and just few pixels (set slow speed on your hand controller). It is doable, a friend of mine was hand-ditherring at least one year with good results. Action takes few seconds and I guarantee you will not see stripes on your next combined master file.

My experience is, that since I was using ditherring technique, there is not trace of the artifacts you are presenting up there. My photos are here http://astrofotky.cz/~Oko1/

Offline TobiasLindemann

  • PixInsight Addict
  • ***
  • Posts: 126
    • Trackingstation
Re: Stripes in background
« Reply #23 on: 2013 May 08 03:24:29 »
I want to thank you all for your help. Especially Jan and Rüdiger helped me much.

For others who have the same problem here es a summery which will solve my problem:
-dithering
-more bias (>100)
-use another exif-viewer to read out the temperature of the camera (exiftools)
-better temperature-match of light and dark frames



The only open question is: when should I set "raw" as a input hint? Only at the calibration of the light frames, in all calibration processes, or in calibration and integration processes?

Greetings Tobias

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: Stripes in background
« Reply #24 on: 2013 May 08 09:24:10 »
just for calibration, or whenever the CR2 file is going to be opened for the first time by pixinsight.

after that the files will be in fits format and the raw format hint does not mean anything.

Offline TobiasLindemann

  • PixInsight Addict
  • ***
  • Posts: 126
    • Trackingstation
Re: Stripes in background
« Reply #25 on: 2013 May 08 10:09:47 »
thanks, sounds logical  :)

by the way the right hint is: raw cfa
IMO only "raw" does nothing
« Last Edit: 2013 May 08 10:33:40 by TobiasLindemann »

Offline TobiasLindemann

  • PixInsight Addict
  • ***
  • Posts: 126
    • Trackingstation
Re: Stripes in background
« Reply #26 on: 2013 May 08 18:03:16 »
So, now I have created a much better masterbias with 270 frames and much better matching darks. The result is not perfect but much better than before:

http://www.tracking-station.de/zeigl/stripesoldnew.JPG larger screenshot
Unfortunately I have not much darks that matches the lights, so far. I think this is the problem why the stripes are still there. I think it is really difficult to get the right darks and consistent lights too. So dithering is the way to go.
Thank you all for your help  :)

Offline 3p0

  • Newcomer
  • Posts: 1
Re: Stripes in background
« Reply #27 on: 2013 September 11 07:23:45 »
just adding my 2 cents worth...

I'm fighting with the same issue (rgb streaks in long exposures). The best solution is indeed getting good calibration frames.

I think there was a rule for taking darks that says you should have 5 times as much dark exposures as light ones. That results in good statistical reduction in the noise.

So for a total of 6x10min of light frames you'd need 30x10min darks. And that's a minimum.

Another thing that should help with the colour streaks is aligning your guide scope axis exactly with the photography axis. Even half a degree of difference in the axes will result in some distortion in the image. Also the polar alignment should be good, and everything in your rig should be really tight (don't leave any room for flexure).

I live in Finland, so most of the year it's cold enough not to bother with the darks, as the temperature gets down to -30°C on good nights, and there is no heat noise at all in regular DSLR images (even at 30min exposures!)

Clear skies!

Offline TobiasLindemann

  • PixInsight Addict
  • ***
  • Posts: 126
    • Trackingstation
Re: Stripes in background
« Reply #28 on: 2013 September 11 10:17:27 »
Hi 3pO,

you are absolutely right, but it costs a lot ot time to take so many darks.
Now I have the solution for this problem:
I take the darks at other nights, or days at the balkony or in the fridge. Then I sort the darks according to the temperature in a dark library. To do this automaticly a friend of mine has created a batch file for exiftool that sorts the darks in different folders. Another batch gives a temperature-list of the lights. So you you have always the right darks for your lights. Even in the same night the lights haven´t the same temperature during the night. So I take the same quantity of darks with the same temperature as the lights, or a mulitple of it. IMO now you can do all darks in one pot and make the masterdark. With this masterdark you can calibrate all lights. This procedure works very good for me. You can download the sorting bach at the following link:
http://www.tracking-station.de/Tempsort/_5exiftool_temp_sort_rename_move.Bat
What you also need is this the following file and of course exiftool:
http://www.tracking-station.de/Tempsort/_0format.fmt
The batch for the temperature-list you can download here:
http://www.tracking-station.de/Tempsort/_0exiftool_templist.Bat
What you have to do is, to copy all files in the directory of exiftool and simply start the files. If you have distrust in my files you can have a look at them in the editor. It is all no rocket science and easy to understand.

Greetings
Tobias