Author Topic: Bad Photos or bad workflow? Need help.  (Read 4520 times)

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #30 on: 2018 May 01 14:05:50 »
O.K., thanks.

Here is my result after integrating the lights with subtracting all masters, Automatic Background Extraction, Background Neutralisation, Colour Calibration, Histogram Transformation, HDR Multiscale Transform and Curves Transformation.

In my opinion it is not nice regarding 5 hours exposure. What do you think?

Offline Greg Schwimer

  • PixInsight Addict
  • ***
  • Posts: 210
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #31 on: 2018 May 01 14:27:22 »
Post a fresh link to the unprocessed master light frame so we can have a look.

For comparison, this is a 2 year old master light of the same object. Total time was 200 minutes, exposures were 1200s each - way longer than necessary. Since the image is from the same camera (A7S) it might be useful for comparison purposes.

https://www.dropbox.com/s/glq3ak1upagwwmh/light-BINNING_1.xisf?dl=0

Note that if you open that frame and look at the Statistics process you'll see 100% count for the R, G, and B channels. If those numbers are less than that for your masters you'll want to find out why. Most likely something went wrong during calibration.

Processed version is below, also from 2 years ago.
- Greg
Scottsdale, Arizona, USA

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #32 on: 2018 May 02 00:13:24 »
Hi Greg,

this picture is hundred times better than mine! Wow!

Here is the new link: http://gofile.me/3WOrk/HaHC1jRfL

The problem is, that the deviations from picture to picture is great, so that StarAlignment did not work.
So after ImageCalibration (with all masters) and then debayering, I had to use DynamicAlignment (before I could integrate), which needs a long time.

So if you do a new ImageCalibration or Debayer, I could do the DynamicAlignment, in order you will not spend too much time.
Or you align only 8 pictures which is the minimum.

Thanks in advance for this great offer!

Armin

Offline Greg Schwimer

  • PixInsight Addict
  • ***
  • Posts: 210
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #33 on: 2018 May 02 01:39:05 »
I downloaded the file integration.xisf. First thing I noticed is the total pixel count is a bit less than the full sensor size. Is this image cropped? Statistics don't reveal any clipping <- this is good. An unlinked STF gives a decent result. I see the remnant of a satellite or airplane path from left to right, passing just in front of M51. If you can't get rid of it completely during integration or processing maybe consider dropping the frame that is contributing it. At first glance I'd say the integration looks pretty good.

I did a quick and partial processing on it to see what I could get at easily. I was able to get a decent result in not too much time. A much better result can be achieved for sure.  Screen capture below. Here's what I did:

 - DBE ( several passes )
 - Photometric Color Calibration
 - TGV Denoise w/ mask & support for L and chrominance
 - stetched w/ Histo Transform
 - Extracted L mask, apply inverted
 - Curves Transformation for contrast
 - HDR Multiscale Transform (mask not inverted here)
 - Curves Trans for saturation & final tweak

I've attached the process container as well.

I think you have something decent to work with here. More data is always nice to have if you have the time for it.



- Greg
Scottsdale, Arizona, USA

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #34 on: 2018 May 02 02:35:58 »
You processed this picture out of my integration.xisf ???!!!Wooooooooow !!!!!  Very nice!!!!!

Yes, I did a dynamic crop.

And yes, there was a satellite passing M51. I thought this will be processed out through the integrating process?
Then I will drop it.

I think I´ve to watch some tutorials, because some processes you did, I don´t know by now.

What do you mean with "more data is always nice ..."? More exposures?

I used Curves Transformation, too. But what do you mean with CT for contrast and CT for saturation?
Please, maybe you could send a screenshot of the CT or is it implemented in the process container?

Thanks a lot!!

Regards,
Armin

Offline bulrichl

  • PixInsight Old Hand
  • ****
  • Posts: 437
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #35 on: 2018 May 02 02:41:54 »
Hi Armin,

without doubt, Greg's tips are very valuable for the post-processing steps. However, you are still having a severe problem with your calibration, and this must be resolved first!

I had a look at your master_bias_new.xisf, master_superbias_new.xisf, master_dark_new.xisf and master_flat_new.xisf. MasterBias, Superbias and MasterFlat are OK now, but the MasterDark isn't, it is severely clipped. Once again: this seems to be a pre-calibrated MasterDark, obtained by either calibrating the bias frames and integrating the calibrated bias frames or by calibrating the integrated bias frames. Both of these approaches are not recommended!

You should delete your MasterDark (master_dark_new.xisf) and build a new one in this way:
Simply integrate the dark frames with the settings recommended in https://www.pixinsight.com/tutorials/master-frames/ . Finished, that's it.

Then proceed with these calibration master files performing the light frame calibration (settings: see https://pixinsight.com/forum/index.php?topic=11968 , image 2, left hand side or image 2, right hand side.

I guess the flat field correction can be improved yet, by increasing the exposure time for the flat frames by a factor of 3.5 to 4.0. However, whereas the modified procedure for building the MasterDark is an error correction, this is an optimization.

Good luck!

Bernd

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #36 on: 2018 May 02 03:05:37 »
Hi Bernd,

I really really did simply an integration of the dark frames. With exactly the same settings I did for the integration of the bias frames.
I did it immediately after integrating the bias frames.  :'(
And I deleted the old one before. :'(
Could it be, that different ambient temperature of the darks and lights generate this problem? Because I generated the darks two days after the lights.

For the Image Calibration of the light frames I see one little difference to my settings: I did not enable "Evaluate noise" in the Output Files section.
I will change this.
And I enabled "Calibrate" and "Optimize" in the Master Dark section, because when I remember right, you recommended it in your tutorial when using DSLR without temperature control and have flats with very short exposures.

Flats: I have chosen the exposure time for the flats in that way, that the histogram is in the middle.  :-\

Armin



Offline bulrichl

  • PixInsight Old Hand
  • ****
  • Posts: 437
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #37 on: 2018 May 02 03:57:04 »
Hi Armin,

Quote
I really really did simply an integration of the dark frames. With exactly the same settings I did for the integration of the bias frames.
I did it immediately after integrating the bias frames.  :'(
And I deleted the old one before. :'(
Oha, I'm sorry, now this is my fault - your Master Dark (master_dark_new.xisf) isn't clipped at all. I apologize for my overhasty statement.

Quote
For the Image Calibration of the light frames I see one little difference to my settings: I did not enable "Evaluate noise" in the Output Files section.
I will change this.
This is not necessary, it will not change anything.

Quote
And I enabled "Calibrate" and "Optimize" in the Master Dark section, because when I remember right, you recommended it in your tutorial when using DSLR without temperature control and have flats with very short exposures.
As I don't detect any "amp glow" in your MasterDark, I guess this is the way to be preferred as well.

Quote
Flats: I have chosen the exposure time for the flats in that way, that the histogram is in the middle.  :-\
Yes, but in the middle of what? The middle of the camera's histogram? DSLR cameras show an already stretched histogram, so this is not correct. You could take a series of frames with different exposure time (bracketed exposure?) and view the image statistics or the histogram in PixInsight. Your camera has a resolution of the AD converter of 14 bits (corresponding to a maximum value of 2^14 - 1 = 16383).

Bernd

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #38 on: 2018 May 02 04:31:00 »
Hi Bernd,

yes, I meant the histogram of the camera. So the histogram of the unstretched picture should be in the middle?
I will do a series of exposures and watch the histogram in PI.

By the way, where can I see the histogram in PI? In Histogram Transformation?

Armin

Offline Greg Schwimer

  • PixInsight Addict
  • ***
  • Posts: 210
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #39 on: 2018 May 02 09:24:53 »
What do you mean with "more data is always nice ..."? More exposures?

I used Curves Transformation, too. But what do you mean with CT for contrast and CT for saturation?
Please, maybe you could send a screenshot of the CT or is it implemented in the process container?

By more data I mean more exposures. This will help reduce noise further and also help raise the signal to noise ratio for the image, especially the dimmer parts.

The process container has the processes I used. I'd skip the TGV Denoise part for now. The mask I use is a simple extraction from the integrated image.
- Greg
Scottsdale, Arizona, USA

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #40 on: 2018 May 02 10:59:43 »
Hi Greg,

I could open the Process Manager and see all your processes. By double clicking I can open each process and see which settings you´ve made.

Did you just do 8 times the same DBE? I cannot see any difference when processing this.

I don´t understand "The mask I use is a simple extraction from the integrated image." But I think I´ve to watch a tutorial for generating masks, right?

And I don´t understand what you did in that many Histogram Transformations (HT).
In my processing I took the "New Instance" button once from the SCT and laid it to the HT in order to stretch the picture.
But I will also read more about the HT.

Thanks,
Armin

Offline bulrichl

  • PixInsight Old Hand
  • ****
  • Posts: 437
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #41 on: 2018 May 02 11:05:15 »
Hi Armin,

Quote
yes, I meant the histogram of the camera. So the histogram of the unstretched picture should be in the middle?
I will do a series of exposures and watch the histogram in PI.
The Peak in the histogram of the raw flat frames shall be about in the middle of the data range of the camera. In your case (14-bit ADC) this means: 0.5 * 2^14 ADUs = 0.5 * 16384 ADUs = about 8200 ADUs. In PixInsight you can view it either in the histogram or in ImageStatistics (mean or median).

Quote
By the way, where can I see the histogram in PI? In Histogram Transformation?
Yes, in the Histogram Transformation module. The image in question must be opened, and in HistogramTransformation you select that View, from which the histogram shall be displayed.

Bernd

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #42 on: 2018 June 08 14:10:10 »
Hi Bernd,

yesterday I did some flat frames with various exposure times: 0.2s, 0.3s, 0.4s and 0.5s

In the histogram module I can see, that with rising exposure time the histogram moves from the left corner a little bit to the middle. But it is still far away from the middle. Also in the ImageStatistics the mean values go from 260 (0.2s) to 650 (0.5s) which is far away from 8200.

So I have to chose such an exposure that I will get 8200? I think this would be some seconds.

Armin

Offline bulrichl

  • PixInsight Old Hand
  • ****
  • Posts: 437
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #43 on: 2018 June 09 01:56:45 »
yesterday I did some flat frames with various exposure times: 0.2s, 0.3s, 0.4s and 0.5s

In the histogram module I can see, that with rising exposure time the histogram moves from the left corner a little bit to the middle. But it is still far away from the middle. Also in the ImageStatistics the mean values go from 260 (0.2s) to 650 (0.5s) which is far away from 8200.

So I have to chose such an exposure that I will get 8200? I think this would be some seconds.

Hi Armin,

yes. However, one additional point is important: the data that I gave are valid only whith ImageStatistics set to 16-bit [0,65535].

At the beginning of this thread you wrote that you are taking the flat frames with a flat field box. Exposure times of a few seconds seem unusually high then. Did you dim the flat field box?

Bernd

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #44 on: 2018 June 09 03:37:03 »
Quote
At the beginning of this thread you wrote that you are taking the flat frames with a flat field box. Exposure times of a few seconds seem unusually high then. Did you dim the flat field box?
Yes, I dimmed it a little bit. Should I drive with maximum performance?

Quote
However, one additional point is important: the data that I gave are valid only whith ImageStatistics set to 16-bit [0,65535].
I´ve set the ImageStatistics to 14-bit, because you wrote that I´ve got a 14-bit ADC in my Sony. :-\
When set to 16-bit in ImageStatistics I get 1000 (2s) to 1800 (5s).