Author Topic: Bad Photos or bad workflow? Need help.  (Read 4511 times)

Offline bulrichl

  • PixInsight Old Hand
  • ****
  • Posts: 428
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #15 on: 2018 April 28 07:36:01 »
Hi Armin,

I am sorry, but your master calibration frames seem all to be wrong:
- The Superbias has these constant vertical pattern, and its intensity values don't suit to the MasterDark. There was no Bias frame, so I cannot help.
- The MasterDark has too large intensity values compared with a Light frame. There was no Dark frame either.
- The MasterFlat has these constant vertical pattern as well, presumably because the flats were calibrated with the (defective) Superbias. (The flat frames are underexposed.)

In the calibrated light frames the red channel is severely clipped, and the blue channel is somewhat clipped as well.

In your M52a.jpg I see horizontal colored banding and a strange constant vertical pattern or grid.

I guess the error has been made during preparation of the master calibration frames, but I don't know exactly what went wrong. Please. proceed according to the guide.

Bernd

Offline dvonhand

  • Newcomer
  • Posts: 4
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #16 on: 2018 April 28 07:55:05 »
Hey,

Are you dithering your frames at all?  Dithering is moving the camera by a couple pixels between each frame so that the same point in the image hits different pixels in the camera sensor each time.

I've found that when using one-shot color sensors I have to dither otherwise I get color artifacts in the image.

You may also want to check your individual master light frames (e.g. the master red, master green, and master blue) to make sure there aren't artifacts in those images.  If there are artifacts in those images then any of the channel combination and color corrections won't work or will be harder to do.  It looks like you have some kind of moire pattern in your master light frames.

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #17 on: 2018 April 28 08:52:13 »
Sorry, my son has birthday party, so just sharing the links of bias and darks:

http://gofile.me/3WOrk/GEK8snLDO
http://gofile.me/3WOrk/Iol8LpwJy

I know dithering, but at the moment I don´t dither.

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #18 on: 2018 April 29 04:10:05 »
Hi Bernd,

you´re right. Something strange happens with my calibration files.

I examined my master_bias and zoomed in and saw a strange pattern. I used exactly the integration settings, you recommended.
I compared it with the master_bias made with the BatchPreprocessing script, which looks different.
Do you know what happens there?

Armin

Offline bulrichl

  • PixInsight Old Hand
  • ****
  • Posts: 428
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #19 on: 2018 April 29 08:45:39 »
Hi Armin,

I took a look at one Bias frame (DSC02160.ARW) and a Dark frame (DSC02220.ARW). They look OK to me.

However all of your master calibration frames are defective. In my guide I didn't describe the prepration of the master calibration files in detail but referred to Vicent's tutorial https://www.pixinsight.com/tutorials/master-frames/ . I guess your error was, that in the integration of bias and dark frames, in the section 'Image Integration', you didn't set 'Normalization' to 'No Normalization'. Please check, that all the settings are made like in Vicent's tutorial when integrating Bias, Dark or calibrated Flat frames. (The settings for the calibration of Flat frames are different again!)

I suggest the following approach:

1) Delete all master calibration frames, delete the calibrated Flat frames, delete the Integrations and the images that were derived from the Integrations.
2) Prepare MasterBias and MasterDark according to Vicent's tutorial. 
3) Calibrate the Flat frames, deviating from Vicent's tutorial, only with the new MasterBias (section 'MasterDark' shall be unchecked, no MasterDark needed in this step).
4) Integrate the calibrated Flat frames with the settings in Vicent's tutorial.
5a) With the new MasterDark and MasterFlat, calibrate your light frames with the settings in my guide, image 2, left hand side.
or
5b) With the new MasterBias, MasterDark and MasterFlat, calibrate your light frames with the settings in my guide, image 2, right hand side.

I am curious about your results.

Bernd

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #20 on: 2018 April 29 10:51:45 »
Hi Bernd,

in my last post I sent the integration setting, too. There youj can see, that I´ve set it to "No normalization".
But I do, as you recommend. I´ve deleted all the files. But integrating the bias and afterwards the darks to a master, there is this pattern again.
Please see the zoom into the master_dark in this post. It is the same pattern as in the master_bias, which I posted in the last post.

I know Vincent´s tutorial. I´ve read it also.

Also strange, I told before, when I´m calibrating with the BatchPreprocessing Tool, these pattern don´t exist.

I think it does not make sense to continue with the work flow, if we don´t know the reason for this.

Are there any other sytem settings or options?

Nevertheless thanks, Bernd.

Armin

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #21 on: 2018 April 29 12:10:01 »
One additional insight:

I don´t get these pattern for master bias and master dark when generating with the BatchPreprocessing script.

But there is pattern in the master flat when generating with the BatchPreprocessing script.

Offline Greg Schwimer

  • PixInsight Addict
  • ***
  • Posts: 210
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #22 on: 2018 April 29 12:16:23 »
The hatch pattern is the bayer matrix and can only be seen in an non-debayered frame that has captured light. On bias and dark frames I believe you won't be able see the matrix because there is no light on the sensor to reveal it. On flats and lights you should will be able to see it.

I'm not sure why it shows up in one bias master. Maybe some flats got into the mix?
- Greg
Scottsdale, Arizona, USA

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #23 on: 2018 April 29 12:53:59 »
Hi Greg,
I do not see it on the bias and dark frames, but on the bias and dark masters.

Offline bulrichl

  • PixInsight Old Hand
  • ****
  • Posts: 428
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #24 on: 2018 April 29 17:21:34 »
If your settings in the integration module are correct when integrating bias and dark frames, the only possibility that I can imagine is (like Greg already suggested) that there are defective bias and dark frames respectively. You can easily check that with 'Blink', generating the statistics. Then compare median and mean values and inspect in detail deviating frames (-> histogram).

If you don't find the reason this way, I'm helpless.

Anyway - it is not plausible for me at all that the problem occurs with the Integration module and not with the batch processing script.

Bernd

Offline bulrichl

  • PixInsight Old Hand
  • ****
  • Posts: 428
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #25 on: 2018 April 30 02:13:05 »
Sorry, I really can't reproduce what you experience.

Finally I loaded 2 of your bias frames (DSC02160_bias.ARW and DSC02190_bias.ARW), made a copy of each and integrated these 4 (actually only 2) files with ImageIntegration and the suggested settings. (The only reason for copying the files is, that ImageIntegration refuses to integrate less than 3 files.) The result was completely as expected: no vertical pattern, see attached screen-dump.

I want to add, that I have installed the new RAW module, see https://pixinsight.com/forum/index.php?topic=12126.0 . Maybe that the old, dcraw-based DSLR_RAW module has problems with Sony raw file format ARW? Perhaps you try the new RAW module as well. However, how should it be that the batch preprocessing script doesn't show the problem? I can't imagine that either. So the reasoning remains: either bad settings or some bad files on your side.

Are you working with the current version of PixInsight (01.08.05.1353), and did you install all updates for modules?

Bernd

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #26 on: 2018 April 30 04:22:08 »
Aaaaaaaah!
At least one success! Thanks !!!
After the RAW update my master_dark_new looks like this:

 :) :) :)

I will do the next steps and give answer.

Armin

Offline Greg Schwimer

  • PixInsight Addict
  • ***
  • Posts: 210
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #27 on: 2018 April 30 10:49:39 »
Is that new master debayered - it shows that it is (RGB in the header)? You don't want it to be. Check the settings for the DSLR RAW format handling here:

     - View->Explorer Windows->Format explorer
     - Select DSLR_RAW
     - Select Edit Preferences
     - Select Pure Raw, then OK.


The preferences window should look like below:




- Greg
Scottsdale, Arizona, USA

Offline ArminPro

  • Newcomer
  • Posts: 24
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #28 on: 2018 May 01 12:55:59 »
Hi Greg,

just coming home from a trip ...

No, it was not debayered, but the "pure raw" option was not enabled.
So I enabled "pure raw" and generated the masters again. Now in the header there stands "gray". Is this o.K.?

Another question is to the xisf: Are the following preferences o.K.?

Armin


Offline Greg Schwimer

  • PixInsight Addict
  • ***
  • Posts: 210
    • View Profile
Re: Bad Photos or bad workflow? Need help.
« Reply #29 on: 2018 May 01 13:50:01 »
So I enabled "pure raw" and generated the masters again. Now in the header there stands "gray". Is this o.K.?

Hopefully that helps. Masters should not be debayered, else you're not getting an accurate representation of every pixel due to the interpolation done during the debayering process.

Quote
Another question is to the xisf: Are the following preferences o.K.?

They look OK to me. I checked mine and see my xisf preference menu differs from yours a bit. Screen shot of mine is below. These are the defaults. I've never needed to change them.
- Greg
Scottsdale, Arizona, USA