WBPP Problem With Darks

Scotastrophe

Member
Oct 2, 2021
13
2
I am getting on with the learning associated with Pixinsight and trying to understand where problems arise, why that is and how I can fix them. This one has me stumped though so I'm hoping it's something others have come across. With help I have managed to get WBPP to work with data I recently captured and have produced a decent (for me) result at the end using Lights (LRGB), Bias and Flat frames. My issue is when I introduce dark frames the process does not complete and I get an error message saying "Process Suspended" with a pop up saying, "Background level is zero. Please select a background area with a higher signal level. We recommend checking your image since it could have a large proportion of clipped data."

I have made sure the dark frames are at the same temperature, exposure and gain as the light frames however I still get this failure.

When I open the image that is on screen when the process stops it doesn't look good, far too dark even when stretched.

I have attached a link to some screen shots of the control panel and the log file as well as the abovementioned frame. Any advice or explanation welcomed.

Thanks

 

pfile

PTeam Member
Nov 23, 2009
6,933
297
the best way to proceed here is to set everything up in WBPP and then look at the control panel - take a screenshot of the control panel and post that screenshot here.
 

robyx

PixInsight Staff
Jul 17, 2019
398
138
44
Zurich
I am getting on with the learning associated with Pixinsight and trying to understand where problems arise, why that is and how I can fix them. This one has me stumped though so I'm hoping it's something others have come across. With help I have managed to get WBPP to work with data I recently captured and have produced a decent (for me) result at the end using Lights (LRGB), Bias and Flat frames. My issue is when I introduce dark frames the process does not complete and I get an error message saying "Process Suspended" with a pop up saying, "Background level is zero. Please select a background area with a higher signal level. We recommend checking your image since it could have a large proportion of clipped data."

I have made sure the dark frames are at the same temperature, exposure and gain as the light frames however I still get this failure.

When I open the image that is on screen when the process stops it doesn't look good, far too dark even when stretched.

I have attached a link to some screen shots of the control panel and the log file as well as the abovementioned frame. Any advice or explanation welcomed.

Thanks

From your screenshots I don't see anything wrong in the configuration, exposures match, flats are calibrated using the Bias, filter names and session keyword looks ok.

The issue, then, needs to be investigated by inspecting your raw files directly and do some checks.

This is a shortlist of what I usually check at first:

1. I run WBPP without flats to determine if the issue is there because of the darks or because of the flats.
2. I assume the issue is due to a dark frames overcorrection (flat correction never produces black pixels), so I check the FITS header of a dark frame and a light frame to determine if exposure, offset, gain, camera model, acquisition software is really the same. Very often I find differences which means that darks need to be done again with the same software/camera configuration
3. it could be that all parameters matches, then you need to check how much signal you have in the light frames with respect to the dark frames
3.a if the light frames signal is more or less the same as the dark frames where you have a low signal then you need to add a pedestal to avoid zero-truncation after calibration.
3.b if lights contain less signal than darks you need to understand why you have this mismatch and most probably redo the darks matching the same software configuration you had for light frames
4. if nothing of the previous apply then the investigation continues depending on the specific case.

Always ensure that:

- dark frames have the same gain/offset (and temperature if possible) as light frames
- avoid taking darks and light frames with different software, if you do I strongly recommend using the same drivers (do not use native drivers with one and Ascom drivers others)

I hope this helps you to start your investigation, if you still struggle in finding the issue then you can share the files directly so we can work on the data to figure out what happens.

Robyx
 

ngc1535

PixInsight Ambassador
Feb 1, 2014
982
207
AdamBlockStudios.com
Isn't that error coming from LPS?

The calibrated files you show look black-clipped (which indicates an over subtraction for some reason). Usually there is something wrong with the master darks or biases.

Are the values of the screenshot (Blue) data you show actually "0" in between the stars?
If so, then you need to look at the value of the dark compared to the values of the data in your lights and see if it makes sense.

-adam
 
  • Like
Reactions: Scotastrophe

Scotastrophe

Member
Oct 2, 2021
13
2
From your screenshots I don't see anything wrong in the configuration, exposures match, flats are calibrated using the Bias, filter names and session keyword looks ok.

The issue, then, needs to be investigated by inspecting your raw files directly and do some checks.

This is a shortlist of what I usually check at first:

1. I run WBPP without flats to determine if the issue is there because of the darks or because of the flats.
2. I assume the issue is due to a dark frames overcorrection (flat correction never produces black pixels), so I check the FITS header of a dark frame and a light frame to determine if exposure, offset, gain, camera model, acquisition software is really the same. Very often I find differences which means that darks need to be done again with the same software/camera configuration
3. it could be that all parameters matches, then you need to check how much signal you have in the light frames with respect to the dark frames
3.a if the light frames signal is more or less the same as the dark frames where you have a low signal then you need to add a pedestal to avoid zero-truncation after calibration.
3.b if lights contain less signal than darks you need to understand why you have this mismatch and most probably redo the darks matching the same software configuration you had for light frames
4. if nothing of the previous apply then the investigation continues depending on the specific case.

Always ensure that:

- dark frames have the same gain/offset (and temperature if possible) as light frames
- avoid taking darks and light frames with different software, if you do I strongly recommend using the same drivers (do not use native drivers with one and Ascom drivers others)

I hope this helps you to start your investigation, if you still struggle in finding the issue then you can share the files directly so we can work on the data to figure out what happens.

Robyx
Thanks again Robyx. I have gone through these steps as indicated I think.

1. Ran without Flats and same issue occurred.
2. I believe the fits headers match in these fields.
3. I ran statistics on both frames (attached) and then ran the process with a pedestal of 999 for all the lights with the same result.

Subsequently, having read there the ASI1600MM doesn't work well with Bias frames I removed these and replaced with Dark Flats and with the pedestal still in place got the same error.

I then ran 1 bias, 1 flat, 1 dark, and 3 luminance and that processed fine.

If I remove the darks it works also.

I'm kind of worn out now but I suspect maybe there are some rogue frames somewhere or its beginners user error.

Any suggestions welcome.

Steve
 

Attachments

robyx

PixInsight Staff
Jul 17, 2019
398
138
44
Zurich
Thanks again Robyx. I have gone through these steps as indicated I think.

1. Ran without Flats and same issue occurred.
2. I believe the fits headers match in these fields.
3. I ran statistics on both frames (attached) and then ran the process with a pedestal of 999 for all the lights with the same result.

Subsequently, having read there the ASI1600MM doesn't work well with Bias frames I removed these and replaced with Dark Flats and with the pedestal still in place got the same error.

I then ran 1 bias, 1 flat, 1 dark, and 3 luminance and that processed fine.

If I remove the darks it works also.

I'm kind of worn out now but I suspect maybe there are some rogue frames somewhere or its beginners user error.

Any suggestions welcome.

Steve
Look ad the mean value of your dark frame, it's 4889 ADU while your light frame's mean value is 3224 ADU, this is a non-sense since the light frame should contain "at least" the signal you see in the dark frame plus the light gathered from the scope.

There is a substantial difference in how you shoot the dark frames with respect to how you take light frames, try to figure out how you did both and spot what was different!

One note: I assume you have a cooled camera! I see that light frames were taken at 22:31 while darks were taken at 15:48, I hope the sensor temperature was the same for both (did you forget to turn on the cooling for darks?)
 

Scotastrophe

Member
Oct 2, 2021
13
2
Thanks again Robyx. I’ll see if I can work it out but I thought I had matched the necessary elements. I guess this is a good way to learn. If you learn from your mistakes I’ll be an expert in no time. I’m pretty sure the cooler was on but I’ll definitely check that too.
 

ngc1535

PixInsight Ambassador
Feb 1, 2014
982
207
AdamBlockStudios.com
"There is a substantial difference in how you shoot the dark frames with respect to how you take light frames, try to figure out how you did both and spot what was different!"

Rob: There could very well be nothing wrong with Scota's acquisition of darks. This sensor/camera appears to have dark variations on short timescales(in my experience) ...I do not know why. I made a video about it.

Scota,

I was a little sad you went through that effort of running WBPP more times- my suggestion would have gotten you the to place that Rob got you!

In fact, your sensor has a history (in my experience) of this very issue.
In the case I show below, the dark was again the culprit and it was messing up the person's flats. But the underlying issue is precisely the same.

Watch Part 3 below that proves this (though there is cool stuff that will interest you regarding the first two parts:

 
Last edited:
  • Like
Reactions: Scotastrophe

Scotastrophe

Member
Oct 2, 2021
13
2
Isn't that error coming from LPS?

The calibrated files you show look black-clipped (which indicates an over subtraction for some reason). Usually there is something wrong with the master darks or biases.

Are the values of the screenshot (Blue) data you show actually "0" in between the stars?
If so, then you need to look at the value of the dark compared to the values of the data in your lights and see if it makes sense.

-adam
Thanks Adam, I think this sounds right as Robyx has also suggested. I'm going to retake the darks in my darkroom and see if there was some light leaking into the darks and biases before.

Steve
 

Scotastrophe

Member
Oct 2, 2021
13
2
"There is a substantial difference in how you shoot the dark frames with respect to how you take light frames, try to figure out how you did both and spot what was different!"

Rob: There could very well be nothing wrong with Scota's acquisition of darks. This sensor/camera appears to have dark variations on short timescales(in my experience) ...I do not know why. I made a video about it.

Scota,

I was a little sad you went through that effort of running WBPP more times- my suggestion would have gotten you the to place that Rob got you!

In fact, your sensor has a history (in my experience) of this very issue.
In the case I show below, the dark was again the culprit and it was messing up the person's flats. But the underlying issue is precisely the same.

Watch Part 3 below that proves this (though there is cool stuff that will interest you regarding the first two parts:

Thanks for the response ngc1535. I'll watch these later and see if it answers my problem.
 

Scotastrophe

Member
Oct 2, 2021
13
2
Look ad the mean value of your dark frame, it's 4889 ADU while your light frame's mean value is 3224 ADU, this is a non-sense since the light frame should contain "at least" the signal you see in the dark frame plus the light gathered from the scope.

There is a substantial difference in how you shoot the dark frames with respect to how you take light frames, try to figure out how you did both and spot what was different!

One note: I assume you have a cooled camera! I see that light frames were taken at 22:31 while darks were taken at 15:48, I hope the sensor temperature was the same for both (did you forget to turn on the cooling for darks?)
Having now taken my darks in a completely dark room the mean values are consistently 329ADU therefore I conclude that light leakage into the dark frames is most likely then cause. I'll take the other half later and redo the biases as well and see what happens. Fingers crossed.
 

Scotastrophe

Member
Oct 2, 2021
13
2
Look ad the mean value of your dark frame, it's 4889 ADU while your light frame's mean value is 3224 ADU, this is a non-sense since the light frame should contain "at least" the signal you see in the dark frame plus the light gathered from the scope.

There is a substantial difference in how you shoot the dark frames with respect to how you take light frames, try to figure out how you did both and spot what was different!

One note: I assume you have a cooled camera! I see that light frames were taken at 22:31 while darks were taken at 15:48, I hope the sensor temperature was the same for both (did you forget to turn on the cooling for darks?)
Once I'd replaced the darks WBPP worked like a dream. so thanks again for your help. I must apologise for seeking advice on what turned out to be a capture issue rather than a Pixinsight issue. I guess as I have just bought my first proper astrophotography camera I was used to taking darks in the field before and just underestimated the need for darkness as well as the lens cap and jacket over the top. Here is a link to a quickly processed version of the end result.


Cheers
Steve