Having trouble with compound light pollution gradients

[macnmotion]

Hello,

I've tried every variation of DBE I can think of to remove the light pollution gradients in an image of NGC 2683.

If anyone would like to try, I've placed the .xisf file here:

https://drive.google.com/drive/folders/1GTcInNs8NT5XejaR15gWoerNqkhkM312?usp=sharing

Any suggestions on how to approach these gradients would be appreciated. I'm attaching a screen shot of the L stack. Thanks.

[pfile]

here is a DBE process icon... it is on Workspace04 and don't forget to use "Merge Process Icons..." when loading it, so that you don't delete any process icons you've already made on your desktop.

unfortunately you'll find that something went wrong in the flattening, as the image has embossed concentric rings all over it under the light pollution. which is a shame because this image is optically excellent and there are a ton of tiny galaxies in the frame. you can however just not stretch it so much and hide all that stuff that's down in the floor.

rob

[pfile]

also, considering the image more carefully and looking at the gradient, i'm guessing that 1) this is an FLI camera or another camera that has an iris-like shutter, and 2) the flats are too short (< 2-3s) so that the shadow of the shutter is evident in the flats.

if these things are true, you should consider new flats with a dimmer light source and longer duration.

rob

[macnmotion]

Thanks for the reply. You're right it seems that the flats haven't done a great job, and if we can make some better flats perhaps DBE will give a better result. We'll take a look at that.

[macnmotion]

Hi again. We shot new flats, 6 seconds so shutter shouldn't come into play. Still seeing imperfection when running your (very dense) DBE. Not sure at all what the x pattern is. Still dust motes that haven't been picked up by the flats. Any thoughts?

The new .xisf is in the same folder:

https://drive.google.com/drive/folders/1GTcInNs8NT5XejaR15gWoerNqkhkM312

[pfile]

at this point i'd have to look an uncalibrated light, a flat sub, and a calibrated light. the "embossed" look of a dust bunny on a flattened sub is usually indicative of dust motes that have moved slightly between acquisition of the lights and the flats. that motion can either be due to the dust particles actually shifting around as the filter wheel turns, or it can be due to the filter wheel not perfectly centering the filter from move to move. if the images were dithered, the slight movement between each sub in the registered images further enhances the embossed look.

your filename says "RILA" so if this came from one of those OS scopes i'd say that there's a few more possibilities. with really fast telescopes, the light cone is very steep and so it's possible that a slight difference in focus between the light and flat could cause this. another possibility is that stray light is entering the telescope during flat acquisition, especially if there is no shroud.

i assume these are panel flats? can you take sky flats?

rob

[macnmotion]

I've added an uncalibrated light, a flat sub and the calibrated light in the folder:

https://drive.google.com/drive/folders/1GTcInNs8NT5XejaR15gWoerNqkhkM312

The calibrated sub looks OK to me. We shot L subs on 4 nights, and calibrated subs from each night look the same to me (I've added a folder with one calibrated L from each night), I don't see dust motes even when using extreme stretch. So it doesn't make sense to me why I'm seeing all that dust still in the final stack.

FYI the process steps:

1) calibrate lights (super bias, master dark, master flat)
2) cosmetic correction (not done on the RiLA subs, not necessary)
3) subframe selection
4) alignment
5) integration

Yes these are panel flats. We are unable to take sky flats at our latitude, the sky brightness changes too quickly and we are unable to take uniform sets of flats. The images are not dithered.

[ngc1535]

Hi,

I checked out the frames and I think something more subtle is going on. (However, I am usually wrong- please let me know so I will make new poor guesses in the future!)

Rob's guess was a nice one... but looking at the flat of course there isn't evidence for flat issue with regards to shutter. Also, a little soapbox action here, "light pollution gradient" is a catch all description that is usually *less likely* than other more boring reasons. In your case the field of view is on the order of 1 degree based on the apparent size of NGC 2683. So the "features" that are not calibrated out are 10-20 arcminutes. Light pollution, even from the moon, does not vary on small scales such as this other than very smoothly. Ok... off soapbox.

So we haven't seen the bias... or the dark frame. Darks are tricky creatures. Presumably you used a 6 minute dark frame to calibrate. Is there any evidence of scattered light in the master dark frame? As part of calibrating your images... this would appear as potentially darkish things that wouldn't have another explanation. There have been many times that astrophotographers take darks...yes at night, but near their computer monitors... and just a wee bit of scattered light is detected.

Anyway, just another crazy theory to throw out there.

-adam

[pfile]

its actually not unusual to not see subtle artifacts in a single calibrated sub... you end up only seeing them when you stack a bunch of images, because they are so faint.

is there a chance that only some of the subs in the stack are messed up?

are you really using 4 telescopes simultaneously on one target? if so is there any chance the calibration files got mixed up?

i'll take a look at those files.

rob

[pfile]

Hi,

I checked out the frames and I think something more subtle is going on. (However, I am usually wrong- please let me know so I will make new poor guesses in the future!)

Rob's guess was a nice one... but looking at the flat of course there isn't evidence for flat issue with regards to shutter. Also, a little soapbox action here, "light pollution gradient" is a catch all description that is usually *less likely* than other more boring reasons. In your case the field of view is on the order of 1 degree based on the apparent size of NGC 2683. So the "features" that are not calibrated out are 10-20 arcminutes. Light pollution, even from the moon, does not vary on small scales such as this other than very smoothly. Ok... off soapbox.

So we haven't seen the bias... or the dark frame. Darks are tricky creatures. Presumably you used a 6 minute dark frame to calibrate. Is there any evidence of scattered light in the master dark frame? As part of calibrating your images... this would appear as potentially darkish things that wouldn't have another explanation. There have been many times that astrophotographers take darks...yes at night, but near their computer monitors... and just a wee bit of scattered light is detected.

Anyway, just another crazy theory to throw out there.

-adam

yes, i can't speak to the true origin of the gradient... i just noticed kind of a diagonal line and the hint of a "flower" shape and realized i had seen that before.

but if you use the process icon i made and remove whatever the gradients are, you'll see that there's really some kind of flattening issue. the image of the aperture of the telescope is all over the frames. so i was concentrating on solving that...

rob

[pfile]

so, i can see the bad flattening (meaning, the concentric rings) in NGC2683_L_360_00000082_NGC_2683_c when i apply the DBE, but for instance i don't see it at all in NGC_2683_LIGHT_L_360s_BIN1_28C_017_20180118_233709_132_PA0_c.

is it possible that the flats just don't match? meaning the flats were taken on night 1 and flatten night 1 just fine, but the dust motes shifted on night 2,3 and 4, and the flats from night 1 are no longer good? i've actually had situations where the evening sky flats differ from morning sky flats as dust motes moved on the same night.

and having looked at the subs i agree with adam - all of the subs have almost identical gradients. it's very difficult to tell from a stack what the origin of a gradient is, but if you see the exact same artifact in multiple subs then there's probably something wrong with the calibration.

in the screenshot, the dark patches around the stars is because i was lazy and just used the same DBE icon. but on the right you can pretty clearly see the shadows of the dust motes.

rob

[macnmotion]

We shot this galaxy on two telescopes, a RiLA and a RiFAST. The RiFAST was used for RGB data. I do have 60 subs of L from the RiFAST but in order to find the source of this issue I've left those out of the equation for now. So I have about 140 L subs from the RiLA from 4 nights (out of a 7 night stretch).

We have tried two sets of flats. We had flats taken about 4 days before the first day of shooting L. We took new flats after the final day of shooting L. Note that our Flat process is very manual and hands-on. We cannot shoot sky flats at our 13 degree latitude, the sky brightness changes too quickly. And since we are now shooting remotely during the week, there will be no way to have nightly matching flats. Even if we were at the observatory every day there would be no way, it's too involved and time consuming to set up and shoot. So we will have to do the best we can with the flats we are able to get.

The new flats we shot the other day are longer than the first set and eliminate the shutter artifact.

I'm in the process of going through all 140 calibrated subs one by one to see whether the dust is limited to a subset.

Our darks were shot at night (no way to cool the cameras enough during Thai daytime hours). We shoot 3 minute darks and use PixInsight's optimization to match exposure length for our Light subs.

We use a Super Bias created from 100 bias subs.

[ngc1535]

Hi Again,

It may not be your issue...but as I said, you might consider if there are any sources of light in the observatory during the dark acquisition.
Regarding the darks... yes using PI's optimization is good. Of course you need to dither your data. However, it is generally better to acquire a dark that is longer than the data you are calibrating and scale from a larger thermal frame to a smaller valued one. Going the other direction runs the risk of multiplying the error/uncertainty. I would be really really impressed if you did the math and took enough 3 minute darks to minimize the noise! (Read Berry and Burnell for one mathematical treatment.)

Regarding flats- it is some what important to hit the flat panel in both alignment and tip/tilt. The flat panel I have used is on the dome. Dome is homed and then moved to flat position in order to minimize pointing errors. If you are manually illuminating surfaces and such to generate flats- indeed that will result in errors. There is something to the mechanical reproducibility of pointing and illumination from the panel (or the sky at a fixed position) that just can't be done easily with diffused light.

If you are something of a patient photographer that goes for the nth degree- you can always generate night sky flats which, if you get enough signal, will perfectly calibrate your data. It is really the best you can do... but takes time, time that could be spent imaging! Fabian Neyer has a good presentation regarding this process for amateurs. Night Sky flats can act as a fiducial to measure the quality of the other methods you are using.

-adam

[pfile]

ok - so i think most likely the rings and stuff are just flats that don't match. IMO you'll have to find a way to shoot flats every evening and morning. i have to ask though why it's so difficult to shoot panel flats, unless there is just no way to mount a flat panel in your observatory. because even low-end acquisition software can control the brightness of a panel and provide some way to slew the telescope to point at the panel. with approximately 10 zillion dollars worth of telescopes and mounts, it seems like automating flats would be the least of your worries

you might try with a plain old master bias as well. when you calibrate the flats are you using matching darks, or are you scaling those too? or just using the master/super bias for the flats? it may help to have proper flat darks, who knows.

rob

[macnmotion]

First I want to thank you guys for taking the time to talk through this with me.

We hadn't considered night flats. I'll read up on this. Note that we are not using flat darks, we're using the same darks used for calibrating our lights.

This is a large scope, we haven't found any type of EL panel specifically made for this size (we do have EL caps we can use for our RH 300's, but not for our 400mm RiFAST and 600mm RiLA). We instead use an LED TV, and play a "white movie" on it through VLC on an attached computer. We cannot mount it on the dome, as the domes are clamshells. Instead, we have it on a mini lift to bring it up so that we can point our scope horizontally at the screen. We use a sheet of diffuser paper over the screen to help even out the lighting. And we use brightness controls on the TV and in VLC to get it in range for each filter. We are targeting ADUs in the high 20 thousands (ranging from 27,000 - 29,000).

If anyone has any ideas on how to improve our flats within the limitations of our setup, we'd love to hear them. Especially if we could remotely control the process (really just the light source, as everything else is already automated).

That said, I did do some more sleuthing. I've found some answers, and am left with some questions. As I said, we have flats from both Jan 12 and Jan 20. All our subs were shot in between. I created Lum stacks from each of the four nights we shot, unregistered, so that any dust motes would add together and be easier to see. For each night I calibrated once with the Jan 12 flat, once with the Jan 20 flat. I found that I need to use the Jan 12 flat for subs shot on Jan 12 and 13, and I need to use the Jan 20 flat for subs shot on Jan 16 and Jan 18. That should at least solve most of the dust issues. Following are 4 comparison shots (all using PI boosted autostretch):






I then took the Jan 18 subs and calibrated in different combinations of flats, darks, bias. Here is a labeled screenshot showing all combinations, as well as the master flat used during calibration.



All .xisf files for all of these comparisons are in this folder:
https://drive.google.com/drive/folders/1J3ZXsHzZXEezDu1Ok87y_cq2SgVdTGg8

As you can imagine we would like the best calibration solution possible considering our circumstances. We spend a lot of time on acquisition, and a lot of time on processing, it's a shame to be handicapped at all.