BPP Not recognizing Master Flat

[javajunkie2121]

Hi:

I have PI version 01.08.03.1115 updated as of November 4 2014.

 I had used the ImageCalibration followed by ImageIntegration processes to create a bias-reduced Master Flat.  I have been trying to use this Master Flat with the BatchPreProcessing script for my lights, have the "use master flat" clicked, have loaded it into the correct tab..but I get a warning under diagnostics that I have not selected flat frames to calibrate light frames?  This had worked before ..not sure what's going on?

jeff

[pfile]

BPP might not be associating the flat with the lights properly. to try to fix this you can use "add custom" to add the flat to BPP, setting the type to "flat field" and entering the filter name.

rob

[javajunkie2121]

Hi Rob:

I tried your suggestion, but I got the same error message under diagnostics.."no flat frames have been selected to calibrate light frames"

I tried creating the Master Flat twice. I'm using the IP4AP tutorial method (PI-12-Stack-7) as I had done before successfully (before the updates). The only difference I see from when warren keller made that is a noise evaluation field set to multiresolution support in ImageCalibration.  otherwise, I loaded flat frames, entered Master Bias for ImageCalibration.  I then loaded the calibrated flat frames into ImageIntegration, combination=average, normalization=multiplicative, weight=I don't care, and percentile clipping and equalize fluxes in pixel rejection, used default settings under pixel rejection (2).

this creates a file that isn't recognized as a flat in BPP, whether I use "load flats" or use "custom".

I then tried creating the bias-reduced master flat in Nebulosity, and used that Master Flat version with BPP....it worked without issue.

..so there is something going on with the ImageCalibration and/or ImageIntegration steps that wasn't happening pre-update that is causing BPP not to recognize the file as a master flat

jeff

[pfile]

i don't know under what conditions BPP would refuse to associate a flat with a filter. i am not sure BPP really "knows" any kind of frame is really that type of frame, except of course for hints in the filename or the fits header. my understanding is that "custom" should override any of the automatic matching logic that BPP would try to apply... so i wonder if there is some other problem, like the image dimensions of your flats/bias/darks/lights somehow mismatch for some reason... but that's just a wild guess.

rob

[javajunkie2121]

yes I think there's something about the Master_Flat file that is created using ImageCalibration/ImageIntegration now vs. when I last used it (October 7th) that is causing BPP not to recognize it as a Master_Flat..even when I load it using "Load Flats" or when I use the "Custom" approach and label the file as a flat field

..I took the exact same flat files and created a bias-reduced master flat in Nebulosity and then returned to PI's BPP and it worked fine..so don't think it's an issue with my original flat files.

jeff

[jkmorse]

Jeff.

See this thread and my suggested fixes and the results towards the end where using the "signed as physical" tag helps with flats:

http://pixinsight.com/forum/index.php?topic=4880.msg51087#msg51087 

Hope that helps,

Jim

[jkmorse]

Jeff,

Note this likely means you will need to do things manually and not use BPP, but I never use that anyway since then I can't control the details.

Best,

Jim

[javajunkie2121]

Jeff.

See this thread and my suggested fixes and the results towards the end where using the "signed as physical" tag helps with flats:

http://pixinsight.com/forum/index.php?topic=4880.msg51087#msg51087 

Hope that helps,

Jim

thanks for the reply Jim.

not sure I completely understand the old thread, but I checked the FITS format settings and it did not have the second item (signed integer images) checked, so I changed that.  I think you're saying that for creating the master flat I need to put "signed-is-physical" in the input hint for both image calibration (calibration of flats with Master bias) and imageintegration processes?

jeff

[jkmorse]

Actually, when creating the master flat, first just try putting "lower-range 0 upper-range 65535" in the input hints in the Image Calibration tool only and do not use in the ImageIntegration step.  Then, when you are processing your lights, put the 'signed as physical" input hint into the ImageCalibration tool.

If that does not work, then reverse the two and see if that works. 

[gordonewen]

I have been having exactly the same problem as Jeff over the last few days. I hadn't appreciated it was the new version that gives this problem when using the BPP script, but that explains why it worked for me a couple of weeks ago. I will try using the hints as suggested in the thread referred to below and use Image Calibration and Integration for preparing flat masters.
However, that leaves a problem; the BPP script basically doesn't work. Will this be fixed so that it works as before? I am sure many beginners with PI (including myself) would be looking to use this script, and it did give pretty good results.
Regards
Gordon

[jkmorse]

Gordon,

Its not an issue with the new version.  This has been an issue for some time and seems to be hit or miss as to when it will rear its ugly head.  It is worth reading the posts I attached to my earlier note since they explain the root of the problem.  This is an industry problem and not one imbedded in PI as such.

Jim

[gordonewen]

Jim, thanks for the reply. I guess I will have to go the calibrate, debayer, align etc route. I'll check out the links you mention.
Regards
Gordon

[gordonewen]

I am still getting very odd results. I have used the BPP and loaded lights, darks and flats (uncalibrated) which should give me an image, albeit with a little noise due to not using flat darks. BPP did not give a diagnostic message stating that I had not used a flat frame, so that was good. Having run the script I got a master light that looked like the flat hadn't been subtracted and was a lurid red colour, not the orangey green I would expect from light pollution. Clearly something is going wrong, can anyone advise? The flat field subs do look like flat fields so should work.
As a comparison, I used the BPP tool in Nebulosity (which I use for image acquisition) and this also gave an image that looked like it hadn't been flattened but was the right colour.
I tried the routine of adding a hint to use 'signed-is-physical'  in image integration and got the same results. Feel like I am banging my head against a wall.............. tell me I am missing the point somewhere........
Regards
Gordon

[jkmorse]

Try doing it outside BPP and see what results you get.  Also, did you create your masters using the hints as well as I described in my note above?

[gordonewen]

Jim, I need to go back and have another go; I tried all the ways suggested and haven't yet got anywhere. I have discovered one problem with nebulosity which I use for image acquisition; it saves my bias frames with 2 less rows and columns than the lights, flats and darks, rather baffling. It seems the 2 extra columns are added to the lights etc. I believe Craig Stark is looking into it.

I read that PI will be introducing an image acquisition module at some point; it can't come soon enough in my opinion as it would to appear to offer a totally integrated workflow then.

Jeff, how did you get Nebulosity to create a master flat? Did you do it as part of processing some light frames? I can't see how you can do it otherwise.

Regards
Gordon