Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - pfile

Pages: 1 ... 268 269 [270] 271 272
4036
interesting, i was seeing the 'MRS failed' message as well. the target in question is the NGC7000 area and there are quite a lot of stars there. sorry i didnt mention this, didn't realize it could have been a bug.

4037
so in real life, how close should one expect to get to the theoretical numbers? i have a stack of 97 dslr images (120s exposures of the NGC7000 area). after tweaking the rejection thresholds i only managed to get up to around Dss of 4.0,4.4,4.0 (R,G,B). while i was working on a region of interest in a brighter part of the frame, i was at about 6.0,3.8,4.0 if i remember correctly. does this just mean that my subframes are underexposed? my histograms were very far to the right but i was working during a full moon and under badly light polluted and humid skies.

4038
Bug Reports / Re: ImageCalibration problem
« on: 2010 May 25 13:59:48 »
Juan,

thanks very much for looking at this. i've been busy with work and home stuff the last few days and have not been able to spend any time on this.

i suspected that the flat frame was just wrong after thinking about it for a while. although it is possible to manipulate this flat until it divides properly, it seems like a proper calibration routine should never attempt to do this.

while i'm not terribly sure why the apogee driver change would cause such a drastic difference between the flat illumination and the science frame's illumination, it is a fact that this flat was created from subs from the older version of the driver, which was producing 4098x4098 frames. there is a glimmer of hope - lightbuckets has completed generating the new calibration master frames with the new camera driver. maybe tonight i'll be able to download those and see what happens in ImageCalibration.

edit: whoops, i am confused. it looks like all my flats were generated after the driver update. so in theory they should be "correct" both in the sense that the came from the same driver and also that they were taken within a couple of days of the science frame aquisition. at this point i really don't know what to say, or what to do next. when i massaged the flats they did seem to properly correct the dust donuts. i did not look carefully enough to see if they had messed up the overall illumination.

4039
Bug Reports / Re: ImageCalibration problem
« on: 2010 May 22 15:00:39 »
no, no, i understood. that's exactly what i did. i'm left with a histogram that looks pretty much the same as the original, except no "junk" at the low end, and a big spike at the left edge of the histogram where all the black pixels were replaced with 0.65.

what i am getting at is simply this: the outer corners of the flat have values in the 0.7 range. dividing the light by 0.7 is the same as multiplying those pixels by 1.4. however, if you take a background pixel of the science frame that's near the middle and divide it by a background pixel that's near the edge, you get about 1.014. that's the factor you'd have to multiply an edge pixel by to make it as bright as a center pixel. multiplying it by 1.4 is way overcorrecting the science frame.

that's what i'm trying to understand. maybe this means the master flat is just "wrong" because it gives a greater variance between center pixels and corner pixels than i see in the science frame? still this does not explain why your version of the flat works properly.

4040
Bug Reports / Re: ImageCalibration problem
« on: 2010 May 22 10:52:19 »
i can't explain this yet, but i have tried what you are suggesting and i get the same result, whether i divide the flat or i use ImageCalibration. the science frame is still way blown out. i guess i still don't understand the basics here - if you divide any part of that image by 0.65, won't that part just be ridiculously bright?

are you also using pixinsight 1.6 ?

as for the raw-ness of the science frame, there are definitely donuts there, but they are hard to see. they are more obvious when i've stacked the 20 uncalibrated science frames i have and also applied curves. in the attachment i've also given a crazy STF so they can be seen. i did ABE on the image to try to correct the vignetting.

the cause of the junk at the bottom end of the histogram seems to be a stacking artifact. if you look at the bottom of the flat there are 10 rows of dark pixels. it's almost like maximDL has tried to register the flat subexposures before stacking them. in fact my darks look the same way. this would be a fatal error as far as i'm concerned. EDIT: i think this stuff is actually an artifact in the old apogee driver. i can see that my raw science frames from last year look the same.

also, NKV: all the calibration checkboxes were unticked.


4042
Bug Reports / ImageCalibration problem
« on: 2010 May 21 22:23:28 »
as seen near the end of this thread: http://pixinsight.com/forum/index.php?topic=1930.msg12207#msg12207



the flat frame started life as an f32 fits file, as integrated by MaximDL. i have converted it to i16 to match the raw science frame.

if i use the flat frame as-is in ImageCalibration, the result is as in the top right image in the screenshot above: the flat has pretty much destroyed the science frame.
if i use the histogram tool to compress the data in the flat all the way to the right and use that new flat in ImageIntegration, the results are satisfactory.

it seems like ImageCalibration is not scaling the flat properly? or is this user error?

here are the science and flat files in question:

http://public.me.com/ix/pfile/pixinsight_1.6_flat_problem.zip?disposition=download+1274505738440

thanks.


4043
General / Re: calibration of lightbuckets.com images?
« on: 2010 May 21 13:49:58 »
when i get home tonight i can post the fits files somewhere. maybe someone can take a look at this, perhaps it is a bug?

of course in this example i have not calibrated the master flat frame (nor have i set bias or darks when calibrating the science frame) but i don't think it should matter - i get the same result if i put the master bias and master dark into ImageCalibration as well.

4044
General / Re: calibration of lightbuckets.com images?
« on: 2010 May 21 13:17:30 »
can someone advise me regarding this?



the top row of images are : the raw science frame, the master flat frame and the resultant calibrated frame. the statistics window is set to the master flat frame.
the bottom row of images are: the flat frame, scaled with the histogram tool as shown in the histogram settings. the calibrated frame is on the right. it looks pretty reasonable.

is it to be expected that i'd have to manipulate the flat this way in order to get correct division in ImageCalibration? clearly the topmost calibrated frame is completely screwed up. my problem with this is that what i've done to the flat to make the calibration right is completely ad-hoc.

4045
General / Re: calibration of lightbuckets.com images?
« on: 2010 May 19 23:33:22 »
on #2 that looks like what happened to me when the file was saved with compression in maximDL.

4046
General / Re: calibration of lightbuckets.com images?
« on: 2010 May 19 13:40:21 »
do you get a dark scaling factor of 0.000 when it tries to subtract the dark from the light frame?

this might be specific to the recent driver updates on LB-0003, but the darks i currently have are shifted up by 10 pixels with respect to the lights. this causes pixinsight to fail to find a correlation between the light and the dark, and so it can not scale the dark.

i do have some older data from LB-0002, but i never tried to calibrate it with PI. i'll have to give that a try and see what i get.

i am still having problems with pixinsight's handling of the flat frame. can i upload one light and the master flat somewhere and maybe juan or vincent can look at it? (did vincent write the ImageCalibration tool?)

4047
Announcements / Re: Version 1.6.1 — Delayed
« on: 2010 May 19 11:43:53 »
thanks for the heads-up. it will be worth the wait for the 64-bit macosx version, for me anyway. i've made the leap to snow leopard on my main computer.

4048
General / Re: calibration of lightbuckets.com images?
« on: 2010 May 19 11:20:46 »
Quote
well, you saw steve's reply on lb.com. he seems to feel that if the images work with maxim and ccdstack then it's our problem

Well, whoever 'steve' is, he is entitled to his opinion.

That does not make him 'right', and it further demonstrates that Maxim and CCD Stack must be pieces of software that are quite happy to 'kludge' their way to a solution. And, if that is the case, it also demonstrates that they are probably, more than likely, at the 'root' of the problem as well.

steve owns lightbuckets, owns the telescopes, etc. etc. they use maximDL as their main capturing/processing tool.

i may be putting words in his mouth here. he has always been very accommodating when things go wrong and they are pretty customer-oriented for what they are doing, but when it comes to maximDL they seem to draw a line in the sand. at some point their auto-calibration failed, and i could not calibrate my own files. at the time i did not know why - it turns out to be the problem i am discussing now. but anyway they went back and calibrated the files for me by hand with maximDL, but the files were saved as compressed fits. pixinsight could not open these files; i thought they were corrupted. i encountered a slight bit of resistance but they eventually resaved them as uncompressed fits files for me. at the time though the reaction was sort of like "these open okay in maximDL, so, uh...?"

It took me a little time to appreciate Juan's stance on this - and it really is quite simple at the end of the day. Until such time as our CCD cameras have analogue-to-digital converters in them that provide an output with MORE than 16 bits of resolution (and that would have to be more than 16 bits of USEFUL resolution) then there is - in my opinion, of course - absolutely NO justifiable reason to save that image data in anything OTHER than 16-bit format.

If someone DOES HAVE a reason for doing this - with RAW CCD DATA - then I would like to hear it, in order that I might stand corrected.

And, if a piece of software is then going to use a 16-bit FITS format, then the programmer had better spend some time understanding how to use the BSCALE and BZERO (standard) FITS keywords. (Heck, it took me less than an evening to figure it out for myself, and I had NO IDEA what the FITS system even was at the time)

And, finally, the CBLACK and CWHITE keywords are TOTALLY NON-STANDARD. They could just as easily have been called MICKEY and MOUSE, and they would have done EXACTLY the same job, but ONLY for those few 'kludged' software packages that agreed to use them.

And it is for these reasons that I cannot ever see myself voluntarily using 'kludgeware' such as CCDStack or Maxim - after all, if they are willing to include this kind of 'work around', what other 'horrors' could be lurking in the depths of their code?


i'm really just guessing here about what maxim is doing with CBLACK and CWHITE. i have to assume though that maxim is doing something before using these files as calibration masters... the levels are just way too high. they are actually being written out as f32, but apparently with a range of [0.0-65535.0].


4049
General / Re: calibration of lightbuckets.com images?
« on: 2010 May 19 09:29:33 »
From http://www.cyanogen.com/help/maximdl/FITS_File_Header_Definitions.htm

CBLACK – indicates the black point used when displaying the image (screen stretch).
CWHITE – indicates the white point used when displaying the image (screen stretch).

So these values do not help getting mathematically correct black/white.

I did set the FITS reader mode to ask if values are out of range, but when I get out of the field of upper value after entering 65536, it changes itself to -9223.372036854776500
 ???  ???  ???

-- bitli


well, you saw steve's reply on lb.com. he seems to feel that if the images work with maxim and ccdstack then it's our problem.

i'm starting to think that even though those keywords are for display only that both maxim and ccdstack might be using them to determine the blackpoint in the file. and i'll bet that if you save the stacked bias master as i16 maxim would put the same levels in the file, so that's probably not a solution.

it does not make sense to me for bias frames of a CCD cooled to -30C to have such high values. maxim has this goofy "auto stretch" setting when saving files, and i'll bet that's got something to do with the pixel values in these calibration frames.

i'm still having problems with the flat 'subtraction' when running ImageCalibration (obviously it's division...) i manually divided the flat into the light and found by trial and error that i had to compress the flat histogram into the 0.990 - 1.0 range or the result was wrong. and if i used the modified flat in ImageCalibration, the flat got scaled somehow and the results were the same.




4050
General / Re: calibration of lightbuckets.com images?
« on: 2010 May 18 15:38:18 »
i am talking to myself here, but as far as LB-0003 goes, i think i'd better wait for them to deliver the new calibration masters.

for fun i read in the dark as i described above, and tried to calibrate with it, choosing optimization and bias subtraction. PI found no correlation. so then i manually subtracted the bias frame from the dark, and then scaled the dark with pixel math. i then calibrated a light frame with the bias frame and the manually calibrated dark; here is a 9:1 zoom on part of the image:



as you can see the dark is shifted up about 10 pixels. i guess the apogee firmware update must have done more than just change the size of the file...

Pages: 1 ... 268 269 [270] 271 272