Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - TinySpeck

Pages: [1] 2 3 ... 5
1
General / Re: PixInsight Big Files
« on: 2020 January 27 18:05:12 »
Here's what I've found.  I can indeed create & change an image, save only the project, quit PI, ignore the warnings, and restart the project without a hitch.  All images and history are preserved whether they were individually saved or not.  So what in the world are those dire warnings for?  When you save your project you DO NOT risk permanent pixel information or process history loss.  Those warnings are a complete red herring.

I've also found that all the mystery files which accumulate in my project folder do seem to be necessary to recreate the project.  I can't delete all but the files from the most recent session, for example.  All mystery files (all the way back to the oldest) are also renamed to different random names each time the project is started (or exited?).

I'm not using bundled projects, so I bet that explains the files.  The aggregate of all the files seems to represent the whole project.  Next project I will do bundled, so at least I'll only have one file to deal with and to back up.

Thanks for the tips, Rob, I learned an important lesson about PI.

2
General / Re: PixInsight Big Files
« on: 2020 January 27 08:04:25 »
OK, thanks again Rob.  I'll experiment with this this evening.

3
General / Re: PixInsight Big Files
« on: 2020 January 26 15:48:07 »
anyway besides saving a final image in some lossy format or maybe saving an image to disk that is needed by a process (like GradientsMergeMosaic), theres no reason to save individual files... just save the project and all the views (and their history) is saved.

That sounds just like it ought to be.  But when I Save Project and then exit PI I get dire warnings of pixel information being permanently lost unless I save whatever working images I have open.  Images are labeled <*new*> until they are saved, also, which makes me think they need saving, and if I close an image which hasn't been saved I'm told pixel information will be permanently lost.  It sure doesn't sound like images & histories are being preserved in the project!

4
General / Re: PixInsight Big Files
« on: 2020 January 26 11:21:42 »
Oh, that's interesting!  So you save your project files to a different place than you Save Project?  That seems odd and counterintuitive to me, but I will give it a try as soon as PI is done with the lengthy process it's involved in now and report back.

5
General / Re: PixInsight Big Files
« on: 2020 January 26 10:23:00 »
Thanks, Rob, you're always a great help and I appreciate it.

By "operating folder" I mean the folder where my project is stored.  The .seal and all image files are kept there.  I specify it the first time I Save Project.

I looked at my Global Preferences with my project open and the Swap Storage Directories shows only one: a Temp folder in my App Data / Local path (which I believe is the default).  I don't see a path to the current project folder anywhere in the Preferences.

I'm attaching a snapshot of my current project folder, with the mystery files building up, if that helps.  It looks like maybe the mystery files are written when PI closes, judging by the file times from yesterday.

As an experiment, I moved all the mystery files (only) to a temp folder, started PI, and loaded the project.  It showed this error stream on the console:

Reading image: Image_a
*** Error: Parsing ImageWindow MainView child element (line=12264 offset=6): No such file: D:/gb/Astro Image Processing/NGC6946 Fireworks Gal/Image_2.data/ZGUCXV1ZM-000001
* Loading images
Reading image: bkgd1
*** Error: Parsing ImageWindow MainView child element (line=44871 offset=6): No such file: D:/gb/Astro Image Processing/NGC6946 Fireworks Gal/Image_2.data/ZGUCXV1ZM-000008
* Loading images
Reading image: range_mask
*** Error: Parsing ImageWindow MainView child element (line=44954 offset=6): No such file: D:/gb/Astro Image Processing/NGC6946 Fireworks Gal/Image_2.data/ZGUCXV1ZM-000009
* Loading interface states
* Loading embedded XPSM
Reading icon: Process01
Reading icon: Process02
* Regenerating masking relations
* Updating workspaces
** Warning: The project has been loaded partially with 3 error(s).
1.209 s


The .xisf image files don't even load without the mystery files, so some of them at least seem to be critical.


6
General / PixInsight Big Files
« on: 2020 January 25 15:26:03 »
As I work on an image, many mystery files build up in my operating folder.  They range in size from a few dozen kB to 100 MB, and often total about a GB after a while.

I back up my computer daily, including my current image processing folder, and I get many GB of accumulated mystery files even using incremental backups. 

What are these files for?  Do I need to restore them to restart a project after a computer crash?  Can I reduce or eliminate them?

7
Thanks, Rob, that sounds like it should do it.  I'll give that a try (as soon as I can get back to the project...)

8
General / Weighted Batch PreProcessing (WBPP) cosmetic correction?
« on: 2020 January 13 18:21:15 »
I'm trying this script out for the first time, and I can't find anything about the Cosmetic Correction box on the Lights tab.  To use it you need to specify a "template icon".  Can anyone explain what that is?

9
General / Re: 1.8.7 out of memory errors
« on: 2019 October 09 08:05:59 »
Thanks, Juan, I see what you mean.  I will turn off large-scale pixel rejection and try again.  It also occurred to me that I could break this job into two parts and then combine them like you'd do with HDR processing.

And good luck with that perpetual motion machine.    ;D

10
General / 1.8.7 out of memory errors
« on: 2019 October 08 21:01:26 »
I'm stacking the largest collection of subs I've ever tried, and running into out-of-memory errors during ImageIntegration.  I'm closing all programs except PI.  I have the latest PI release and updates (including one from today), and the latest Windows 10 with updates.  I'm using the new automatic buffer sizes option (but also fixed buffer sizes).  I got through the initial 'no pixel rejection' integration after a couple out-of-memory failures, but now I can't proceed with pixel rejection.  After reading the subs in a half a minute or so from the cache, the console reports:

Integration of 1806 images:
Pixel combination .................. average
Output normalization ............... additive + scaling
Weighting mode ..................... custom keyword: SUB_WEIGHT
Scale estimator .................... iterative k-sigma / BWMV
Pixel rejection .................... Winsorized sigma clipping
Rejection normalization ............ scale + zero offset
Rejection clippings ................ low=yes high=yes
Rejection parameters ............... sigma_low=4.000 sigma_high=4.000 cutoff=4.000
Large-scale rejection clippings .... low=no high=yes
Large-scale rejection parameters ... lsr_layers_low=2 lsr_grow_low=2 lsr_layers_high=3 lsr_grow_high=4

* Available physical memory: 11.869 GiB
* Allocated pixel buffer: 319 rows, 8.894 GiB
* Using 12 concurrent pixel stack(s), 1.338 GiB


* Integrating channel 1 of 3:
  Analyzing pixel rows:     0 ->   318:   3%
*** Error: Out of memory
<* failed *>

There is a pause of a couple minutes after "analyzing pixel rows", and then the memory failure happens.  The amount of physical memory reported varies by a GiB or two each time I try this.  I have tried rebooting.  I had success once on the initial integration by setting fixed buffer sizes of 32 MiB and stack of 2048 MiB, but now that doesn't seem to help.

It seems like PI should never run out of memory; it should adjust its processes to use what's available.  Is this problem a function of the new automatic buffer sizes?

Can anyone suggest anything?

11
When I did that it looked like nothing happened, but ColorClip showed up in the Scripts / Utilities menu.  You may need to restart PixInsight to see this.

12
It works fine on 1.8.7 for me.  This was after installing the update from 20191002-windows-x64-core-c.zip, but that was a single bug fix and shouldn't have made a difference.

Can you describe what problem you're seeing?

13
Thanks for the report @niccoc1603.  I just downloaded 1.8.7 myself last night, and will look into this tonight.

14
Announcements / Re: PixInsight 1.8.7 Released
« on: 2019 October 01 11:18:20 »
Wow!  Fantastic work and a great, thorough explanation here.  I can't wait to give it a test drive!

15
General / Re: Superpixel = supergreen
« on: 2019 September 25 08:20:59 »
Thanks for this explanation and the help with my problem, Rob.  This improves my understanding of the linked/unlinked STF.

Pages: [1] 2 3 ... 5