Author Topic: How Does Everyone Store their Data Long-Term?  (Read 292 times)

Offline brent1123

  • Newcomer
  • Posts: 6
    • View Profile
How Does Everyone Store their Data Long-Term?
« on: 2019 February 23 13:54:26 »
Prior to using PixInsight, each of my projects, once completed, would include my Lights and Master Flat (Darks and Bias kept in a library), original stacked image, and the ~5GB Photoshop File which contained all my edits for future reference, totaling perhaps 10GB or so

Now that I use PixInsight, I keep the same original files and Master Flat, unedited stacked image(s), but the Project file for a standard LRGB image is now reaching about 80GB in size, not including all of my field data and calibration images.

I've taken to screenshotting all of my processes and putting them into Word documents detailing the steps I took for each image and then and saving my masks, followed by deleting the project file when complete. Is there a better way to store my data? I know storage is cheap and one can easily buy several TB's worth of backups, but I'd still like to reduce what I need to keep both for going back and referencing how I edited my images in the past, and also so I can easily re-edit my images in the future when I've found better methods to do so.

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4103
    • View Profile
Re: How Does Everyone Store their Data Long-Term?
« Reply #1 on: 2019 February 23 14:59:45 »
i guess you could save process icons for each processed image in the project onto the PI desktop and then delete the images themselves, then save the project. in theory if you saved all the process icons for calibration/staralignment/integration too then you could replicate everything in PI without having to write anything down, as long as you saved all of your raw images from the camera...

rob

Offline tchitty

  • Newcomer
  • Posts: 6
  • PixInsight Convert
    • View Profile
    • My Astrobin Page
Re: How Does Everyone Store their Data Long-Term?
« Reply #2 on: 2019 February 24 15:18:10 »
I store all my raw FITS files straight out of the camera and have (2) 2TB external USB 3 drives I store copies on. I don't store process icons, but every separate image is a PixInsight project from the get-go. I'm hopeful that my processing skills will improve and I can have a go at the raw data again.

TomC

Offline chris.bailey

  • PixInsight Addict
  • ***
  • Posts: 225
    • View Profile
Re: How Does Everyone Store their Data Long-Term?
« Reply #3 on: 2019 February 25 00:45:09 »
I work on projects on the local drive but I save the originals and the calibration masters to a RAID NAS and back that up twice to USB drives.

I prune out project files after a while on the basis that if you go back and re-process old data, there will be some new PI tricks to use so better to start from scratch.

Offline andreasmax

  • Newcomer
  • Posts: 29
    • View Profile
Re: How Does Everyone Store their Data Long-Term?
« Reply #4 on: 2019 March 08 13:01:02 »
using Adobe Lightroom for managing all my images. i store them on a 2-bay Synology (raid 1) with attached usb backup harddrive for security.
when processing i export the images to some folders:
L for the lights
D for the darks
F for the flats
BIAS i use preprocessed MasterBIAS stored also an my raid

cheers,
max