disk space ...

[John_Gill]

Hi,

So I work thru my Bias, Darks and Flats and create the Masters.  Surely I will never have to work with the raw Bias, Darks and Flats again so I can delete them forever.  I suppost I can do the same for the Calibrated, CosmeticCorrected, Debayered and Registered images ...

Can I hit the delete button ....?

[vicent_peris]

Hi,

I would keep the registered and master folders. That will let you integrate new masters in the future without running everything again.

Best regards,
Vicent.

[John_Gill]

Thanks Vincent,

Now where is that delete button ..... Yah! more space for more images 

[Niall Saunders]

Hi John,

My personal preference is to save all of my RAW data - and, once I have created suitable 'masters' of each frame-type, I will ZIP the each of the folders containing the corresponding file groups (Biases, Darks, Flats, Lights, etc.).

Then, as time moves forward, and it becomes more and more obvious that I will never be likely to require those compressed folders again, I will usually dump these off my working drives, onto DVDs (although they are so unreliable, this action has to be questionable - it may even be more effective to rely on USB thumb-drives).

Basically, hard-drive storage is now 'cheap enough' that I don't overly concern myself. I keep an eye for offers of new, high-capacity, known-brand, hard-drives as retailers move their stock on towards SSD-type drives. I (mostly) buy in 500Gb drives, and have an external USB-connected drive caddy (capable of taking either 2.5" or 3.5" drives)  into which I simply drop an HDD for long-term backup.

Both my new desktop and my old desktop PCs have 4-bay quick-release caddies that carry 2.5" drives, and I managed to get my hands on some 40-odd 240Gb 'Crucial' SSDs from a local IT company that had been tasked with upgrading the entire PC fleet for a major oil company. Of the many hundreds of drive images that they cloned, these units failed whatever tests were asked of them, and were replaced under warranty - the IT company were literally going to throw the faulty units in the skip, but had to erase these first. They came to me asking if I had a simple method of bulk-erasing the data. I didn't, but offered to do the job for them in exchange for the blank drives themselves (once audited as having been correctly erased). So, I have more than enough storage space to keep me happy for years to come!

[John_Gill]

Hi Niall,

My laptop is nearly full, 2 x 2TB portable drives are at 75%, my Small Magellanic Cloud project is at 500GB, the Antennae project is at 150GB and growing fast .... Google drive is starting to complain, so I will email you the balance for safe keeping .... hahaha

Looking up (and around for more storage)
John

[Niall Saunders]

I will email you the balance for safe keeping

. . . joking apart, once everybody's Internet access speeds become 'usable' (mine are definitely not  ) then the idea of 'globally distributed storage' could become feasible.

A colleague and I already pay a reasonable subscription fee for a huge anount of offline 'cloud' storage and that benefits us because of mutual interest in the material being stored. However, whilst he has multiple locations that offer him >30Mb connection speeds, I can only get 20% of that (and even less again for 'uplink' transfers. That means that, virtually every time we meet, we are swapping huge USB thumb-drives - thus allowing me to get data uploaded into the cloud.

Like I said, keep your eyes on local adverts, etc. Pick up several smaller drives (very inexpensive nowadays) and store your data on these.

[vicent_peris]

Hi,

I have 10, 2 TB external disks for my data. Five of them at home, and five of them at my mother's home. Each pair of disks have the same data. That's my "personal cloud". ;-)

I store everything, from the raw to the master lights, and processing projects. That lets me a very quick access to any product, which has been very convenient to me in some moments. Of course, it's a lot of data... But those disks are cheap and you don't keep them running 24/7.


Best regards,
Vicent.

[msmythers]

In my system I use 2 SSD's for the OS, for working software and temp storage. I have two internal 3 TB drives for the daily working data usage but I use external drives for backup storage. I have 8, 3 TB drives that store everything I done or used in the digital world for the last 35 years. I also have enough extra space to grow for some years to come. All external drives are USB 3.0

Since I run Windows I use a cheap external USB 8 relay controller board to turn on and off the power to my external drives as I need them. I have a separate 12 volt, 15 amp power supply to power all the drives. It is more then enough to start up the drives all at the same time but that never happens. This system allows me to control the power of the external drives from my desktop or remotely via the desktop. Some drives might only get turned on once every couple of months, some 5 or 6 times a month. This has reduced the power on time of the drives tremendously.



Mike   

[vicent_peris]

Hi Mike,

Please could you share the model of the board? I would be interested to have something similar.

Thanks,
Vicent.

[msmythers]

Vicent

Do a search on eBay or even the web in general for '8 Channel USB Relay 12v Programmable Computer Control for Smart Home'. There are a number of different versions but I opted for USB A model as it can be powered by the same external power supply I'm using for the drives. I also have a 2 relay version that I use on a remote computer. It's powered strictly from the computers USB port.

Here is a link to the software and code for the controller. There may be better code out there now but this worked for me.
https://drive.google.com/file/d/0B13r3kYqTf8NTVBDeDZFZ1p2ekU/view?usp=sharing


Mike

[Niall Saunders]

Hi Mike,

I suppose that your system is much the same as mine (USB3-connected external drive bay) except that you can onviously get your PC connected to more than one drive at a time, and that you don't need to be next to the machine tp physically swap the drives.

Like you though, we both benefit from drives that more or less spend all of their time'powered down' - thus increasing their long-term 'spinning life'. However, perhaps the down-side is the possibility that a drive fails, catastrophically, on power-up.

[msmythers]

Niall

Power-up failure is a possibility but I've haven't had that type of failure in well over 10 years. Most failures I have or sectors failures or full read/write electronic failures. Main motor drive or head motor drive has not been a problem. I've had so many drives fail within the 3-4 year mark and they were all powered full time. That lead me to re-examine the what the failures were due to. I was having failures from all manufactures and different models. The only real constant was heat. My thought was if the drive is not spinning then it's not generating heat from the motor control IC, the motor itself or the heated air from the spinning platters.

My absolute most important data is backup over multiple drives. Small very important data is on as many as 5 drives. I have a monthly process I follow for backing up all my computers and it doesn't take more then a half hour. You just have to follow the process. Sound familiar. Gigabit wired ethernet and 5GHz Wifi in the house helps also.

You might like this, my computer is behind me and on the other side of the room. I attached a monitor swing arm to a table next to my recliner. Nothing like a 27" UHD monitor and a recliner. I like to process in comfort.


Mike