Diving deeper on External Snapshot Backups
In Don’t be the turkey // a “belt and suspenders” approach to Personal Digital Asset Management, I presented my Personal(💜🌈) Digital Asset Management (“PDAM”) workflow using Lightroom Classic + Dropbox + periodic external backups, along with the “Cheesy Rainbow Rave” framework to achieve a “belt and suspenders” approach to risk management. Here below we will dive deeper into external “snapshot” backups, which provide additional redundancy within this framework.
💜🌈 // It is appropriate to include the modifier “personal” to distinguish the tools we are discussing here, since the term Digital Asset Management (“DAM”) typically refers to enterprise-level solutions that catalog large volumes of photography assets, as well as managing permissions/access across a large set of users.
As my asset library has grown larger, its total volume actually exceeds the available storage capacity of my local system drive. Thanks to Dropbox’s selective sync feature, I can hive off certain assets from my local drive, such that they remain consolidated (as placeholders) within the file system on my local drive (within my Dropbox folder), but their contents are only pulled down from Dropbox-cloud when/as needed. I try to limit the hiving off to less critical assets, and/or heavier ones (such as video — this is the reason I have separate top-level folders for my video assets). Furthermore, I am careful to make sure that the assets hived off from my local drive have been redundantly backed up in one or more external snapshots.
As such, Dropbox-cloud in fact constitutes the only single fully comprehensive copy of my asset library, since the copy on my local drive is not complete (and Lr-cloud is at best partial). Therefore, I must protect against the possibility that Dropbox could be wiped out by a black swan (figure 2). To borrow a term from the world of finance, we could think of this as “counterparty risk”. Cloud providers have redundancy built-in to their networks such that, if one of their server farms went down, your data is (probably?) not lost. But it would be foolish to ignore the possibility that Dropbox itself could blowup entirely — pretty unlikely for sure, but it could happen (eg. nuclear war, or an intense solar storm — see below).
Moreover, there’s also what you could call “account risk” (within the “counterparty risk”)… eg. there’s an urban legend of some guy who merged his Google accounts and lost all the photos he had been storing for years in Google Photos. That’s definitely a potential risk (especially with Google I think — as mentioned above, I dislike the complexity and lack of transparency of the Google ecosystem). I feel it’s probably more of a risk with the “one-stop-shop” photo products like Google Photos that you don’t realize is somehow dependent on your Google account — as opposed to data stored in Dropbox-cloud as a literal file/folder structure, transparently mirroring what one has on the system drive… probably less likely to accidentally lose access to the latter through some unintended account change since cloud storage is the core product offered by Dropbox — as opposed to Google Photos, which is just an ancillary product within a massively complex ecosystem.
To protect against such risks (however unlikely), I therefore make periodic “snapshot” backups to external solid state drives (“SSD”). Let’s unpack this…
Snapshot
I make a backup to the external drive, and then I seal it and never touch it again. I’m wary of routine, continuously connected external backup systems (such as Apple’s Time Machine), since one of the risks one needs to protect against with an external drive is that the entire system could be compromised by malware/virus, which could then easily infect the connected external drive. So physical and temporal separation seems important to me.
Periodic
I make such external snapshot backups approximately once per year.
Gap Risk
Since the snapshots are only periodic, there would admittedly be some residual “gap” risk, ie. data loss in the time interval since the last snapshot. But, since I already have triple redundancy in place with Dropbox, the external drive is more of a last resort in case of a catastrophic loss of data integrity across the other 3 systems.
Partial
In the past, I used to make periodic snapshots of my entire photo collection. As my asset collection grows larger however, and as I have begun to accumulate more 4K video, my PDAM workflow has evolved. I no longer keep redundantly making new external snapshots of my entire asset library including older assets, but rather only of the last year or two of new assets — each annual backup would include all the new assets since the previous backup, plus redundant extra copies of prior years (for extra redundancy), as space permits to fill up the drive. So, in reality, the complete backup of my asset library might be spread over several SSD disks, but I keep these all in one single, waterproof and shockproof Pelican case (figure 3).
Solid State Drive
I don’t know if you had the same experience: back in the aughts, I had to replace my computer system every few years because, after 2–3 years, it started to run much slower and just stopped working properly. Ever since hard drives became commonly replaced by SSD about a decade ago, computers seem to last a lot longer. Since hard drives are mechanical devices with moving parts, it only makes sense that degradation would occur over time — especially with laptops that get moved around a lot. I have a couple of Macs with SSD system drives that are pushing 10 years old and they still work perfectly fine. So while this is admittedly anecdotal, I think SSD drives are much more robust than hard drives.
As with any new technology, for the first few years SSD drives were continually increasing in capacity and lowering in price. Recently however, the specs and pricing seem to have stabilized. A quick search on Amazon and B&H reveals (figures 4+5) that drives I purchased last year and even 3 years ago are still in the lineup. The size of drive depends on your asset pipeline, but for me 2 TB per year for around $150 gets the job done. Since I am purchasing these drives for single use snapshots, it’s not necessary to pay more for faster bitrate.
Other considerations
According to the old “3–2–1 rule for backup”, a couple of blind spots remain here notwithstanding. To properly protect against a far-fetched cataclysmic loss, I should be keeping external backups >>
- in different physical locations
- on CD/DVD/BD optical discs, not SSD
SSD drives could be significantly affected by a solar storm or nuclear war, primarily due to the potential for electromagnetic pulses (“EMP”) which can fry electronic components, including those in SSDs, if the intensity were strong enough, thereby causing data loss or complete drive failure. Meanwhile, a standard CD/DVD/BD itself is unlikely to be directly damaged by the EMP generated in such events, as optical discs are generally considered more resilient to EMP compared to electronic components like transistors in hard drives or RAM chips. Key points to consider:
- EMPs >> Both solar storms and high-altitude nuclear detonations can generate EMPs, which are surges of electromagnetic energy that can induce currents in electrical circuits, potentially damaging sensitive electronic components like those in SSDs.
- Severity depends on intensity >> The level of damage to an SSD would depend on the intensity of the EMP, with a very strong pulse potentially causing complete data loss.
- Radiation hardening >> Some specialized SSDs designed for military applications are “radiation hardened” to withstand higher levels of radiation, making them more resistant to EMPs.
- EMP resistance of optical media >> The physical structure of CD/DVD/BD discs and their data storage mechanism are not easily affected by an EMP, making it more likely to survive such an event than electronic components within a computer.
- Protection measures >> Storing your CD/DVD/BD discs in a Faraday cage could further minimize any potential risk from an EMP.
- Storage capacity >> Traditional CD/DVD’s have relatively limited storage capacity, but newer types of BD-R and M-Discs offer more. I’m just now learning more about this, in the meantime I would refer you to Daniel Rosehill Why Use Optical Media For Data Archiving In 2024? An FAQ. That being said, even with 25 GB BD-R discs, it’s still going to take hundreds of discs and a lot of time to backup my asset collection that’s tracking on the order of 10 TB. BD-R XL 100GB discs sounds a bit better, but it still sounds like a pain.
If this all sounds crazy, just remember the inductivist turkey — everything’s fine, right? …until it’s not. Most of us think nuclear war would be pretty unlikely, but is that only because we’ve never experienced one? For a sobering reality check, I recommend reading Ray Dalio’s Principles for Dealing with the Changing World Order: Why Nations Succeed and Fail, or his more recent posts, such as Pick A Side And Fight For It, Keep Your Head Down, Or Flee (June 25, 2024).
Either way, as explained in Don’t be the turkey // a “belt and suspenders” approach to Personal Digital Asset Management, a proper understanding of risk means not only identifying the likelihood (or unlikelihood) of different scenarios, but also understanding whether you can tolerate the cost of certain negative “tail” outcomes. It’s up to each of us to decide what risks we are willing to tolerate when it comes to our digital assets.
Please feel free to reach out with any questions or comments >>
