Home > computer/data > data storage

data storage

MD5 Checksums bring peace of mind

If you ever worry about a bit error happening to your files and not finding out about it, you should use MD5 checksums (or some similar method) to be able to verify that the file has not changed.

The general theory behind a checksum (or “Message Digest” = “MD”) is that it provides a unique 128-bit number for each and every file, based on its content. If one bit changes, the MD5 checksum (sometimes called “hash”) changes. The checksum is repeatable, does not permit discovery of two different files that produce the same checksum, and is non-reversible (i.e. you can’t create the content from the checksum).

Read more

Hard disk formats for interchange

The question of how to format hard disks (i.e. what file system to use on them) for easy interchange is another FAQ. A recent experience brought home the fact that it is more complex than one might hope. The computer industry is headed towards universal readability, but it is not there yet. The most-able-to-be-read-and-written format appears to be FAT32, although my friend Eric Jacobs makes the point that NTFS is a more robust hard disk file system, and I have to agree.

Read more

Digital storage file systems and topologies

Once again, an interesting post on Jill Hurst-Wahl’s Digitization 101 Blog. She started by discussing tape backup issues. In the comments, I discussed my solution of using multiple spinning disks. Another commenter, Ike, provided an extensive review of file system options and his opinions on what works (and doesn’t) for long-term storage. Ike’s comment is fascinating and has lots of food for thought. Here is the post.

Read more

File backup strategy and utilities

My storage systems have grown to keep up with storage needs. I am currently running two NAS units in RAID-5:

Unit #1 for client audio projects is a Netgear ReadyNAS NV+ with four 1500 GB drives, providing about 4.3 TiB of storage.

Unit #2 for personal projects and general data is a Thecus N5200 Pro with five 1000 GB drives, providing about 3.6 TiB of storage.

These two units are then duplicated off-site and connected by a fibre optic link (currently running at 100 Mb/s while the rest of the network is running at 1000 Mb/s).

Read more

40th Anniversary of Apollo 11 Moonwalk and the loss of data

There has been much discussion over the last few years about finding and saving original tapes of the Apollo 11 Moonwalk videos downlinked from the moon. There is also an exciting project going on to restore and digitize the Lunar Orbiter tapes from more than 40 years ago.

The Lunar Orbiter tape digitization folks have just posted a commentary that bears reading by all archivists who are holding tapes. You may link to it here. The main site is www.moonviews.com

NASA, in their press conference yesterday held at The Newseum, admitted that the original 14-track 1-inch instrumentation (IRIG) tapes that contained the slow-scan video direct from the moon were most likely recycled and reused for later missions. Apparently, over 350,000 reels of instrumentation tape were recycled by NASA over time. No one apparently thought to preserve the 45-odd reels of the original moon walk.

Read more

Another NAS contender…

revised 9:15 AM to add Synology

I was reading some additional NAS discussion on a photography list serve and was introduced to yet another NAS vendor that deserves consideration, QNAP. They appear to have wide distribution in North America (I found two online retailers that list them, CDW.CA and CanadaComputers.com. While the comparison was made to the Drobo products in the discussion I saw, these products appear to be similar in pricing and quality to Thecus and Netgear and offer some interesting options. Thecus units are more difficult to obtain, but Netgear units are also available at the two online dealers listed for QNAP. Yet another name just came up in the discussion, Synology. QNAP, Synology, and Thecus are all headquartered in Taiwan, while Netgear is headquartered in the U.S.

Read more