Friday, May 27, 2016

Legacy systems in government records

So yesterday got to chatting with FB friends about this article from the BBC: US Nuclear Force Still Uses Floppy Disks.

If you're anything like me, I'm pretty sure the first thing you thought of when you read this headline was Homer Simpson:

But this isn't just funny trivia for "news of the weird;" there is an actual point: digital records keeping is not as robust and cheap and easy as everyone thinks. That's because most (all!) of us think about digital records in the short term: my credit card payment receipts, my blog posts, my portfolio of html, whatever. Slap 'em on a hard drive, put 'em in GoogleDrive, done. And for most individual purposes, that is OK.

It is not OK, however, when you're talking about records of enduring value. We've probably all ALSO had this experience: "Oh, man, where is that file from 2002, the database of donors?" Oh, it's right here on this ZIP disk. Where's the drive? Um.

Or maybe you do have the drive still. Awesome! So you pop it in, rockin' right along, and then you realize all the files are WordPerfect. Or Lotus 1-2-3. Or VisiCalca. Or file extension .001, whatever that is. Do you still have that program? Heck, does it even still work with your current operating system?

Some libraries and archives (and museums) will keep legacy hardware (and OS and software) up and running so that you can still access historic digital records, but this is a cost-intensive approach, as the article about the nuclear floppies also mentions. Keeping all that old hardware up and running is expensive, especially once you have to do it all in-house, as support is no longer provided by the creators and replacement components may or may not be available.

None of this is insurmountable. You just need to migrate stuff, refresh media, backup like crazy, and so forth, right? Yeah, basically. It can certainly be done--barring any weird restrictions on where and how things can be stored and, especially, duplicated. You don't want to store stuff for the Dept. of Defense containing sensitive/classified info on cloud services mirrored on servers in other countries, for instance. And I think I've mentioned before, some state statutes specifically mandate what formats of certain documents can be considered legally-binding--if only paper and micro-formats, like fiche, are referenced as being an acceptable originals, you may not be able to use your PDFed versions for certain, usually high-stakes, applications. Changing file formats may also have implications for legal admissibility, as it will alter metadata and thus checksums, used to monitor e-records for changes and thus demonstrate maintained authenticity.

Again, this can all be worked around, but NONE of it is cheap. It requires tools (multiple secure and duplicate servers; maybe also cloud space if you can get the security up to snuff) and, most importantly, lots and lots of staff time. Frankly, that's true for both maintaining-as-legacy and converting, but current models suggest that converting to standardized formats, regular conversion, and avoiding physical media (CDs, disks!) ends up being cheaper in the long run, with less data loss. (Exception: converting from analog media, like microfilm, to digital media, has not been shown to be cheaper, for preservation copies. Your access copies can be digital, no problemo: they'll be subject to wear-and-tear, but that also don't have as many security and authenticity issues, because you can always backtrack to your security/preservation copy for, say, a certified copy, if needed. That's why a lot of archives STILL keep master copies of permanent-retention records on film or fiche.)

And so, the US Nuclear Force is still using those floppies. The Treasury, also referenced in the article, is using software written in assembly language code, and thus tied to specific (now antiquated) hardware. State governments are working in obsolete or on-the-cusp software that can't play nicely with modern OS and that are not accessible to constituents. Email retention policies are either non-existent or not understood, enforced, and supported (as in the Hillary Clinton email scandal, which apparently really also involves all former Secretaries of State, as well).

This means two things: our government is not working efficiently, and what should be open records made available to any interested constituent may not be being preserved in any accessible or meaningful format. And I'm not saying this to criticize or ridicule the government or, still less, its employees. In my experience, government employees are doing their best, but in this era of increased emphasis on "smaller government" and cutting budgets, the government and its workers are being less and less empowered to obtain the tools they need to do their jobs. If you can't afford to replace your DOS-based system from the 1980s, your office keeps limping along and trying to provide services even though you can't even run the darn program on a computer with fast (by current standards) processing speeds. If you're an archive whose budget is tied to selling photocopies (cough cough), it's frankly going to be hard to make enough money to buy or rent server space to store e-records on. And if you have nothing to store e-records on, you just plain don't retain e-records. How can you? So those documents, and the associated government accountability and transparency, goes out the window.

Don't get me wrong. I'm not lobbying for, you know, free caviar for all government employees ever Friday or photocopiers with built-in espresso machines for all government offices. But if you want the product, you have to give the workers the necessary tools. Archives budgets in particular are among the first impacted in any round of budget cuts; people need to realize that that means that they simply can't provide an adequate level of service without some very basic necessary tools. No server; no emails; no transparency. And that's bad for all of us.

No comments:

Post a Comment