Working with Zip Files

Storing Zip files inside a digital preservation system such as Preservica is generally not seen as good practice.

Its generally better practice to unzip the file before ingest and store each individual file, these can then be identified and preserved individually.

The reasons are manyfold

• The contents are hidden. Format identification (e.g. DROID), fixity checks at file level, validation and metadata extraction only see “a ZIP”. You can’t tell whether it holds PDFs, obsolete WordPerfect files or executables.

• Preservation actions can’t reach the files. You can’t migrate a format or plan for obsolescence on individual files you can’t see.

• One damaged byte can cost you many files. Corruption in the central directory or a compressed stream can make several files, or the whole archive, unreadable. Uncompressed files are far more resilient to errors.

• It’s harder to find and use. You can’t describe, search, render or give access to individual items.

Sometimes you will need to store a zip file, for example the file is ZIP underneath but has its own documented format, such as DOCX, XLSX, EPUB and ODF.

The Zip format is used frequently as a package transport format, for example zipped Bagit packages etc. The Zip format here should only be used for transport and delivery, never as the stored archival format.

Having said all that, sometimes Zip files cannot be avoided, when the Zip is the record itself, for example a software release, a dataset exactly as published, or evidence where the original package matters.

In the case where a Zip file has been stored, accessing the individual files inside the Zip stored in Preservica requires downloading the entire Zip file and extracting its contents to get to the file of interest. For large Zips this could mean downloading many GBs of data to access a small file.

pyPreservica has now added some additional functionality to allow users to interact with Zip files without having to download them.

The first is a call to allow users to list the contents of a Zip file,

from pyPreservica import *

client = EntityAPI()
asset = client.asset("9fd239eb-19a3-4a46-9495-40fd9a5d8f93")
for bs in client.bitstreams_for_asset(asset):
    for name in client.bitstream_zip_names(bs):
        print(name)

This call will print the name of every Zip entry to the console without having to download the Zip file. You can use this to quickly check the contents of the Zip. This call is very quick as only parts of the Zip containing the header information is accessed.

Once you have the name of the entry you need you can use it to extract it directly from the Zip file stored in Preservica.

For example, if you have a very large Zip file containing multiple video files and a small XML mets file, if you only need access to the XML, you can extract it locally using:

from pyPreservica import *

client = EntityAPI()
asset = client.asset("9fd239eb-19a3-4a46-9495-40fd9a5d8f93")
for bs in client.bitstreams_for_asset(asset):
    client.bitstream_zip_content("mets.xml")

This will write the contents of the file into a local file called mets.xml.