DT PixelFlow: Artificial Intelligence and Application

March 23, 2021 | by Hannah Storch

Artificial Intelligence (AI) has great implications for cultural heritage preservation. Gathering metadata for a collection can be time-consuming and labor-intensive, often involving the individual knowledge and experience of one person. Without this identifying descriptive metadata, valuable information can be lost and collections can remain incomplete. AI can be used as part of a metadata workflow to reduce the cost and tediousness of enriching a collection with enhanced metadata records.

Along with creating digital preservation-grade derivatives and deliverables, DT PixelFlow can use artificial intelligence to describe an item’s material, text, and image content. This type of descriptive data extraction allows not only for the leveraging of existing assets but also for the salvaging of descriptive metadata information before it, or the context required to create it is lost to time and memory.

Material

While the material type is a common metadata field, it is often automatically generated with more generic information, such as text or film. By implementing AI analysis, DT PixelFlow has the ability to automatically suggest the material type and item categories with greater depth of detail. This is a capability we are currently developing and exploring as part of the PA ArCHER Grant with Smithsonian Center for Folklife and our partners RIVERai.  The goal is to have DT PixelFlow automatically determine the types of documents, such as pieces of correspondence or legal briefs, and then further categorize them into groups such as memos, contracts, or letters. Learn more about the PA ArCHER Grant and its progress here.

We are very excited to explore this capability with other institutions and collections. If you have a collection you think would benefit from large-scale automatic material-type description using AI, with human QC, please contact us.

Text

Along with providing Optical Character Recognition (OCR), DT PixelFlow is able to provide a deeper analysis of the text and written content of an image to provide valuable contextual information. By using entity extraction, DT PixelFlow provides context to information that might otherwise seem like unconnected data, such as recognizing an address, formulaic greeting, or date from a string of numbers and text. Similarly, this type of entity analysis can find known entities such as proper names, which could enable an institution to successfully search for and gather together all of the images relating to a particular person or place.

DT PixelFlow’s AI analysis is not only able to recognize and transcribe the text within an image but also understand the conveyed sentiment and style of the text, interpreting the emotions, such as positive and negative or happy and sad, behind them.

These kinds of deeper analyses are set up on a project-by-project basis to ensure the analysis is relevant to the collection, the institution, and the stakeholders of the results. If you think your collection might benefit from AI analysis of the structure, content, or sentiment of the OCR’d text, please contact us for a consultation and we’ll help you understand what is possible and, just as importantly, what is practical.

Image

Artificial intelligence can identify objects and individuals inside of photographic or pictorial collections. With digital records, if this information is not extracted, cataloged, and linked to the image, this descriptive information can be lost to volume – obscured by the sheer scale of images one might have to look through manually to find given content. We can provide object detection and/or face detection in DT PixelFlow. This enables us to isolate and identify objects that are both in the foreground and less prominent in images.

DT PixelFlow also has the ability to identify and categorize general objects, locations, and information using keywords. This information can be general or tied to a specific institution or collection. For example, keywords for a collection of slides belonging to a natural history museum could have more refined and accurate metadata with keywords pertaining to that particular type of collection, location, or scientific study. Within the image, DT PixelFlow is able to recognize and extract both big depictions and small details, from the recognition of landmark features to facial features and human emotion. If there are specific individuals of interest, we can even train DT PixelFlow to automatically identify the faces.

Packaging

Once the descriptive metadata has been derived through AI analysis, it can be packaged in many different formats to make it more accessible to the user and institution. After it has been interpreted, the information can be embedded into the final image file, ensuring that the data is always linked to the image and that they can be updated together in the future, or output in other formats. For usability, this descriptive data could also be generated in a txt file, document format, or included in a new or existing spreadsheet. To learn more about how to make the most of your metadata, check out our recent metadata article.

Impact

Traditionally the accumulation of descriptive metadata has been a specialized as well as a tedious and labor-intensive process. With AI analysis and application, Pixel Acuity is able to maintain a high level of accuracy while increasing efficiency and accessibility, and as needed we can leverage our highly skilled staff to provide human QC on top of the automatic detection provided by our AI Combining our in-house software, our experience in cultural heritage collections, and our talented team, we are now able to assist in the preservation of collections through descriptive categorization and contextualization derived from AI research and analysis as well as digital surrogacy.

Contact us to learn more about our digital imaging services and how we can bring artificial intelligence to your workflow.