Technical Art History Series: Digital Tools for Cultural Heritage: Unlocking Art History

The trunk from Postmaster Simon de Brienne and his wife Marie Germain (Figure 1), contains around 2,600 letters from all over Europe send to addressees in The Hague between 1689 and 1706. The letters in the so-called Brienne collection, held at Sound and Vision The Hague, in the Netherlands, were never delivered. They were all ‘locked’ through a complex folding method, and 600 of them were never opened due to their fragile state. However, in 2021 an interdisciplinary team from experts across the world were able to virtually unfold a letter and thus unlock a piece of history.

Figure 1 Trunk from postmaster Simon de Brienne (Sound and Vision, The Hague)

The online Technical Art History Series: Digital Tools for Cultural Heritage, aims to stimulate scientific and interdisciplinary discussions on cutting edge research in computational imaging methods and sensing technologies, combined with art history and digital humanities, applied to Cultural Heritage. The series bring together researchers from the humanities and sciences, presenting their research to allow an open forum where advanced knowledge on some of the most intricate and fascinating problems in cultural heritage is shared, and opportunities for interdisciplinary collaborations are promoted. In related blog posts, we will reflect on each session. The series is organised by the Rijksmuseum, the Computational Imaging Department from the CWI, Amsterdam, together with the Venice Center for Digital and Public Humanities, University Ca’Foscari, Venice.

The first group of speakers, who presented their work on May 11th 2021, is part of a multidisciplinary team of researchers who joined forces to establish a novel method capable of revealing the content of letters from early modern Europe, ‘locked’ using specific folding methods and hence many of them remaining unread for centuries as they are too fragile to unfold. Their findings, based on cutting-edge x-ray technology, and a new computational algorithm, are published in a recent Nature Communications article ‘Unlocking history through automated virtual unfolding of sealed documents imaged by X-ray microtomography.’

In the lecture, Jana Dambrogio, Thomas F. Peterson Conservator for Massachusetts Institute of Technology (MIT) Libraries, describes Letterlocking as ‘the practice of folding and securing a writing substrate to become its own envelope.’ Jana explains that before the invention of the gummed envelope in the 1830s, almost all letters were sent using letterlocking: ‘Letter correspondence was almost entirely relying on letterlocking, and it was used by all kinds of people, not only the rich and powerful. Letterlocking made global communication possible, as fundamentally as computer coding does with e-mail today.’ During our session, Jana and her collaborator Daniel Starza Smith, lecturer in Early Modern English Literature at King’s College London and co-director of the Unlocking History research group, gave a hands-on tutorial on how to make a locked letter yourself in which we all participated (Figure 2), following the same patterns as shown in Figure 3.

Figure 2 Step-by-step illustration of the locking sequence shown in Figure 2, made by following the indications of Jana Dambrogio during the hands-on tutorial
Figure 3 Locking sequence of virtually unfolded letterpacket DB-1627 in the Brienne Collection, Sound and Vision The Hague, The Netherlands (UH0217)

David Mills, imaging physicist working at the Dental Institute at Queen Mary University of London, who started collaborating with the letterLocking Team in 2015, explains how in 2006 the question was asked: ‘Can you see ink on parchment using your X-ray scanner?’, and how the answer was yes, and more besides.’ During the lecture he discussed the need to use 3D CT scanners to detect multiple and overlapping ink texts and, importantly, he emphasised that ‘there is no systematic change to the collagen chemistry/structure that is an effect of X-Ray microtomography radiation used to read documents.’ Radiation damage was only encountered after CT-scanning soaking wet parchment. But, as he concluded: ‘this is not the type of material we would scan anyway.’ The CT scanner used at Queen Mary University of London is a MuCAT-2 X-Ray machine composed of an X-ray generator and a camera in a box which can slide across. The object is placed in between. Each time the camera moves, it records an image, then the object is rotated, and another image is acquired. The process is iterated until multiple thousands of images are recorded. Layers of paper can thus be visualised, as well as written text if the writing material contains metals, for example if an iron gall ink was used.

CT-scans of the ‘locked’ letters demonstrate that Davis Mills could see the multiple layers of the folded paper, and see some of the writing. Cross-sections from the CT scan of letters from the Brienne Collection are shown in Figure 4. However, due to the intricate folds and the presence of several layers of paper, they were not able to fully read the letters’ content, ‘until we met Holly [Jackson] and Amanda [Ghassaei] at MIT’, he admits.

Figure 4: Courtesy of the Unlocking History Research Group archive. The letters are from the Brienne Collection, Sound and Vision The Hague, The Netherlands

Holly Jackson, a BA student in electrical engineering and computer science at the Massachusetts Institute of Technology (MIT), and Amanda Ghassaei, researcher at Adobe Research, co-developed the virtual unfolding algorithm. Using the 3D CT reconstructions from David, Holly and Amanda implemented an algorithm to turn David’s input into a 2D reconstruction of the text (Figure 5).

A crucial aspect in this research project is that the letters could remain closed in real life. The main steps of their method include segmentation (i.e. separating text from the surrounding environment), unfolding (i.e. transform the 3D segmented regions into 2D embedding), hybrid mesh propagation to complete the full reconstruction of a letter both in 2D and 3D, and texturing, which aims to reconstruct the actual text of the letter packet.

Figure 5: Computer-generated unfolding sequence of sealed letter DB-1538. In their paper, the authors describe how “virtual unfolding” was used to read the contents of sealed letterpackets from 17th-century Europe without physically opening them. Courtesy of the Unlocking History Research Group archive. The letters are from the Brienne Collection, Sound and Vision The Hague, The Netherlands

Not only can they reconstruct the text, but also obtain curvature information, which allows the generation of a crease pattern map, which is useful to understand how the document was folded. Without the virtual unlocking, the letter would need to be unfolded in order to read it, and information on the folding pattern could be lost. They demonstrate the algorithm on four letter packets from the Brienne Collection, three of which are still sealed and closed today. One of the group’s collaborators, MIT Professor Erik Demaine, is an expert in so-called computational origami. Holly explains how Prof. Demaine anticipates that the virtual unfolding pipeline can be extended ‘to generate crease pattern maps of origami art from origami masters of the past, to verify the folding patterns of their art without damaging the art-piece itself.’

‘The field of letterlocking emerged originally from the material study of early record keeping, and it has influenced modern conservation studies’, continues Daniel. Using the imaging methods presented will protect these fragile documents while still revealing their contents and material features. These findings pave the way to novel interpretations of Early Modern portraits, engravings, and other images featuring locked letter packets, and locked letter packets appearing in novel and poetry.

Dzemila Sero, Francien Bossema and Erma Hermens from the IMPACT4Art team

The session took place online, which did not impede an enthusiastic participation in a letter locking demonstration!

A big thank you to Elisa Corrò from the Venice Center for Digital and Public Humanities, University Ca’Foscari, who organised the online session.

  • The recording of the first session is available here:

Preview of the Second Session The second session will feature two complementary talks on the Ghent Altarpiece, Hubert and Jan van Eyck’s masterpiece. Aleksandra Pizurica, professor in statistical imaging modelling at Ghent University, will discuss about artificial intelligence in art conservation and preservation applied on the Ghent Altarpiece. In the second lecture, Geert van der Snickt, tenure track professor at the Conservation-Restoration department, Antwerp University, will demonstrate how non-invasive macro scale imaging experiments had a direct impact on the choices made during the conservation treatment of the Ghent Altarpiece.

Stay tuned!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s