The Royal Society

Science in the Making

London, UK

Putting a quarter of a million images of 30,000+ historic manuscripts online for the first time
Creating high-quality search and inspection tools for Historians of Science to support new discoveries
Revealing the interconnections among people and objects through an elegant interface, enriched with detailed metadata sourced from across the web

The Royal Society is a learned society for the promotion of scientific knowledge, based in London. It was founded in 1660 by a group of natural philosophers, including Robert Boyle, John Wilkins, and Christopher Wren. The Society was granted a royal charter by King Charles II in 1662, which gave it the title of “Royal Society of London for Improving Natural Knowledge.”

Throughout its history, the Royal Society has been at the forefront of scientific discovery and exploration. Its Fellows have included some of the most famous scientists and thinkers in history, such as Isaac Newton, Dorothy Hodgkin, and Stephen Hawking.

“The Society’s fundamental purpose, reflected in its founding Charters of the 1660s, is to recognise, promote, and support excellence in science and to encourage the development and use of science for the benefit of humanity.”

The Royal Society mission

In close collaboration with the Royal Society team, Cogapp developed Science in the Making. Science in the Making is the culmination of over eight years of work by upwards of 25 contributors. Its purpose is to make the full collection of archival materials related to the Royal Society journal article collection, published from 1665, freely available online. These archival materials consist of high-quality scans of the original items held in the Royal Society’s physical archives. Many of these original notes, illustrations, and manuscript letters are now available online for the first time.

Building a better online world for audiences

A powerful search

To help facilitate discovery, at the core of the site is a powerful, responsive search experience. Whether you are an academic researcher or an interested member of the public, it is designed to make the searching process fast and intuitive. Items can be filtered by a number of facets, including format — while most items in the archive are written manuscripts, the collection also includes hundreds of original diagrams and illustrations, some of which include details not found in the published version.

Screenshot of the histogram interface used for filtering search results by date: a modal overlay, with the title 'Filter by Dates active' shows a graph ranging from 1561–2020, split into ten year sections; the graph peaks around 1900 and trails off towards the left and right. — Intuitive date filtering using histogram interface

To aid researchers focused on a particular time period, when filtering items by date, a histogram interface provides a visual representation of the distribution of search results over time. Users can easily see the number of results for each year and identify patterns or trends in the data. The pages for individual Fellows also include a histogram of their contributions over time.Each item is accompanied by metadata from the collections catalogue that can be used to facilitate continuing academic research. We also surface connections between records and people wherever possible; we hope these new perspectives will aid researchers and lead to the discovery of new relationships between people, places, research, and materials.

Deep-zoom imagery

A key requirement of the project was to showcase these historical materials in an intuitive, visually appealing way. In total, there are over 250,000 images of archive items, some of which are over 350 years old.

Every detail of the high-resolution imagery can be zoomed into using our custom International Image Interoperability format (IIIF) viewer, built upon the open-source OpenSeadragon viewer. The core experience is enhanced with extra functionality specifically designed to encourage exploration and aid discovery. Browsing individual archive materials, some of which amount to hundreds of pages, is made simpler with a filmstrip of thumbnail images, as well as a grid view. The viewer also includes an image adjustments panel, where users can modify contrast, saturation or brightness to uncover subtle details, and rotate images to better read marginalia and other notes.

The interface of the deep-zoom viewer: the main image is a highly-detailed greyscale map, showing the Peak of Tenerife from above. Below the main image is a strip of thumbnails showing other images from the Tenerife Papers of Charles Piazzi Smyth. — Deep-zoom image viewer showing watercolour map of the Peak of Tenerife

Another screenshot of the deep-zoom viewer: the main image is a painting from the Tenerife Papers of Charles Piazzi Smyth captioned 'The cloud horizon Westward from Guajara, showing the summit of Palma above, and the base of Gomera below the cloud'. The image is rotated 90 degrees clockwise to make a note in the margin easier to read. — Deep-zoom image viewer with the advanced image manipulation tools active

Embeddable branded viewer

To promote wider engagement with these unique materials, we built a custom shareable version of the viewer for embedding on other websites. This incorporates the Royal Society branding to maintain consistency and links back to the Science in the Making site, encouraging visitors to further explore connections between items.

An embedded version of the image viewer showing the bound cover of a book. The caption reads 'Catalogue of nebulae observed by William Herschel, compiled by Caroline Lucretia Herschel'. There are buttons for making the viewer full-screen and navigating between pages. — Embeddable version of the image viewer showing a catalogue of nebulae observed by William Herschel

A unique resource for historians of science

The primary audience of the site is intended to be historians of science and their peers. To support these users, we created features targeted towards those doing academic research:

PDF files containing scans of every page for an item, including a branded cover sheet, are available for printing and offline study
Every item page includes one-click copy functionality for academic citations, page URLs, and embed codes.
For items with related publications, there are links to the Royal Society’s journals site where users can download the full article, view metrics of its popularity, and explore other articles which cite it.
For users who may prefer to use one of the many powerful tools which support IIIF, we generate a manifest file for every archive item. A link to view the item in Mirador viewer is also provided which lets users compare items from Science in the Making alongside other IIIF-enabled collections.

The IIIF image viewer, Mirador, which shows two documents: on the left is a painting from the Tenerife Papers of Charles Piazzi Smyth and some metadata about it in a panel; on the right is a sepia-toned stereosopic photograph of the Peak of Tenerife from the Getty collection. — Here, an example from the Science in the Making collection is shown in Mirador viewer, along with with an example from the Getty collection.

Left above: Charles Piazzi Smyth, Tenerife Papers of Charles Piazzi Smyth, 1856-1857, MS/626, The Royal Society Archives, London, https://makingscience.royalsoc..., accessed on 20 April 2023.

Right above: Charles Piazzi Smyth (British, 1819 - 1900), photographer Peak of Teneriffe, from Orotava, on the northern coast, 1858 The J. Paul Getty Museum, Los Angeles, 84.XB.1182.3.

Engaging the science-curious public

In recent times it has become more important than ever that the general public feel included and engaged in public conversations around science. Science in the Making is a unique resource that illustrates the birth of the scientific method and plots the impact that it has had on the world.

As such it was a requirement of the project that the collections be presented in such a way as to pique the general public’s interest in the history of science and how this history informs both our present and future. To support this requirement, the site includes an ‘In focus’ section, designed to highlight some of the interesting stories that can be explored using the Science in the Making items.

A screenshot from the 'In focus' section of the site site, showing a deep-dive article about nebulae. A colourful photograph of a nebula takes up most of the screen; it is partialy covered by the title 'A nebula of papers'. — The 'In focus' section of the site allows for long-form narrative pieces supported by the collection

Using the Craft CMS content management system, editors can create editorial content that is engaging and friendly to a wider audience. The flexible Matrix field type in Craft was employed to create a library of reusable, reorderable content blocks. As well as standard functionality like text, images, and embedded media, editors can also include links to featured archive items and Fellows; there’s also an option to embed a deep-zoom viewer for a chosen archive item.

The Royal Society felt it was important to highlight the problematic nature of some of the material included in the archives and provide broader historical context for archive items. Early studies weren’t held up to the same ethical guidelines and regulations which govern modern scientific research. As such there is material in the collection containing offensive language, negative stereotypes, prejudiced scientific concepts, unethical experiments and disproven hypotheses.

The ‘In focus’ section of the site uses these materials to explore the origins of contemporary racism, sexism, and other discriminatory attitudes. It is also being used to shine a spotlight on people, especially women, whose achievements were overshadowed during their lifetime. One example is the story of Caroline Herschel, whose important work in astronomy, including the discovery of several comets, nebulae, and star clusters, was often misattributed to her brother, William Herschel.

Building a better online world for the Royal Society

Technical approach

The site infrastructure consists of three main parts:

An Elasticsearch datastore
A IIIF image server
A Craft CMS backend

These are supported by Cogapp’s own Honeysuckle system to harvest and process collections data, and a custom image pipeline using Cogapp Refinery Services, our scalable cloud-based image and data processing tools.

We integrate closely with the Royal Society’s archive management system, CALM, enabling the Science in the Making project team to easily add material, make changes and access usage data using a system they are already familiar with. When item and people data is harvested from CALM, it is stored in Elasticsearch to ensure the large dataset can be quickly and efficiently searched, filtered, and sorted by users.

Whenever a new archive item is scanned by the team at the Royal Society, they simply upload the images to an AWS S3 bucket using a web browser. These images are then automatically processed by a Lambda function which does two main things: firstly it converts the images for use by our IIIF server and secondly, it registers the existence of the image in an Elasticsearch index. The image is then immediately ready for display on the website. You can view more about how the site utilises IIIF in this lightning talk given by Anne McLaughlin at the Royal Society and our Technical Director, Tristan Roddis.

The website, built with Craft CMS, communicates with the other parts of the system via a custom module. Thanks to Craft’s highly extensible module system based on the PHP framework, Yii, it was easy to display information about any archive item or person throughout the site. Craft’s flexible field types made it simple for content editors to embed previews of both items and people within long-form articles.

We built the frontend of the site iteratively, starting with a prototyping phase. Using the utility-first Tailwind CSS framework, we were able to build a basic but visually appealing version of the site quickly. The project coincided with a rebranding process for the wider Royal Society, so this approach enabled us to get feedback from stakeholders and start thinking about the best way to present content. Once we agreed on a branding direction, we could easily update the site with the correct colours, fonts, and graphics to make it feel like a cohesive part of the Royal Society’s online presence.

Enriching collections data from external sources

The Royal Society’s CALM system provides the core data for archive items but we wanted to provide as rich an experience as possible. This led us to build an innovative data harvesting system which fetches data from third-party, community sources to enrich the material held by the Royal Society: Crossref is used to retrieve publication titles, authors and dates of related publications; we developed a process to include Wikidata images to enhance pages related to individual Fellows; in this case Wikidata provides portraits of the Fellows themselves.

These are just a few of the ways we can provide a richer experience without the need for extra work by the Royal Society team — in addition to filling in any gaps in the data from CALM, there is potential in the future to filter and facet on data not in the original dataset. For example, using Wikidata we could link Fellows who studied at the same institution at the same time or expose student/teacher relationships.

A screenshot of the 'Person' page for Scottish scientist D'Arcy Wentworth Thompson, showing a portrait, and a list of biographical information including: dates of birth and death, nationality, gender, and date of election for Royal Society fellowship. There are also links to his Wikipedia page and the Royal Society catalogue. — This D'Arcy Wentworth Thompson page includes this headshot from Wikidata harvested using the Virtual International Authority File (VIAF) ID.

Creating an environment where everyone can do their best work

At Cogapp we believe that a supportive and creative working environment produces the best results all round; for our clients and also for our team. A great first step towards fostering such an environment is meeting in person and getting first-hand experience of our client’s day-to-day life.

We visited the Royal Society at the beginning of the project to immerse ourselves in their culture of scientific discovery. As part of the visit we explored the physical archives and were shown a selection of highlights. An important part of working together on a complex project such as this is a mutual appreciation for one another’s professional experience. Having seen the passion and expertise on offer at the Royal Society we were excited at the prospect of working together and we think the results can be seen in the final result.

Ben, Steve, and Tristan stand in front of ornate bookshelves containing bound copies of the Royal Society’s publications. — The Cogapp team visiting The Royal Society

To further understand the requirements of the project, we interviewed key stakeholders, including Fellows, senior figures at the Royal Society, and Historians of Science. These discussions inspired the development of unique features such as seeing item metadata in the viewer, navigation between pages in fullscreen mode, and maintaining zoom level between images to speed up the reading process.

To ensure the project’s success, the team at Cogapp employed an Agile development process that allowed us to quickly adapt to changing requirements and collaborate closely with the Royal Society. We worked in two week sprints, with the Cogapp team presenting their work at an end-of-sprint demo. Stakeholders from the Royal Society were encouraged to provide feedback during these calls or asynchronously using our shared project management tool, Pivotal Tracker. In addition to the sprint demos, regular communication was maintained via weekly Zoom calls. This environment of collaboration enabled us to create a user-friendly website that not only reflects the Royal Society's refreshed branding but is also tailored to the needs of its intended audiences.

The Science in the Making project exemplifies Cogapp's dedication to working closely with institutions in the GLAM sector to develop innovative digital solutions. Our collaboration with the Royal Society was a fantastic opportunity to use our expertise in integrating existing systems, automating data enhancement, and creating user-centric experiences.

The website demonstrates the importance of scientific preservation, collaboration and knowledge building, and the value of understanding the history of science and its modern day applications. We look forward to the platform growing and continuing the Royal Society’s positive impact on science research and education.

Science in the Making website

https://makingscience.royalsociety.org

Press

Royal Society: Four incredible objects that made science history – BBC News

Royal Society digital archive puts letters of Galileo and Newton online – The Times

Notes by Charles Darwin and Isaac Newton to made available online – ITV News

Find out more

If you want to hear more about our work, or you’ve got a project in mind that you think Cogapp could help with, please get in touch.