When we sit down to solve a jigsaw puzzle, there’s always one thing we take for granted: the picture on the box. Without that point of reference, we’d be pulling our hair out, trying and failing to rebuild a jumbled pile of miscellaneous pieces.
That’s exactly what was happening in the ancient Roman city of Pompeii, where over 10,000 fragmented pieces from 2,000-year-old frescoes have been lying around for decades, waiting for someone to solve the puzzle. Now, a team of scientists led by the Venice-based Italian Institute of Technology may have found a solution: train a robot to do it.
[Photo: courtesy IIT]
Dubbed “Reconstructing the Past: Artificial Intelligence and Robotics meet Cultural Heritage” (RePAIR), the project is funded by a €3.5-million grant (just under $4 million) from a European Commission that supports high-risk projects working toward “radically new future technologies.” The project will be developed in two phases: first, an algorithm will reconstruct the puzzle digitally, then a pair of robotic hands will put the puzzle (i.e. the frescoes) back together. This is the first time that AI will be used as an archeological tool at such a large scale, and the first time that robotic hands will be put in charge of so many pieces. If the project works, the scientists are hoping to deploy the technology in other cultural heritage sites around the world, like historic churches in Italy, or even the Ancient city of Palmyra in war-torn Syria.
A key part of the project will be teaching the algorithm how to study like an archeologist and think like a puzzle master. The puzzle-solving AI was developed in collaboration with a team at Ben Gurion University of the Negev, in Israel, and it works like an infinitely more intricate version of the popular memory game “find the pair.” The computer software compares all the fragments in pairs and rates their degree of similarity based on the shape of the pieces, how they fit together, plus how the illustrations on the fragments match up. Normally, this process can be done manually (using a computer), but the team is now teaching the algorithm how to compare pieces on its own.
[Photo: courtesy IIT]
Thanks to a team of archeologists from the University of Lausanne, who have attempted to solve the puzzle in the past, they already have about a dozen reconstructed clusters (about 10 pieces each) that they are feeding the algorithm. If the computer can put those pieces back together, they will know the system can be deployed at a larger scale.
[Image: courtesy IIT]
The robot will be deployed in Pompeii sometime next summer, but for now, the scientists are working on several projects in parallel. While one team is building the algorithm, another is 3D scanning a large sample of fragments so they can be put into the database (once the robot is fully operational, it will scan them on its own). And another team is working on the physical infrastructure and robotic hands that will eventually pick up the pieces and rebuild the frescoes. “One day, you will take all the pieces, put them into a room, lock the door, come back after a few days, and you will find the fresco completely reassembled,” says Marcello Pelillo, a professor of computer science and artificial intelligence at the University of Venice. (Though he admits things probably won’t go quite so smoothly.)
The fragments come from two separate rooms (including the ceilings) in a building …….