This presentation was presented as the keynote address for the 3D Digital Documentation Summit held July 10-12, 2012, Presidio, San Francisco, CA.
Rocket Science and 3D Analyses in the Preservation of Artistic and Historic Works
More than sixty years ago development began on the nuclear-propelled ORION Spaceship destined for a manned mission to the planet Saturn. This Defense Department program encompassed laser simulation of nuclear ablation as well as 3D holographic imaging of rocket exhaust plasmas and ultrasonic shockwaves within the space vehicle. After the cancellation of the Orion Space Program, portions of the remaining experimental and theoretical capability found their way to Venice, Italy, in support of the UNESCO effort to save the “Queen of the Adriatic” from the Aqua Alta threat. In 1972 this international project revealed the feasibility of 3D holographic recording of Venetian monuments, holographic interferometric and ultrasonic diagnoses of artwork interiors, laser divestment and consolidation of deteriorating artwork surfaces, and several art diagnostic procedures such as Digital Image Processing, inside-out MRI, and x-ray CT scanning. In subsequent decades these technologies have been embraced by the preservation and conservation communities resulting in the founding of a number of specialized professional societies. Dr. Asmus’ presentation will also describe projects involving such scientific tools for the conservation of sculpture in Italy, image restoration of the “Mona Lisa”, recovery of Qin-Dynasty terra cotta warrior polychrome, cleaning of the Buddha thumb, and restoration of The San Francisco Art Institute’s “Beat Culture” icon (“The Rose”) by Jay DeFeo. The photon divestment technique was selected to restore the Parthenon for the 2004 Olympic Games and is now being employed to restore Egyptian murals.
I’m sure that all of you at some time or other have heard the expression or maybe used the expression, “Well, it’s not rocket science.” I’m here to take the contrarian point of view to suggest to you that it is rocket science.
My rocket science began when I was in high school and I had a couple of science teachers and they were part of the naval reserve. They got me a summer job at a naval research laboratory. The first thing the director of the laboratory did when this scrawny high school kid showed up was to give me a desk and have me read a patent. I guess that was the only thing that he thought I could do. The patent was entitled, “A Secret Communications System” and it was dated 1942 and it was owned by the navy and it was classified as secret. The author of the secret communication system was Hedwig Eva Maria Kiesler. There was some paperwork that went along with the patent document and the paperwork said this is already been shown to be feasible as a mechanism of communication because it was used in Hollywood for a ballet performance, where this communication technique was used to synchronize sixteen player piano’s.
So I went back to that laboratory for a couple of summers as a high school student, and eventually the scientists and engineers there did build a communication system. They set up the transmitter for this communication system on the island of Kwajalein, which is near Bikini, and they set up a receiving site on the Hawaiian Island of Niihau. Then they launched a missile from Johnston Island, and when it reached altitude it detonated a rather large H-bomb.
Some of you here that have gray hair like me may remember that back in the cold war days, there was a technique that was talked about a lot, which had to do with nuclear blackout. The ionization from a high altitude explosion would simply create so much ionization that communications would be blacked out for most of the surface of the earth. This spread spectrum communication technique from the patent looked like it might be a way of keeping communications alive during a high altitude nuclear explosion. Now just as an aside, the scientific director of that project was a Brit. His name was Freeman Dyson. His father was Sir George Dyson, who was the director of the London Symphony Orchestra, but Freeman had gone the wrong way, he had become a theoretical physicist and later on, he won the Nobel Prize for his modeling of the nucleus of the atom. The unfortunate thing was that his model for the nucleus wasn’t quite up to date because the Scalar Higgs Boson that you’ve all been hearing about had not yet been invented, so that was the one thing missing from Freeman’s model for the nucleus of the atom.
But he was a very inventive guy, he’s’ still alive, and he decided that we all get behind a manned mission to Saturn. He invented a way of doing that which he named Orion, which was natural for a nuclear physicist; it was a nuclear propelled spaceship. The Orion spaceship received its thrust from a sequence of explosions of atomic bombs. Well, he did a lot of the calculations that addressed the issue of the feasibility of such a machine but he went back to Princeton to work on the atomic nucleus. The project was turned over to the theoretical physicist, Marshall Rosenbluth and Ted Taylor, who is sort of famous because he’s the one who invented the briefcase atomic bomb. This would fit in with the Orion spacecraft.
By this time, I had got out of high school, and I went to the place where they were building the Orion and they did build some test devices and on the walls of the offices you saw slogans that said, “Saturn in 1970,” and that was the schedule. A number of us were supporting this program by doing diagnostics to see whether this was actually feasible and that the theoretical work that Freeman Dyson had done was valid. So we had to analyze the plasma that provided the thrust for the spaceship. We had to analyze the shock waves, the ultrasonic shock waves that go into the space vehicle. We had to figure out ways of having this incredible impulse without simply destroying the spaceship.
So everything that I’ve done in the subsequent six decades has had to do with applying those technologies that we developed to analyze Orion to problems in the arts. One of the first things that we did was to do holography of these plasmas generated by the radiation from the atomic device explosion. So we were having to make bigger holograms of bigger subjects than had ever been done before. One my colleagues, one of the pioneers in holography, a man named Ralph Worker, decided we ought to try to make holograms that could be of commercial interest. So he got together with his friend, the Nobel Laureate, Dennis Gabor, who invented holography and received the Nobel Prize. They made a portrait of Dennis Gabor, one of these is the real Gabor and one is the holographic reconstruction of Gabor.
Ralph contacted Neiman Marcus in Dallas, Texas. There was a time when you could take your family around Christmas time to Neiman Marcus and they would make a hologram of your entire family. Then you could send out a holograph Christmas card with your whole family in three dimensions. So Ralph got a little bit of notoriety and our program got a little bit of notoriety. One day I was in the lab and the phone rang and I said, “2471, Asmus speaking,” and then I heard a man’s voice and he said, “My name is Paul Getty, perhaps you’ve heard of me. I deal in petroleum.” I said, “Yeah, I think I’ve heard of you.” He said, “Well, I have a problem. I have a ranch house in Malibu, and I’m going to turn it into a museum, and I’m going to take all of my Greek and Roman antiquities from my home in Sutton Place and I’m going to put them in the museum. I’m going to feel kind of lonely there with all of my statues disappearing, so how about making holograms of all of my statues and put the holograms in my home in Sutton Place as the statues disappear and go to Malibu?” So he started sending us two, three, four, five million dollar statues. Of course, the university people were very uneasy about this in terms of insurance and so forth, but that’s another story. As we were getting moving on making holograms of Greek and Roman antiquities, Mr. Getty died and the need for these holograms went away.
In the meantime, we got back to what we were really supposed to be doing which was working on Venice. I was actually in an oceanographic laboratory, Scripps Oceanography at the time, and a number of the people at Scripps Oceanography were mainly concerned with the demise of Venice, with the storm tides in the Adriatic sweeping the city away and as Lord Byron suggested, that the Queen of the Adriatic would sink beneath the waves of the Adriatic.
Now on a nice sunny day when the winds are coming down from the Dolomites, it doesn’t look like Venice has much of a problem. If you’re there in the winter time, dining can be difficult and so difficult for tourists in particular. But if you’re a marble statue, survival can be the real issue and the marble in Venice, as I’m sure most of you, if not all of you are well aware of, is dissolving under the air pollution, the moisture, the attack of vandals as well, and pigeons.
So, while we’re working on modeling the storm tides in the Adriatic, it occurred to us that maybe this is really where the holography would come into its own. Let’s make holograms of all of Venice before Venice is gone and then we’d have archival, high resolution, three dimensional records of the Venice that used to be. So we set up a laser and I guess I ought to point out what’s here. Here we go. After all, optics is a big part of 3D recording, so I guess we have to be prepared with lasers. So anyway, there’s the laser and there’s Ralph Wuerker, who was the first person to make really big holograms, and Georgio Guattari, University of Rome and that’s the laser and there’s the holographic plate and there’s the power supply behind Walter Munk, who is the Director of the Oceanographic work there. He’s the one who I worked for and he apparently was the first expert on waves and tides. He was the one who told Eisenhower when the Normandy invasion should take place based on the oceanographic conditions at that time. The reason the laser is there and the reason we’re making holography was that Walter dropped a few hints in Rome one time that had something to do with the Club of Rome and all of a sudden we got permission to do this.
We started by making holograms of Saint John the Baptist, who’s back there behind the holographic plate, which is a polychrome wood carving by Donatello. It’s very fragile and we figured that, Venetians told me that it was probably worth 150 million dollars, others said it was priceless and so we thought we’d start by making holograms of it. Here is the hologram or the photograph of the three dimensional holographic reconstruction of the image. One other thing I’d like to point out is those dark spots in the wall really are water spots because when the Sirocco wind would start blowing, rain would come in on top of us, on top of our laser with the thousands of volts and things like that were a little bit unnerving. But after a few weeks, we’d made fifty holograms of art works and these were put on display at the Academia Museum. The press showed up and in a day or so, newspaper headlines that appeared in the Venice Gazette, that the laser had saved Venice. They claim the title of the only city in the world that had been saved by laser. I was talking to one of the reporters or TV people and I said this whole thing is a miracle. I said making big holograms of large objects and obtaining a long coherent length from your laser of meters instead of millimeters or centimeters, which was more traditional, I said I never thought that we would number one, get our laser into Italy, and number two, thought we’d get through customs, and I never thought we’d get our laser to work and if we did get our laser to work, I figured our coherence length would be zilch, and we’d never be able to make holograms. I said within a couple of weeks, we had the laser working, we had several holograms made, and everything was working beautifully. The newspaper reporter said, “Well that makes sense because where your laser is, is holy ground. “ He said, “This building you’ve been working in was a church and you’re exactly where the sanctuary was,” and then he said, “No, it shouldn’t have made any difference, because the church was de-sanctified, so you could have done your holography anywhere and it wouldn’t have made a difference.” The he said, “Aha,” this is so Venetian, he said, “I know, the de-sanctification was improperly done. There’s still a holy relic there someplace, and that’s why it worked.” I said, “Well, I’ve never heard of de-sanctifying a church, what’s that all about?” “Oh,” he said, “During the war, the enemy occupied that church and they used it as a storage place for munitions. So when we drove out the enemy, we de-sanctified the church because we thought it was no longer appropriate to use that as a church.” I said, “Well, the people I meet around Venice said the Nazi’s never occupied Venice.” He said, “Oh, no, no, I’m not talking about the Nazi’s. I’m talking about Napoleon,” and he spoke as if he knew Napoleon personally. So Venetians have long memories.
To get serious again, there’s a close-up of the hologram and probably the most important slide I have to show today, which is the holographic interferogram of Saint Katherine or Santa Katarina. If you take a double exposure hologram and something changes between those to exposures, like the humidity changes by a few points or the temperature changes by a few tenths of a degree, you have two holographic images that are microscopically different and they interfere with one another, you get an interference pattern and if there are internal things, namely in a third dimension that are different, then you get a difference in the interference pattern and those little bubbles there that show up, are actually indicative of detachments that are deep between the preparation, the gesso preparation and the wood panel that Santa Katarina was painted on. So we found a way of doing non-destructive testing and a way of doing 3D holographic recording and things seemed to be coming along very nicely.
That was 50 years ago I guess, 40 or 50 years ago, 1972. You can do the math yourself since I’m approaching senility very rapidly, I’ll leave you to do the math. A few years ago someone visited and they wanted to see the actual holograms rather than the photographs of the holographic reconstructions. I got out a couple of those original holograms, and I was in despair. The images had gotten very noisy. Parts of the image were no longer visible because they were so faint and so blurred by noise. What I discovered was that, in the intervening forty or so years, the holograms themselves had deteriorated. Now probably we could have done something about this, but as I’m sure most of you know the hologram is simply a photographic emotion on a glass plate. As everybody in conservation knows, photographic emotions deteriorate. So the holograms were deteriorating. It’s with some chagrin and embarrassment that I have to report that the statues that we were going to save the images of are holding up better than the holograms that were supposed to record those images forever.
I went to an AIC meeting in Philadelphia some years ago and one of the best papers I’ve heard in recent years was delivered there by a fellow from Rand. I don’t know whether he’s here, I don’t remember his name, but the title of his paper was something like, “Will Your Magnetic Storage Media Last Five Years, Twenty-Five Years, Forever or Whichever Comes First.” So I went home and I threw away all my 8 inch floppies and then I looked at my 5 ¼ inch floppies and I threw all those away, and I’m still debating as to whether I should throw away my 3 inch floppies or not but that’s another story, and if you go to some conference where there’s a fellow from Rand talking about the viability and longevity of magnetic storage media, I’d highly recommend his talk.
I mentioned a little while ago that we were able to continue operating the laser in spite of rain and window leakage and things coming in on top of us. One night we were working around midnight, which we often did because it was dark then, and we didn’t have a shutter for our holographic, by then we were dealing with large plates and they were just too big for any camera back that we had, and the Sirocco winds blew in and everything was getting wet. This Saint John the Baptist is made of wood. It’s about 500 years old and there’s a lot of polychrome and gold gilding on it and we thought if Saint John got wet, the gold gilding might slide off and we might be responsible. If it’s $150 million dollar statue, maybe we’d be responsible for one and half million dollars of that for repair. So we called the statue restorer of Venice. Her name is Giulia Musumeci and she came over. We told her we had a catastrophe and she asked, “Have you put a rain bonnet on him?” We said, “No, we don’t want to touch him.” So she put a rain bonnet on the $150 million dollar statue and said, “Now everything is fine.” And then she said, “Well as long as I’m here let me tell you something.” She pointed out that they had various international teams come to Venice to do stone cleaning and stone restoration. They were all unsuccessful in trying to save the stone from the cancerous stone, the black stuff that had done considerable erosion of the stone.
Stones and marble are often about the strength and cohesion of a sugar cube or so and the black crust, calcium sulfate mixed with organics, is tough both chemically and physically. She said we don’t have an adequate way of doing the conservation and cleaning and divestment of sculptures and stone. She said, “Why don’t you try the laser.” So we tried the laser and found that we had reinvented the laser eraser if you will, which Art Schawlow, the Nobel Laureate for inventing the laser, also invented and used to manufacture at a company I think called Optics Technology in Palo Alto, except that nobody wanted to spend $500 extra for their IBM bouncing ball typewriters so that they could have a button there so when they made a mistake, the laser would fire and erase the mistake. We found that we could take the black off of statues.
Here’s Meg Abraham, who has done some contract work for the National Park Service and this relief of Isaiah, which is from the eleventh century, was covered with this black encrustation and here Meg has finished cleaning most of Isaiah. You can see the plasma down here where the laser is impacting on a little patch of black that’s left behind and that’s the same plasma of Orion except it’s on a miniature version. She will eventually get up here and clean off his head and things like that.
So it was winter time when we were doing this and there wasn’t much we could do in the evenings except watch television and in Italy these were the days of the spaghetti westerns. A few of you will remember that there was a western called Paladin. This gunfighter had a card and the card said, “Have Gun, Will Travel.” So I thought, gee we’ve invented this laser restoration tool, and I drafted a letter and the first line was “Have Laser, Will Travel.” The next day when I was a little, when the effect of frascati wine had worn off, I went ahead and I sent this letter to everyone I could think of, including people at the Taj Mahal and elsewhere in Egypt and so forth.
Interestingly, the first reply that I received within a week came from the National Park Service and it came from Del Rio, Texas. I guess there’s an office in Del Rio, Texas, or a park there. Probably has something to do with Big Bend and the letter began with the line, “Have Canyon Lands, Needs Cleaning.” The part of it he wanted us to clean was some encrustations of mineral deposits on a 2500 year old Anasazi rock art and here it is when we finished working on it. It took about a month.
After that we found ourselves itinerant laser cleaners going around the western United States, all the way from the Old Red Court House in Dallas to the State Capitol in Sacramento, Frank Lloyd Wright House in Hollywood, some buildings that belonged to Warner Brothers, and we finally found our way to Seattle at the Seattle Art Museum. Well we worked in Volunteer Park on the top of the Hill and they had ten spirit path Ming Dynasty statues. These had been out in the weather for a number of decades and they were covered with lichens and moss and other things, air pollution deposits and so forth. So we cleaned them up and then they took them indoors to the New Seattle Art Museum and put replicas back in Volunteer Park where these statues had been.
A few months later I received a letter and the letter was from China. The cover letter said, somebody wrote up an article in Popular Science, about your restoration of Ming Dynasty statues. He said it got reprinted in the Communist Workers Daily with some photographs of the work we did up in Volunteer Park. He said it speaks of the Ming Dynasty statue, do you think your technique would work on Qin Dynasty statues? Of course, not knowing the difference between a Ming Dynasty and a Qin Dynasty, I said, “Of course it’ll work on Qin Dynasty’s.”
Well anyway, that took us to Shian and instead of having ten statues to restore; I was faced with five thousand statues to restore. That’s a story too long to tell here, but suffice it to say, we didn’t start by shooting these two thousand year old statues. We started by going to the gift shop and buying a little trinket from the gift shop and practicing on that. There are many things that we had to do, and again, that’s too long to go into, but the core of it all was that we were trying to bring back the color.
All of these terracotta warriors had once been polychromed. The second emperor torched the tomb and so these were all charred and burned and everything and they lost their color. There’s more to that story. The University of Berlin is very active in this. But anyway our plan, and what we were successful at to a degree, was to change the valence of the metal ions in the pigments and the particular one that we had some success with was malachite. So we were able in a chamber with a forming gas, to transiently heat the burned malachite, change the burned copper and change it back to malachite and get some patches of color and then computer generate a full warrior color.
We went to a number of places around China and some of them were in the west. My favorite was a place called Fa Min Chi, where there had been a big earthquake in 1987 and a Buddhist temple had been partially destroyed. In the intervening years, there had been a lot of water seepage and so forth, and when the subterranean tomb was restored, they found a little pagoda that had been forgotten about and in this little miniature pagoda, was a scared relic. It was the thumb of the Buddha. In the intervening years, mildew had grown on the thumb and so we got to use the laser to take the mildew off the thumb. That was sort of fun and we were successful. But it was even more than success because one day a fellow came by and he said the top llama has agreed to have his photograph taken with you. Incidentally, what has remained of the pagoda after the ’87 earthquake and here it is restored and the tomb I was showing you was down below here someplace.
So we had this picture taking session and my host told me this was a great honor. At the end of the picture taking session, the head lama extended his hand and so I shook his hand. He went his way and I went my way and all of my hosts and translators and everybody were all excited. They said, “If you get to shake the hand of the high lama, you’re granted one hundred additional years of great good fortune. “ Now this happened in 1990 and so there’s not much of that left. But if you care, I’ll stand at the door and as you leave you can shake my hand, and you’ll each get ten seconds of additional great good fortune. I think this is true because the great good fortune happened in many venues and in many different ways. But in keeping with the theme of this conference, I’ll mention one of those things that happened.
We were invited to Florence and we were invited to the Hall of the Five Hundred, which for many years was the largest room in the world and has these gigantic mural pieces that I’m sure many of you have seen by Giorgio Vasari, battle pieces showing the Florentine armies being victorious over all their enemies, such as the Pisan’s. There had been some historical research that the Florentine’s described to me that suggested that Leonardo DaVinci had begun a mural, sort of in the era of the Last Supper in Milan and that he’d begun working on a battle piece called the Battle of Anghiari. There was some suggestion that in contradistinction to what was generally believed that perhaps Vasari just painted over the Battle of Anghiari and perhaps the Leonardo piece was still there, merely you have to determine where to dig in all of that area. They wanted me to do holograms or holographic interferogram as we’d done in Venice to see if we could get in that third dimension, and interior view as to what was under the Vasari murals.
We did some calculations and some tests at home and we decided that that was not going to work based largely on the dimensions and the materials and so forth. We considered neutron activation analysis for looking inside. We considered using infrared imaging to look inside, x-rays but the walls are meters thick and things like that. So we finally decided that one of those Orion techniques that we used, which was ultrasonic imaging, might work and so there we are shooting ultrasonic waves about a megahertz into the structure. The blue lines up there are our final most interesting c-scan of that area that’s right around there. We found a horizon that was about two meters high and about five meters wide, so that seemed to be about the dimension that Leonardo’s mural was supposed to be. We also found that as in the case of Santa Katarina and the other works of art, the ultrasonic technique was great for discovering detachments. So we could do our c-scans and find detachments and so forth, and a year later the restorers came to restore the Vasari’s but every time they came to one of our detachments, they pulled it off and looked underneath. Indeed we’d found a horizon and it was flat and it was a preparation for a mural but there was no sign of any pigment or painting or design or synothia by Leonardo. That was disappointing. We decided that that whole project, which took a couple years of our lives, was really a waste of time and came up with nothing. Later, in subsequent years, we decided that really we’d learned a lot. Just as we’d learned a lot from Orion, we learned a lot in Florence in dealing with non-destructive testing, ultrasound, and some of the other techniques we used.
So we went back home and we started to applying these in our real work for Cal-Space, which was at that time, associated with the lunar orbiters. So we started using some of these computer imaging techniques. Unfortunately, we had to use mainframes for this; the personal computer hadn’t been invented yet. We started making maps of the mineral resources on the moon and in this particular case it’s a map of where you find thorium emitted from the surface of the moon. We were doing that and I gave some lectures about this at UCLA and elsewhere and a friend of mine from UCLA showed up to look over my shoulder. His name is Carlo Pedretti, perhaps you’ve heard of him. He’s written a number of books on Leonardo. In addition, Carlo was the director of the Elmer Belt Museum of Renaissance Studies at UCLA, and Carlo came and said he had a problem.
Just as Mr. Getty had a problem a few years ago, he said he’d spent the previous summer at Windsor Castle working with Lord Kenneth Clark on cataloging the Queen’s collection of Leonardo drawings, and he said Lord Clark moaned and groaned the whole time he was there. He said, “Carlo, I really want to see the Mona Lisa before I die.” But he said the Mona Lisa was under this web of cleavages and the varnish layer and the paint layer and so forth, and there’s layer upon layer of brown and yellow varnish. Lord Clark said, “Every time I’ve contacted people at the Louvre, they have “arrogantly” refused to do anything about the Mona Lisa.” He asked, “Do you think you could contact your friend and see if you can do a computer digital enhancement, showing what the Mona Lisa would look like if it were restored.”
So Carlo came with that proposition and I said, “That sounds like fun, but I need a really good picture of the Mona Lisa. Can you get an archival color transparency, maybe 8” by 10” or something like that and we can digitize it and we can see what we can do.” Carlo called Lord Clark and Lord Clark called somebody at the Louvre and for three years about once a week, I’d get the news that the photograph was in the mail or that it would be in the mail on Friday, or that I should have it by now. But it never showed up for three whole years and so I’d largely given up and moved my thinking on in other directions.
Then Walter Cronkite retired and he started a program called Walter Cronkite’s Universe. They came out to do something about our work with submersibles and something about our laser divestment activities and so forth, and some producer told him about our plans to do the Mona Lisa. He said, “I’d like to do a video segment on your Mona Lisa work.” I said, “We’ve been waiting three years but they won’t send me an archival photograph. I suppose they think that if they send me a good photograph, I’ll start printing postcards and I’ll go into competition with the Louvre’s gift shop or something like that.” That was on a Sunday and by Friday my mailbox was stuffed with Mona Lisa photographs. Even the Egyptology department of the Louvre sent me a Mona Lisa photograph.
So, the idea would be that we would do the red, green, and blue scanning. The flatbed scanner hadn’t been invented but we thought we could send it up to NASA JPL labs where they had a scanning mica densitometer and we could digitize the red, green, and blue bands. Then if we could figure out what the varnish was, we could subtract that. Optical coherence technology had not been invented yet so we used photo acoustic spectroscopy and analyze some varnishes that were from the Mona Lisa and so forth. I guess I forgot to mention that in addition to getting photographs of the Mona Lisa, Cronkite rented the Mona Lisa. So we had the Mona Lisa for one entire Monday and so we were able to have our own photographs taken of it and so forth. We were able to get some varnish samples and analyze those. So we figured what we’d do would be to subtract the varnish absorption from the red, green, blue files of the Mona Lisa and we’d come up with a Mona Lisa that would look like the Mona Lisa that Leonardo saw a few hundred years ago.
So, for red, green, and blue from top to bottom, you see the original histograms and the distributions there, and then we de-convolve these with the spectral characteristics of the brown varnishes that we had and then through gain bias adjustments, came up with the histograms that are shown below each one of those. But you see the software couldn’t tell the difference between things that were on top of the varnish and things that were below the varnish like the Mona Lisa and so it treated the tinted craquelure on the surface of the varnish the same as it treated the painted image under the varnishes. So we got this thing where the face looked a little better and the sky looked a little better, but every place that you had craquelure it looked terrible. So I figured we had to do something about the craquelure. Well we did have a raking laser light scanning of the surface of the Mona Lisa, so we had a map of the tended craquelure, and so we thought we could subtract that from the image, which we did and that helped. Then we figured the varnishes and web of cleavages in the varnishes, there are many layers of restoration; the Mona Lisa apparently was restored several times and each restorer ground off part of the paint layer and added varnish layers and so forth, but the craquelure in the varnishes and in the paint and in the surface had different spatial characteristics from those lines and things that Leonardo put on the painting. So, we thought we could exploit those differences. So, there’s one band of the Mona Lisa, and there’s the Fourier transform transform in phase space and here’s the Fourier transform transform in amplitude space.
We made a mask. There’s the mask and through various things that we did, we decided that the Fourier space inside that mask was Mona Lisa and everything outside that mask was craquelure and so we applied that mask to that Fourier transform and then took the inverse transform and we got another Mona Lisa with less craquelure. That helped as you can see. Then we decided that the craquelure has different spectro characteristics from the Mona Lisa image itself. Here’s a blue and green map of the pixel values, the ratios and we decided that all these areas up in here is Mona Lisa and this area in the polygon is craquelure and so we removed those pixels and replaced them with the averages of the surrounding pixels and then we got a Mona Lisa that’s a little better.
Now, there’s a lot more to this Mona Lisa then what you can see because the pixel values in the hair in particular are very low, so they didn’t come through in the photograph. But we realized by this time, or at least we took seriously at this time, that the Mona Lisa had been restored many times and that the varnishes were different and they were of different thicknesses, but those different varnishes and those different thicknesses all had fingerprints. So we figured we’d key those fingerprints to the algorithm for cleaning up the Mona Lisa and things got a little bit better. So the one on the right, that’s as good as we could do. I think it becomes very dramatic when you look at individual parts of the Mona Lisa. There’s usually a problem with these digital projectors. This actually isn’t the way things looked but it gives you the idea. Here’s the face with the Mona Lisa unrestored with the craquelure and with the brown varnish and here it is cleaned up. This is all the responsibility due to Jim Arnold. Jim Arnold was at the University of Chicago and his thesis was the discovery of radio carbon dating and then he founded the chemistry department at UCSD and then he founded something called Cal Space and he paid for all the Mona Lisa work. He gave us all the time we wanted on the Cray supercomputer and so forth and so on for a couple of years there so he really was deeply involved and a very important element in this whole thing and he sadly passed away about six months ago of Alzheimer’s disease. He was a brilliant man and I guess it shows even if you’re thinking all the time, Alzheimer’s can still do its thing on you.
Now as soon as we cleaned up the Mona Lisa and people started looking at it on the monitor, we started seeing things; a lot of strange things in the mountains and the valleys and so forth. But the strangest thing of all was that everyone who looked at the monitor said, “Oh, she’s got a necklace, there’s some beads there.” So we did regional enhancement, and we could bring those out and it looked like there was actually two V-shaped necklaces there in pentimente, except the pentimente really didn’t show just looking at it with the naked eye or microscopic examination, but once you conceptually remove the varnish, the pentimente just leaped out at you.
So I sent this photograph to all the art historians who are noted for working on Leonardo paintings and artworks and documents and so forth and asked them what do you think this means. Well, art history friends had been burned by making false attributions and things and this is sort of the peak of that. Carlo had identified a painting called the Tavola Doria, and he said it was by Leonardo. Then the scientists got busy and proved that it couldn’t possibly have been by Leonardo. I don’t think that Carlo gave back the money that he got for the attribution, but anyway, the bottom line is I couldn’t get any art historian to publicly say anything about this necklace.
So for a month, I decided I would be an art historian. I wrote a paper on what I thought the necklace meant and what it had to do with the identity of the person who sat for the painting, when it was done, where it was done, something about the source of the mountains and so forth, and I published this. Well, if you’re somebody who is not associated with the Louvre, and yet you’re commenting on a Louvre work of art. I found out what it was like to be in the crosshairs of the French. Goya had it right, for about a year, I felt exactly like that poor guy with the white shirt there. After about another year, things quieted down a bit and Walter Sullivan, who was then Science Editor for the New York Times, arranged a conference call and Christian Lanier, who is the Scientific Director at the Louvre and I had this forty-five minute debate in this conference call. There are many things I could bring up about that, but his contention was that they had an x-ray of the Mona Lisa and the x-ray didn’t show any necklace. So my necklace was bogus. I said but there are lots of pigments that don’t show up on x-rays and furthermore, your x-ray was made in 1952 and its a terrible x-ray and it doesn’t show anything. So, in kind of a grumpy sort of way, Chris Lanier said, “Well, one of these days, we’ll take some new
x-rays and maybe we’ll do some infrared scanning and we’ll see if there is anything to what you say.
Barbara and I were here in San Francisco a couple of years later, and the San Francisco Chronicle was delivered to our door at the hotel and there on the front page right above the headline was a big box and it said, “Louvre Museum Discovers Necklace on the Mona Lisa.” At about this time, I gave a talk in Rome on this subject and a fellow came up to me, introduced himself as Chris Lanier, and he said, “Those were really interesting slides you showed,” sort of what I just showed you except a few more. He said, “Could I have copies of your slides?” I said, “Well of course, but you know you’re going to have to wait three years for those slides,” and I thought as much suffering as I’ve done, you’re going to have to do more than just wait three years, you’re going to have to provide and introduction to Katherine Denueve. Well, it’s now twenty or so years later, and Chris Lanier does not have copies of my slides and I still haven’t been introduced to Katherine Denueve and so that’s that standoff as things are at the moment.
But things aren’t all bad. I did get many other requests for copies of my pictures of the Mona Lisa and if you go around from museum to museum, mostly science museums but occasionally modern museums and others, you’ll find sets of our photographs showing the Mona Lisa as it appears today, as it might have appeared five hundred years ago, and if the pentimente is real, what the Mona Lisa might have looked like. So things turned out alright after all I think.
But like the Agnari project, it was really a stepping stone to other things. We found ourselves working with digital imaging in a lot of different areas. This is one of our first subsequent to the Mona Lisa work, which is a Rembrandt at the Getty, which you see here reproduced, the Man with the Gorget and it’s just a jumble of things. So we can take this image and adjust the biases and things and we can subtract it from the x-ray and then you can get a rather clear image of the under painting. So, we did things like that quite a bit.
Off the beaten path, we got involved in quite a different thing. Quite some time ago, NASA decided that there should be a history of the Space Program. So they hired a writer from NYU by the name of Alexander Marshack to research the history of the Space Program and its technologies and to write all this up in a major document for NASA. Alexander Marshack started looking into technologies and what led to these technologies and he kept going back and back further and further into history. Finally, he found himself studying ice age technology and he became so enthralled with ice age technology that he never wrote the NASA book. I think he’s the only American who can work in Lascaux and there are several other areas, he got involved in the whole gamut of the ice age art, one of them being the Alta Mira Caves in the Pyrenees. Some people think of this ice age art as sort of ice age impressionism. But Marshack was of a different point of view. He felt that many of these paintings, in particular these, and this has in the last couple of weeks has become quite famous, because there’s a new dating technique that I’m sure many of you are familiar with where they can better date these ice age paintings. They discovered that these Altamira rock art are thousands of years older than the Lascaux art. Marshack’s theory is that these artworks are really calendars; they’re time stamps of rituals that took place. To understand these time stamps, Marshack felt he really needed to know the order in which these lines were laid down.
At this time we’d been doing some work with a medical school on ophthalmology and macular degeneration and you take a photograph of a retina that’s 2 dimensional but there are blood vessels there and some blood vessels are above and below and you’d like to see them in that third dimension. So we developed a computer code with a rotating kernel and that shows which blood vessels are above others and we applied it to Alta Mira and when you apply the rotating kernel, you can very clearly see which lines are on top and which lines are underneath. That worked fine and Alexander felt that this really was a key to his interpretation and subsequently he became very famous as the one who unraveled the mysteries of the ice age cave paintings.
The National Park Service again, I forget who it was, somebody in the National Park Service put me in touch with Mina Wright, who is the White House Architect and we restored George Bush’s Vice-Presidential Office conceptually before they had to commit to the real thing. I won’t go into the details there. And that brought us to I think the project that I’m proudest of which took place right here at the San Francisco Art Institute within sight of Ghirardelli Square and if there’s somebody here from San Francisco, you may recognize one of San Francisco’s own, Jay DeFEO, who was part of the Haight- Ashbury crowd and she has something to do with the naming of the Grateful Dead and she used to read poetry with Alan Ginsburg and Jack Kerouac. She had an obsession with a painting which she originally called The Death Rose and there she is posing with The Death Rose and I won’t go into the symbolism there. Some of that is obvious. But anyway, she got evicted from her flat on Fillmore Street, and they had to take one wall off her flat so they could get the painting out. The painting is seven feet wide, ten feet tall, and one foot thick and weighs one ton. The painting had kind of an interesting history but it ended up bolted to a wall at the San Francisco Art Institute on Russian Hill and it started collapsing under its own weight. A ton of paint, some of which was cured and some wasn’t.
So the art institute entombed it in a ton of plaster of paris. Some years later, actually some decades later, it was decided that it should be restored, so there it is at the art institute ready for restoration and all of the techniques that I’ve described in the previous hour or so, were used to assist Tony Rockwell and Nicola Calderero in restoring the work. There’s just one example of where we did the ultrasonic c-scan trying to determine where the paint was within the plaster of paris tomb, if you will. Well, it got restored. The Whitney Museum paid for the restoration. They then bought the painting and took it to the Whitney Museum in New York. Then they had a show called Beat Culture and it was the centerpiece of that exhibit. Of course, they had a champagne preview of it and in that champagne preview, Thomas Hoving showed up and he looked at the Rose and he said, “That’s the greatest representation of Genesis that I’ve ever seen.” He was writing a book and on pages 10 and 11 of that book, there’s Jay DeFEO’s Rose as the best Genesis painting ever.
I would like to say a lot more about Jay DeFEO but suffice it to say, I disagree with Hoving. The Rose, which she originally called the Death Rose is just shades of gray. She has another painting called The Jewel, and I think The Jewel is probably much more likely inspired by Genesis.
There was one time in my life where on one side of my desk were pictures from the Hubble Telescope. On the other side of my desk were pictures by Jay DeFEO. I was looking at some these one day and I thought, “My gosh we as a nation didn’t have to spend billions of dollars with the Hubble Telescope to get pictures of outer space. We could have just looked at Jay DeFEO’s paintings and we had the same thing. “
There was a surgeon at UC San Francisco. His name was Leonard Shlain. I think he passed away and his contention was that the artists are ahead of the scientists and I think that’s probably the case in her situation.
Well, I need to wrap up. Maybe I needed to wrap up five minutes ago but anyway one last thing I wanted to say about The Jewel and that is as I mentioned earlier, one of the things that is very much in the news now is the God particle, the Scalar Higgs Boson, and I’m sure many of you, on television or magazine, have seen representations of what the Big Bang or the Boson looked like and they also all look like The Jewel in Jay DeFEO’s paintings.
Anyway, I think the things I’ve been talking about here have had an effect on our economy. We’re all interested in the economy. Well, now there are a dozen or so companies that manufacturer laser statue cleaners and there are buildings everywhere that have been restored by laser, including the Parthenon which was restored in time for the 2004 Olympics.
There are a couple of other things I have to say, a few last things. One is, a few years ago, the patent that I read when I was in high school was declassified after fifty years. It turns out that Hedwig Ava Maria Keisler was later known as Hedy Lamar. That was the name that she got from MGM as the most beautiful girl in the world in terms of Louis B. Mayer.
In San Diego we have a company called QUALCOMM and they make the chips that go in all of your cell phones. Those came from a program at MIT. Claude Shannon invented communication theory and his principle student, Irwin Jacobs started the QUALCOMM Company. Shannon is dead now but when Jacobs gives a speech, every once in awhile he said, cell phones are way ahead of where they would have been if it hadn’t been for Hedy Lamar. Now Hedy Lamar in 1992 when her patent was fifty years old and had been declassified, she was awarded the title of “Inventor of the Year.” When she accepted that award, she gave a once sentence acceptance speech. The acceptance speech was, “It’s about time.”
There was a period when I gave a lot of speeches really about the life and times of Hedy Lamar and somebody in the audience stood up at the end and said, “If you’re so enamored with Hedy Lamar, why didn’t you marry her?” I said, “Well, she’s had six marriages and six divorces, one more would be redundant.” But I found my own Hedy Lamar. There she is. I don’t see any difference. Barbara and I went to Atlanta, Georgia, to Georgia Tech, where we were zapping garbage and turning garbage into glass pebbles to make highways out of and a bunch of people who were there were watching us. I didn’t think they were Georgia Tech engineers because they had a lot of body piercings and tattoos and so forth. We had lunch with them and they asked some pretty good laser questions and they said they had a rock band. So that was nice, I had lunch with some rockers and then went home and Barbara and I turned on David Letterman and there they were on David Lettermen.
So, I don’t know how many of you have teenage daughters. Maybe some of you are teenage daughters. I think maybe so. Well I called our younger daughter who was a teenager and I said, “Rosemary, you’ll be proud of your old man, he’s really cool, he had lunch with a rock band,” and she responded, “What high school do they play for?” I said well I think they’re a little big time. I said I heard them talking about platinum and gold and things like that and she said, “Well, what’s their name?” I said, “Well, I can’t pronounce it, there are no vowels, just consonants but I have a napkin with it written down and it’s Lynyrd Skynyrd. Rosemary started screaming and she said something about injustice and that I’d never even heard of Free Bird or Sweet Home Alabama or anything like that and finally she slowed down and I heard this phrase and the two words that juxtaposed were awesome and dad. That made the sixty years of blood, sweat, and tears worthwhile and I hope my comments to you, you’ve found worthwhile and enjoyable.
John F. Asmus is on the Research Faculty of the Department of Physics at the University of California, San Diego. He earned his PhD. From the California Institute of Technology and is the co-founder of the Center for Art/Science Studies at UCSD. In 1990 he was awarded the Rolex Laureate for Enterprise (Polychrome Recovery Of The Qin-Dynasty Terra Cotta Warriors) and became a Fellow of the Explorers Club. He has published 150 articles in professional journals in the fields of lasers, laser applications, photoacoustic spectroscopy, digital image processing, ultrasonic imaging, holography, holographic interferometry, plasma pinch technology, and hypervelocity impact phenomenology. He is the author of 25 patents.
In the decade of the 1960s he was employed by General Atomic Corporation and contributed to the ORION Space Ship, CASABA-HOWITZER, and High-Altitude EMP Programs. In 1967 he became Laser Department Director. During the 1970s he was a Research Staff Member of the Institute for Defense Analyses where he organized and directed studies for DARPA, DDR&E, and JASON on high-energy lasers, Soviet laser technology, chemical and gas-dynamic lasers, high-energy-laser propagation, and the National Laser Laboratory Proposal of the US Presidential Science Advisor.
Over the past forty years, Dr. Asmus has led the art conservation field in the use of 3D holography, lasers, ultrasonics, and digital image processing. He has applied these tools to the problems of divestment, analysis, interpretation, and presentation associated with diverse art conservation activities as well as lunar imaging programs at CALSPACE. He has also been instrumental in the founding of professional art conservation and laser societies. He is a member of UNESCO and EU conservation and preservation commissions on World Heritage Sites including the Parthenon restoration program. He is the Honorary President of the LACONA professional society and is on editorial boards for Springer, Wiley, and Elsevier.