Creating 3D Models of Sea Turtles

Horizontal with Text (1).jpg

About the Digital Life Project

The Digital Life Project was started in November 2016 as a non-profit initiative within the University of Massachusetts at Amherst for one simple goal: to create accurate, high-resolution 3D models of life on earth. Prior to its inception, the art and science of creating 3D animals was largely the domain of 3D animators. While this practice will remain a vital practice, the possibility of scanning real animals remained largely untapped.

A digital “ark” of living organisms would provide an invaluable resource for educators, scientists and conservationists interested in recreating and studying life in all of its manifestations. Imagine a website or VR experience that could provide millions of living animals at the fingertips of students. Or consider how scientists could use digital animals as a 3D museum of sorts, an exciting possibility in an era when traditional museums are financially challenged and are often unable to meet their basic mission of documenting life on earth.

This idea’s fruition can be traced to recent developments in 3D scanning the last few years that have made it viable to capture living animals through the process of photogrammetry, which until recently was impossible with living animals. One of the main pillars of this is the Beastcam™ technology developed at the University of Massachusetts at Amherst which represents a series of custom multiple-camera devices designed for capturing a variety of living animals, from small insects to large sharks and sea turtles.

3D sea turtle campaign

 Photograph by Christine Shepard

Photograph by Christine Shepard

One of the focal points of our efforts has been the “Sea Turtles of the World Campaign”, which aims to create 3D models of each of the 7 species of sea turtles.  Sea turtles play a vital role in the world’s oceans and coral reefs through maintaining the productivity of coral reef ecosystems, as well as cycling nutrients around the ocean and shoreline.

We chose sea turtles as a key group to focus on for several reasons.  First, all sea turtle species are highly threatened, and populations in key parts of their range have gone, or could go extinct within our lifetimes.  In other words, the long-term future of sea turtles remains in doubt, and this fact gave our mission some urgency. A second reason is that high-quality specimens of different sea turtles are rarely available in natural history museums, especially for adult specimens that are challenging to preserve.  Thus, our hope was that accurate 3D models of sea turtles would serve as a valuable tool for scientists and educators to study, learn, and teach about sea turtle morphology. For example, scientists can use our 3D models to study swimming of sea turtles in a virtual environment, thus potentially unlocking mysteries of sea turtle migration patterns.  In this vein, all of our models are free to download for non-profit purposes.

Here we will tell you the story of three sea turtle individuals for which we have created 3D models: Scallywag (Green sea turtle), Shelly (Loggerhead sea turtle) and Blue (hatchling leatherback).  Each of these three individuals has an amazing story to tell that reveals a great deal about the lives of sea turtles, some of which can be seen in the 3D models themselves.

The story of Scallywag and Shelly

Scallywag (ID=CM1648) is a green sea turtle (Chelonia mydas) who survived a shark attack off the coast of Southern Florida. Scallywag was most likely attacked by a Tiger Shark (Galeocerdo cuvier), which occur in coastal waters of Florida and are known to attack and consume sea turtles.  The attack left Scallywag with a significant scar on the shell and a missing right front flipper.  Scallywag was found floating and clinging to life in the surf at Blowing Rocks Preserve on December 23, 2016.  Scallywag was captured, and then transported to the Loggerhead Marinelife Center (LMC). Despite suffering from these scars, Scallywag has been undergoing successful rehabilitation and is now able to effectively swim and move about. Scallywag remains at the LMC to continue the process of rehabilitation.  Shelly (ID=982.000406129037) is a near-adult loggerhead sea turtle (Caretta caretta) rescued at the St. Lucie Power Plant by the Inwater Research Group on June 22, 2017.  Unlike Scallywag, this turtle was in good health, and after some basic measurements and photos, the animal was released unharmed.  Loggerhead sea turtles commonly forage on the bottom of coral reefs for crustaceans and other animals and often obtain barnacle passengers, which this female had in abundance. 

"Scallywag" A green sea turtle photographed at the Loggerhead MarineLife Center - Reconstruction and animation created by Jer Bot.  Click and drag the 3D model to rotate the view

Photogrammetry process

Photographer Christine Shepard used the Beastcam technology to photograph the sea turtles.  She used several tripod-like devices with a total of 20 cameras that allowed her to simultaneously capture a set of images of the same turtle from different perspectives.  As the turtles were being prepared for release or for other purposes, we were able to quickly capture such sets of images of the animals. This process was simple and quick and allowed the teams we worked with to do their work with the sea turtles without a hitch.

 

 The Beastcam technology arranged around a green sea turtle.  Photograph by Christine Shepard.

The Beastcam technology arranged around a green sea turtle.  Photograph by Christine Shepard.

The next step in our process was pooling photos of each sea turtle into the photogrammetry software Capturing Reality.  This software takes individual photographs and integrates them into a 3D mesh, it is then overlaid with a UV map, which results in a highly realistic 3D model, but also one that needed a great deal of work.  Indeed, as the image below of the initial scan of Scallywag attests, the “Raw” scans had significant areas of occlusion (lack of resolution, where the software could not accurately reconstruct the scene). This would be a challenge for the 3D artists.

3D rendering

The 3D artists (Jer Bot and Johnson Martin) all had a daunting challenge in front of them. Each had to take the raw photogrammetry scans, which were large (>5 million faces), and had areas with occlusion, and convert them into low-poly (<20,000 faces) and “complete” 3D models.  Working with Scallywag was a refreshing experience for Jer Bot. Unlike most animal models that are perfect prototypical animals, the shark’s attack had left Scallywag scarred and unsymmetrical. Jer used his background in character building to inform how to accurately reconstruct Scallywag, and he quickly found out that every step of the process represented new challenges that he had not experienced before.

 Meshes of the Loggerhead sea turtle reconstruction

Meshes of the Loggerhead sea turtle reconstruction

Animator Johnson Martin also found the experience both rewarding and challenging.  Although Shelly did not have the same kinds of dramatic scars as Scallywag, the challenge of capturing all the variation such as barnacles and various other individual features, also required significant consideration.  One of the most substantial challenges for each turtle was the joining of the dorsal and ventral halves. In this vein, Blender, which was the primary software tool for all three animators, proved highly useful. Blender's default skinning values gave the animators a great starting point, and enabled them to quickly adjust the posing of both halves until they aligned.  Further, Blender’s built-in sculpting and Krita’s texture painting tools helped with the merging and cleanup of the high frequency geometry and 4k textures.  For Jer Bot, the most rewarding part was reconstructing the swim cycle of Scallywag, which shows how the locomotion has altered with the loss of a flipper.

"Shelly" A loggerhead sea turtle photographed with the Inwater Research Group - Reconstruction and animation created by Johnson Martin.  Click and drag the 3D model to rotate the view

Why these 3D models are valuable

The obvious question to ask is: Why did we invest so much time to create these 3D models of these three species of sea turtles?  One possible answer is that we felt that these 3D models represent a valuable tool for scientists. For example, scientists can use these models to perform computational fluid dynamics to understand the process of swimming and migration in sea turtles.  Or they can use these models to more exactly test how nets entangle sea turtles, and how to design them to enable them to escape. We view these models as playing a vital role in educating the public about the general shape and taxonomy of sea turtles. With these models, any educator with a computer or VR headset can easily show students how sea turtles swim, and how they differ from one another.  Further, the 3D models can be immediately downloaded and used in VR applications, and our team is currently working on immersive and biologically informative immersive VR “animal encounters”.

Finally, the value of these 3D models in our minds is more subtle – as a group of biologists, photographers and artists who care deeply about the biodiversity of life on earth, we also feel a need to help document this incredible richness before it disappears.  The honest truth is that just as others have scanned rare and endangered buildings because they are concerned that the building will degrade, we are also deeply concerned about the future of sea turtles. With the threats of climate change, human population growth and wildlife trafficking, their future remains in doubt.  In this regard, we hope that our models, so lovingly created by our talented 3D artists play perhaps the most important role of all – we hope that they stop and make you think “What would a world without sea turtles look like?” We hope that in whatever small way possible that by viewing these models you agree with us that such a future is not one that you want to be part of.

How you can access the models

These 3D models are available free of charge for creative and educational non-profit work.  For those groups interested in the low-resolution models, these can be downloaded at our Sketchfab site (https://sketchfab.com/irschick).  To access the high-resolution models, or to inquire about licensing opportunities, please contact Duncan Irschick at irschick@bio.umass.edu.

How to support sea turtles and our work

Many partners participated in this work, and are working tirelessly to conserve and study sea turtles.  If you are excited by our work, and want to either help sea turtle conservation directly, perhaps consider directly donating to the Loggerhead Marinelife Center (https://www.marinelife.org/), or to Jeanette Wyneken, who performs outstanding research and conservation work with sea turtles (http://biology.fau.edu/directory/wyneken/index.php).  The Digital Life team continues to scan more sea turtles until we are able to obtain 3D models for all seven species, so you can directly support this work through donations to the 3D sea turtle crowdfunder (https://minutefund.umass.edu/project/4730).

Special thanks to

We would like to say ‘thank you’ to several partners without whom this work would not have been possible.  A number of donors to the UMASS Minutefund and also private donors were instrumental in enabling this work.  We thank the Loggerhead Marinelife Center for allowing us to photograph some of their sea turtles. We also thank the Inwater Research Group, which allowed us to photograph their sea turtles during processing.  Annabelle Brooks assisted with the work and provided valuable guidance throughout. Photographer Christine Shepard was remarkable in taking many amazing photos that allowed the Digital Life team to create photogrammetry models.  Mackay Eyster and Brooke Ackerman wrote much of this blog and also played a vital role in organizing the photos and creating the 3D models before sending to the 3D artists. Finally, animators Jer Bot, Johnson Martin, and David Maas were invaluable in devoting numerous hours creating the final models from the raw photogrammetry scans.

Lean about the Artists

Jer Bot https://vimeo.com/verbal007

Johnson Martin https://johnson-martin.artstation.com/projects/rmvJ2

About Loggerhead Marinelife Center

Loggerhead Marinelife Center is a nonprofit sea turtle hospital that promotes conservation of ocean ecosystems with a focus on threatened and endangered sea turtles. The Center features an on-site campus hospital, research laboratory, educational exhibits and aquariums, and also operates the Juno Beach Pier, which hosts world-class angling and sightseeing. Situated on one of the world’s most important sea turtle nesting beaches, Loggerhead Marinelife Center is open daily and hosts over 350,000 guests free-of-charge each year. In late 2017, the Center launched their Waves of Progress capital expansion campaign, designed to accelerate and amplify LMC’s conservation and education impact. When complete, the facility will offer one of the world’s most advanced and unique experiences for guests and scientific partners. For more information, visit marinelife.org or call (561) 627-8280.

About Inwater Research Group

The Inwater Research Group, a non-profit corporation with a mission to provide the scientific community and general public with information to promote conservation of coastal and marine species and their habitats.

Creating 3D Models of Skinks

Capture.JPG

My name is Leah DeLorenzo, and I am an undergraduate at the University of Massachusetts at Amherst.  I am working on an exciting project in the Irschick lab that is part of a broader NSF grant in collaboration with Cameron Siler, Philip Bergmann and Gunter Wagner whose goal is to research the morphology of Brachymeles lizards. Brachymeles is a genus of Skinks within the lizard family Scincidae. Skinks are among the most diverse groups of vertebrates on earth, with over 1,500 known species. They can range in size from about 3 inches to 14 inches long. Brachymeles skinks are distinctive in showing tremendous variation in limb and toe shape and size, with some species being fully limbed, to others being entirely limbless. They also have varying degrees of axial elongation due to the number of vertebrae that they have lost or gained over evolutionary time. Since they are such a disparate group, they are a great organism to use to study evolutionary trends. There are eight species of Brachymeles skinks that I was tasked to 3D model.  All originated from the Philippines. We wanted to 3D model these organisms using photogrammetry in order to quantify the evolutionary differences between these species. We chose photogrammetry because it is cost-effective and yields awesome results! All you need is a camera, a stool and good lighting.

The Process:

The first thing I did was set up a turning stool with LED lights placed around it. I placed each skink in the middle of the stool and snapped a picture. After I took each picture I rotated the stool in order to get shots of all sides of the specimen. I took about 50 pictures of the dorsal side and 50 of the ventral side each skink. 

After I took these pictures I loaded them into Photoshop Lightroom. Using lightroom  I adjusted the temperature and sharpness settings of each photo to ideal settings for photogrammetry. After  post processing, I loaded the pictures for the dorsal side into Reality Capture. The Reality Capture software uses an algorithm to combine 2D images into a 3D model. After the dorsal side of the specimen was complete, I repeated this process for the ventral side.

I combined both the dorsal and ventral meshes of the specimen using Meshmixer. This proved to be a difficult process, but the end product was worth the countless hours of trial and error. I used an additional program called GeoMagic to smooth out some of the rough edges that were byproducts of the combining process.

Here are two examples of the combined models!

 

 

Using these 3D models we will perform some quantitative analysis to understand more about the evolution of the body axis in these amazing animals.  Through this process, we hope to answer basic questions on how limbless forms have evolved. In January, I will present my research at SICB in San Francisco! In summation, 3D models are awesome and are great tools for researchers!

-Leah DeLorenzo

University of Massachusetts Amherst Class of 2018

How does one 3D Scan a Live Rattlesnake?

Very carefully.  The Digital Life team set out to do something nobody had done before - to create an accurate, high-resolution 3D model of a live Timber Rattlensnake (Crotalus horridus).  For those of you who don't know much about Timber Rattlesnakes, they are among the most persecuted and threatened of all rattlesnakes.  Because they often co-occur in regions of high human habitation on the East Coast of the USA, and because they have a habit of congregating in dens in rocky hillsides, this makes them very vulnerable to those with ill intentions towards them.  Much like sharks (a blog post will come on that later), rattlesnakes are often misunderstood creatures which generate fear out of proportion to the actual threat they pose to humans.

For these reasons, and also because Timber rattlesnakes are very beautiful and striking animals, our team was fortunate to visit NERD in March 2017 - New England Reptile Distributors located in New Hampshire.  The owner Kevin McCurley, and the venomous snake specialist, Tim Velton, both were incredibly gracious with their time and access to their specimens.  As a group working with the University of Massachusetts, we were clear about our limits - we could not touch or handle the specimens, and we wanted to ensure that the entire experience was safe for both the snake and the handlers.  Fortunately, Tim is a professional who exercised care and thoughtfulness when dealing with the snake.

Upon arrival, our first job was to set up the Beastcam technology - we used a system of fixed arms arranged in a circle around the animal that allowed us to take images from every perspective.  The Beastcam technology works on the principle of photogrammetry - the process by which individual photographs are integrated using software such as REMAKE by Autodesk or Capturing Reality,  into 3D geometries that have 3D textures and colors placed on top.  The results from photogrammetry can be startling - its now possible to recreate everyday objects in 3D to an incredibly high resolution.  However, carrying out this practice with a live animal is far more tricky and requires a high degree of coordination and practice.  Fortunately, we had a crack team for this visit - UMASS Amherst undergraduates Michael Perriera, Zachary Corriveau, and and former undergraduate Kalpita Patel.  Jeremy Turgeon also participated as one of our key reptile and amphibian 3D specialists.

We used a white sheet scattered with random black dots as the substrate on which the snake would be placed.  The white sheet would reflect light upon the snake, and the black dots provide landmarks for the 3D software to grab onto.  We used 32 CANON G16 cameras synced with wireless triggers arranged in a circle.  Finally, several 5500K LED bulbs provided illumination, and it was showtime!  Tim brought out the snake, which was a gorgeous older individual (one can often tell the age of a rattlesnake by the number of rattles).  The milky eyes also showed that the animal was shedding.  Very large and old timber rattlesnakes are rare in the wild for the reasons noted above, so it was amazing to experience one in person.  Fortunately, we could snap the shots from a safe distance!  The snakes behavior was calm and showcased what I had learned from working with sharks and other "dangerous" animals- their reputation often exceeds the reality of their behavior.  The snake was curious but never appeared threatened, and the shoot went off without a hitch, and we were able to capture a 3D model of the snake in an awesome pose

Back at the University of Massachusetts, my undergraduate Cory Zeng worked hard to reconstruct the model and clean it up.  We used Capturing Reality to create the initial model, and then REMAKE and MUDBOX by Autodesk to clean up the model.  You can check out the model above!

You are likely wondering, how can you use this 3D model?  First, as noted on our website, you can download the 3D model FREE for creative/non-profit work.  So if you are an artist and have an exciting idea on 3D snakes, we encourage you to work with our model - we would love to hear from you!  If you are a teacher or educator, you can use it in your classroom to show off the shape and morphology of snakes, and also provide an opportunity to talk about Timber rattlesnakes, and the threats they face.   If you are a VR enthusiast, why not try using the Sketchfab VR button and step into a room with a 20-foot snake?  All are possibilities.  Most of all, we hope that you use this content to gain a newfound appreciation and love for these magnificent and misunderstood creatures.

Peace out!

Duncan J Irschick, Director, Digital Life Project

August 2017