Drew Blount’s scientific contributions

Photo identification is an important tool in the conservation management of endangered species, and recent developments in artificial intelligence are revolutionizing existing workflows to identify individual animals. In 2015, the National Oceanic and Atmospheric Administration hosted a Kaggle data science competition to automate the identification of endangered North Atlantic right whales (Eubalaena glacialis). The winning algorithms developed by Deepsense.ai were able to identify individuals with 87% accuracy using a series of convolutional neural networks to identify the region of interest, create standardized photographs of uniform size and orientation, and then identify the correct individual. Since that time, we have brought in many more collaborators as we moved from prototype to production. Leveraging the existing infrastructure by Wild Me, the developers of Flukebook, we have created a web-based platform that allows biologists with no machine learning expertise to utilize semi-automated photo identification of right whales. New models were generated on an updated dataset using the winning Deepsense.ai algorithms. Given the morphological similarity between the North Atlantic right whale and closely related southern right whale (Eubalaena australis), we expanded the system to incorporate the largest long-term photo identification catalogs around the world including the United States, Canada, Australia, South Africa, Argentina, Brazil, and New Zealand. The system is now fully operational with multi-feature matching for both North Atlantic right whales and southern right whales from aerial photos of their heads (Deepsense), lateral photos of their heads (Pose Invariant Embeddings), flukes (CurvRank v2), and peduncle scarring (HotSpotter). We hope to encourage researchers to embrace both broad data collaborations and artificial intelligence to increase our understanding of wild populations and aid conservation efforts.

Photographic-identification (photo-ID) of bottlenose dolphins using individually distinctive features on the dorsal fin is a well-established and useful tool for tracking individuals; however, this method can be labor-intensive, especially when dealing with large catalogs and/or infrequently surveyed populations. Computer vision algorithms have been developed that can find a fin in an image, characterize the features of the fin, and compare the fin to a catalog of known individuals to generate a ranking of potential matches based on dorsal fin similarity. We examined if and how researchers use computer vision systems in their photo-ID process and developed an experiment to evaluate the performance of the most commonly used, recently developed, systems to date using a long-term photo-ID database of known individuals curated by the Chicago Zoological Society’s Sarasota Dolphin Research Program. Survey results obtained for the “Rise of the machines – Application of automated systems for matching dolphin dorsal fins: current status and future directions” workshop held at the 2019 World Marine Mammal Conference indicated that most researchers still rely on manual methods for comparing unknown dorsal fin images to reference catalogs of known individuals. Experimental evaluation of the finFindR R application, as well as the CurvRank, CurvRank v2, and finFindR implementations in Flukebook suggest that high match rates can be achieved with these systems, with the highest match rates found when only good to excellent quality images of fins with average to high distinctiveness are included in the matching process: for the finFindR R application and the CurvRank and CurvRank v2 algorithms within Flukebook more than 98.92% of correct matches were in the top 50-ranked positions, and more than 91.94% of correct matches were returned in the first ranked position. Our results offer the first comprehensive examination into the performance and accuracy of computer vision algorithms designed to assist with the photo-ID process of bottlenose dolphins and can be used to build trust by researchers hesitant to use these systems. Based on our findings and discussions from the “Rise of the Machines” workshop we provide recommendations for best practices for using computer vision systems for dorsal fin photo-ID.

Determining which species are at greatest risk, where they are most vulnerable, and what are the trajectories of their communities and populations is critical for conservation and management. Globally distributed, wide-ranging whales and dolphins present a particular challenge in data collection because no single research team can record data over biologically meaningful areas. Flukebook.org is an open-source web platform that addresses these gaps by providing researchers with the latest computational tools. It integrates photo-identification algorithms with data management, sharing, and privacy infrastructure for whale and dolphin research, enabling the global collaborative study of these global species. With seven automatic identification algorithms trained for 15 different species, resulting in 37 species-specific identification pipelines, Flukebook is an extensible foundation that continually incorporates emerging AI techniques and applies them to cetacean photo identification through continued collaboration between computer vision researchers, software engineers, and biologists. With over 2.0 million photos of over 52,000 identified individual animals submitted by over 250 researchers, the platform enables a comprehensive understanding of cetacean populations, fostering international and cross-institutional collaboration while respecting data ownership and privacy. We outline the technology stack and architecture of Flukebook, its performance on real-world cetacean imagery, and its development as an example of scalable, extensible, and reusable open-source conservation software. Flukebook is a step change in our ability to conduct large-scale research on cetaceans across biologically meaningful geographic ranges, to rapidly iterate population assessments and abundance trajectories, and engage the public in actions to protect them.

Photo-identification of individual snow leopards (Panthera uncia) is the primary technique for density estimation for the species. A high volume of images from multiple projects, combined with pre-existing historical catalogs, has made identifying snow leopard individuals within the images cost- and time-intensive. To speed the classification among a high volume of photographs, we trained and evaluated image classification methods for PIE v2 (a triplet loss network), and we compared PIE's accuracy to the HotSpotter algorithm (a SIFT based algorithm). Analyzed data were collected from a curated catalog of free-ranging snow leopards photographed across years (2012-2019) in Afghanistan and from samples in captivity provided by zoos from Finland, Sweden, Germany, and the United States. Results show that PIE outperforms HotSpotter. We also found weaknesses in the initial PIE model, like a minor amount of background matching, which was addressed, although likely not fully resolved, by applying background subtraction (BGS) and left-right mirroring (LR) methods. The PIE BGS LR model combined with Hotspotter showed a Rank-1: 85%, Rank-5: 95%, Rank-20: 99%. Overall, our results recommend implementing PIE v2 simultaneously with HotSpotter on Whiskerbook.org.

The four currently recognised species of Sousa are all threatened on the IUCN Red List. To date they have not been included in any of the available software platforms that have been developed for the automated matching of cetaceans from photo-ID data. Because of their unique morphology, existing algorithms are unlikely to be successful and new algorithms will be required. A collaboration between more than 35 researchers in the Western Indian Ocean began in 2020 and to date more than 1200 photos of 273 Sousa plumbea individuals from 7 countries (South Africa, Madagascar, Tanzania, Kenya, UAE, Iran and India) have been contributed to a training dataset; one of the largest collaborative efforts of its kind. Flukebook and finFindR are now starting the work of developing the matching algorithms and plans are underway to test the resulting algorithms and to develop a comprehensive plan for matching catalogues throughout the species range. It is hoped that resulting algorithms will also work on the three other species of Sousa, and that ultimately these will help to answer questions regarding movements patterns, home range etc that are important for these threatened species.

Data is deficient regarding humpback whales in south Pacific Mexico. The distinct population segments (DPS) of whales off of Mexico and Central America are listed as threatened (3,264 individuals) and endangered (411 individuals). We conducted 1,303 survey hours off Petatlán and Zihuatanejo, Guerrero between 2014-2017 and determined migratory patterns, site fidelity and habitat use by means of photo-identification, acoustic and genetic analysis in order to contribute to the characterization of the North Pacific humpback whale population and generate local conservation strategies. This presentation details our findings