‘Big Data’ in Animal Industries

September 13, 2021 | Letters

combined image of two dogsAuthor: Dr. Nicole Olynk Widmar, Associate Head and Professor, Purdue University Department of Agricultural Economics

Agriculture, food, health, transportation/logistics and animal industries are all awash in data whether they’re harnessing it in a usable way or not. Just as your smartphone generates a massive amount of data about you, your pet’s wearables are generating massive amounts of data about them. At the same time, photos of wildlife and records of disease in animal populations are being generated in massive scale worldwide, creating complex and rich datasets of varying forms.

While the scale of the data or the size of the datasets are relatively straightforward to visualize, the form and scope of data are more complex. Data may take the form of video or images of livestock animals. Those videos may be analyzed, and the resulting data can be combined with biological data for a comprehensive dataset that encompasses animal behavior data alongside biological data. This overview is simply a tiny snippet of what is already being done in data analytics in animal industries but is offered for motivation and demonstration of the breadth of the potential.

Companion Animal Industries

Activity monitors for pets are extremely popular these days. For example, FitBark Explore is a live, public, interactive digital map of dog health and wellness (FitBark 2016), which indicates at least one use of the data it collects — it’s available for public consumption. While FitBark Explore is one aspect of doggie data analytics, Camp Bow Wow (2021) has a more business-oriented use of data as it analyzes transactions to generate business insights such as how to best service specific customers through tailored offerings or adjusting promotions offered by franchises. Beyond wearables, there are also cameras for indoor and outdoor monitoring. Some allow two-way interaction, some dispense treats and some offer the use of artificial intelligence to differentiate on camera between a dog and a human (Furbo, 2021).

On the darker side of big data, Marr (2016) points out that while the Internet’s love of cats may appear cute, the meta-data embedded in millions of cat media photos has enabled a project called iknowwhereyourcatlives.com (2021). “Any photo of any subject — your home, your valuables, even your children — can be plotted on a map by anyone who chooses to look at the associated meta data,” (Marr, 2016). Although you can choose to remove location data from your photos, very few people go to the trouble. As you can see, many of the technologies we enjoy at home today, not just for our pets, but on social media and in other aspects of our daily lives, are Internet-enabled or contain data that goes unnoticed to us. This data can be used by others to learn about our behaviors, preferences, needs, wants and a variety of other aspects.

Wildlife Industry

In the wildlife industry, the Environmental Investigation Agency has pioneered a strategy of hidden filming and detective work to expose environmental crimes against wildlife and threatened habitats. Ultimately, the agency uses data analytics and intelligence to track illegal trade, including that of endangered tigers and big cats in Asia (Davies, 2013). For example, Microsoft’s AI for Earth helps solve global environmental challenges (Microsoft, 2021), and Google’s TensorFlow is used by Rainforest Connection to monitor for logging and other activity in the rainforest (White 2018).

Similarly, Wild Me (2021) “builds open software and artificial intelligence for the conservation research community,” combining machine learning and software professionals to assist in fighting extinction. Wild Me and the technologies employed can identify individual animal’s identities and employ photos submitted from around the world to contribute to conservation efforts (Winters, 2018). The use of text and images as forms of data widens the scope of big datasets immensely while simultaneously increasing the complexity of use and introducing a variety of new challenges and questions surrounding privacy and other societal concerns.

Livestock and Food Animal Industries

A variety of data collection, analysis and management efforts take place through governmental and public agencies dedicated to food systems and the preservation of plant and animal systems, such as the Animal and Plant Health Inspection Service (APHIS) (USDA, 2021a). APHIS has far-reaching efforts across a variety of animal industries in the U.S. and around the world, including the Cattle Health Program (USDA, 2021b). Specific efforts within the Cattle Health Program include the National Tuberculosis Eradication Program (USDA, 2021c), National Brucellosis Eradication Program (USDA 2021d), Bovine Spongiform Encephalopathy (BSE) Surveillance (USDA, 2021e), Emergency Response Programs that work with other Surveillance, Preparedness and Response Services Centers around the U.S. (USDA, 2021f) and various other programs. The variety of programs undertaken by just this agency and the data collection and analytics required to fulfill the stated missions of these programs is significant. Certainly, the scale of the datasets is one aspect, but the variety of data sources and various forms and uses of data are also critical.

Big data, sensor technologies, artificial intelligence and machine learning can all be employed to improve efficiency and animal health, lessen environmental impacts and potentially increase profitability, but not without challenges (Neethirajan, 2020). Livestock industry initiatives for animal improvement, such as Dairy Herd Improvement (DHI) and other species-specific organizations or investments, have been ongoing partnerships in research and industry advancement for many years. Livestock are increasingly managed in complex and technologically advanced systems involving automatic feeding systems, milk testing or analytics on individual animals, robotic milking systems and even physical activity tracking, and getting all of these systems to communicate with one another is a challenge. 

Livestock and Food Products Markets

Moving beyond livestock animals and into the marketplace for animal products opens up vast amounts of public and private data sources being analyzed for purposes such as targeted marketing, food safety and informed decision making by supply chain actors. The USDA Agricultural Marketing Service (AMS) Datamart offers access to historical Mandatory Price Reporting data (USDA AMS, 2021), and the USDA Foreign Agricultural Service (FAS) maintains data and analysis that are publicly available on U.S. and global trade, production, consumption and stocks — as well as market changes and factors — for a variety of livestock markets (USDA FAS, 2021). The availability of public data and analytics in livestock and agricultural markets through USDA has been a key feature of U.S. agriculture, ultimately shaping the development of those industries (Widmar, 2019). Beyond public data, individual grocery retailers collect and analyze data on individual products, consumer shopping trends and behaviors, and other key factors that influence what they stock, where and how. 

One Health and Societal Relevancy

The abundance and availability of data on global disease surveillance and other national or international human health relevant topics is increasingly relevant in our global economy and highly-connected world. Internet and social media data about veterinary medicine reveals a reasonably narrow scope of conversations about the profession, focusing on pet animal care, but with very little recognition of the veterinarians’ roles in One Health or food safety/security (Widmar, et al., 2020). Researchers are studying diets of humans and companion animals with interest towards the prevalence of obesity (Jung et al., 2017), and the movement towards the use of large and complex datasets to gain deeper understanding than previously available is prevalent in many animal industries. For example, multi-species advancements are possible through projects such as the Dog Aging Project (2021), which is aimed at improving the understanding of health longevity through the use of large datasets.

The potential for big data in veterinary surveillance was highlighted in its own session at the British Veterinary Association Congress at the London Vet Show in November 2016, covering the potential to “fill the surveillance gap” in terms of disease surveillance related to companion animals with which humans have a lot of contact (Clark, 2016). Guernier, Milinovich, and Bezerra Santos (2016) used Internet search metrics to monitor the occurrence of tick paralysis in companion animals to facilitate early detection, and canine vector-borne diseases have been mapped and forecasted by Self et al. (2019). They noted that, in the U.S., human vector-borne disease is monitored formally through the National Notifiable Disease Surveillance System and the Centers for Disease Control and Prevention; however, several of the same vector-borne diseases in domestic dogs are not nationally notifiable, and state-level tracking is varied. There is notable interest in livestock and companion animal disease tracking, but now the literature is increasingly interested in more comprehensive disease surveillance related to companion animals, especially in light of their proximity to people (Clark, 2016).



Camp Bow Wow. 2021. Available online: https://www.campbowwow.com/ (accessed on 29 March 2021)

Clark, K. 2016. Big data, big opportunities. Vet Rec. 179: 616-617. doi: 10.1136/vr.i6646

Dog Aging Project. Available online: https://dogagingproject.org/. (accessed on 29 March 2021)

FitBark. 2016. Fitbark Explore: A Live, Public, Interactive Digital Map of Dog Health & Wellness. Available online: https://public.tableau.com/profile/fitbark?utm_source=thenewstack&utm_medium=website&utm_campaign=platform#!/v zhome/shared/JMW6TQB8M (accessed on 29 March 2021)

Furbo. 2021. Available online: https://shopus.furbo.com/pages/designed-for-dogs

(accessed on 29 March 2021)

Guernier, V., G.J. Milinovich, M.A. Bezerra Santos, M. Haworth, G. Coleman, and R. Soares Magalhaes. 2016. Use of big data in the surveillance of veterinary diseases: early detection of tick paralysis in companion animals. Parasites Vectors. 9, 303. https://doi.org/10.1186/s13071-016-1590-6

I know where your cat lives. 2021. Available online: https://www.iknowwhereyourcatlives.com/ (accessed on 29 March 2021)

Jung EJ, Kim YS, Choi JW, Kang HW, Chang UJ. 2017. The Analysis of a Diet for the Human Being and the Companion Animal using Big Data in 2016. Clin Nutr Res. 6(4):256-266. doi:10.7762/cnr.2017.6.4.256

Marr, B. 2016. Big Data Knows Where Your Cat (and Kids) Live? TechTarget. Available online: https://www.datasciencecentral.com/profiles/blogs/big-data-knows-where-your-cat-and-kids-live (accessed on 29 March 2021)

Microsoft. 2021. AI for Earth. Available online: https://www.microsoft.com/en-us/ai/ai-for-earth. (accessed on 29 March 2021)

Neethirajan, S. 2020. The role of sensors, big data and machine learning in modern animal farming. S. and Bio-Sensing Res. 29, 2214-1804. https://doi.org/10.1016/j.sbsr.2020.100367.

Self SCW, Liu Y, Nordone SK, Yabsley MJ, Walden HS, Lund RB, Bowman DD, Carpenter C, McMahan CS, Gettings JR. 2019. Canine vector-borne disease: mapping and the accuracy of forecasting using big data from the veterinary community. Anim Health Res Rev. 20(1):47-60. doi: 10.1017/S1466252319000045.

Tensor Flow. 2021. Available online: https://www.tensorflow.org/ (accessed on 29 March 2021)

U.S. Department of Agriculture, Animal and Plant Health Inspection Service. 2021a. Mission. Available online: https://www.aphis.usda.gov/aphis/banner/aboutaphis/SA_Overview

(accessed on 29 March 2021)

U.S. Department of Agriculture, Animal and Plant Health Inspection Service. Cattle and Bison Information. 2021b. Available online: https://www.aphis.usda.gov/aphis/ourfocus/animalhealth/animal-disease-information/cattle-disease-information

(accessed on 29 March 2021)

U.S. Department of Agriculture, Animal and Plant Health Inspection Service. National Tuberculosis Eradication Program. 2021c. Available online: https://www.aphis.usda.gov/aphis/ourfocus/animalhealth/animal-disease-information/cattle-disease-information/national-tuberculosis-eradication-program. (accessed on 29 March 2021)

U.S. Department of Agriculture, Animal and Plant Health Inspection Service. National Brucellosis Eradication Program. 2021d. Available online:https://www.aphis.usda.gov/aphis/ourfocus/animalhealth/animal-disease-information/cattle-disease-information/national-brucellosis-eradication/brucellosis-eradication-program. (accessed on 29 March 2021)

U.S. Department of Agriculture, Animal and Plant Health Inspection Service. Bovine Spongiform Encephalopathy (BSE). 2021e. Available online: https://www.aphis.usda.gov/aphis/ourfocus/animalhealth/animal-disease-information/cattle-disease-information/cattle-bse/cattle-bse (accessed on 29 March 2021)

U.S. Department of Agriculture, Animal and Plant Health Inspection Service. Cattle Health – Emergency Response. 2021f. Available online: https://www.aphis.usda.gov/aphis/ourfocus/animalhealth/animal-disease-information/cattle-disease-information/cattle-health-center (accessed on 29 March 2021)

U.S. Department of Agriculture, Agricultural Marketing Service. 2021. Datamart. Available online: https://mpr.datamart.ams.usda.gov/ (accessed on 29 March 2021)

U.S. Department of Agriculture, Foreign Agricultural Service. Livestock and Poultry: World Markets and Trade. 2021. Available online: https://www.fas.usda.gov/data/livestock-and-poultry-world-markets-and-trade. (accessed on 29 March 2021)

White, T. 2018. The fight against illegal deforestation with TensorFlow. Google Blogs. Available online: https://www.blog.google/technology/ai/fight-against-illegal-deforestation-tensorflow/ (accessed on 29 March 2021)

Widmar, N.O. 2019. “Big Data” Provides Insights to Public Perceptions of USDA.” Choices. Quarter 4. Available online: http://www.choicesmagazine.org/choices-magazine/submitted-articles/big-data-provides-insights-to-public-perceptions-of-usda

WILDME. 2021. Available online: https://www.wildme.org/#/ (accessed on 29 March 2021)

Winters, J. 2018. How Conservationists Are Using AI And Big Data To Aid Wildlife. Northwest Public Broadcasting. Available online: https://www.nwpb.org/2018/08/15/how-conservationists-are-using-ai-and-big-data-to-aid-wildlife/. (accessed on 29 March 2021)

Widmar N, Bir C, Lai J, Wolf C. 2020. Public Perceptions of Veterinarians from Social and Online Media Listening. Vet Sci. 7(2):75. doi:10.3390/vetsci7020075