Cow health status through face recognition

The project Happy Cow ID uses computer vision and machine deep learning aimed at replacing expensive, limited and inefficient ear tags and chips with a faster and more precise system to help dairy farmers.

The complete application is expected to be launched in 2022 and will be available for producers worldwide, according to Bruno Carvalho, a researcher at Embrapa Gado de Leite. Furthermore, he has invited all dairy farmers to send images of their cows via this email address bruno.carvalho@embrapa.br and using this protocol. Are you interested on participating?

In an interview with Dairy Global, Bruno Carvalho talks about how they want to make cows (and producers) happier.

How is the project going?

Our main objective is to classify animals on their health status based on the individuals’ facial analysis. For this, we apply image processing and computer vision techniques, in particular, deep learning and artificial neural networks.

Therefore, it is essential to structure a huge dataset of images, defining parameters or features per image that will be useful in the model learning process. One of these parameters is individual identification regarding breed, category, posture and health status.

Currently, we have few images of unhealthy animals. Therefore, we are advancing other characteristics. Our theory is that it may be easier to classify the animal’s health status from its breed or posture. In this process, we have observed that the system improves in accuracy when exposed to the same individual.

As identification by snout is a more complex capture process, identifying individual animals only by face requires automation by one or more fixed cameras on the property.

What is the name of the system, and what stage is it in?

The project is called Happy Cow ID. We are developing it through agile methodologies. It means we divide the project into several fronts – each one is in different levels. We do not have a complete product yet.

After the planning phase, which consisted of establishing the project’s scope, bibliographic body and technological implementation, we structured documents and protocols that help in the correct capture and submission of images for our dataset. As it is a slow process and depends on collaborators, we have obtained about a quarter of the stipulated number of images.

In the implementation phase, we developed a crucial step, which is face detection. Animal faces are selected from the images; pre-processing is what matters. Later, this model can be a tool for framing faces in the final product.

In order to identify animals’ face features precisely, we started from well-established models on facial expression recognition in humans. We are currently adapting these models to achieve greater accuracy, such as the individualisation of animals.

How does the technology work?

Deep learning and artificial neural networks are computational models that mimic human brain functions. In other words, the more samples that are exposed, the greater the learning about its characteristics and patterns. In our case, we direct our model to learn the features on the faces of animals. After this stage, the model will receive images of new individuals and classify their health status automatically.

What are the goals?

The objective is to produce a technology capable of predicting, with high confidence, the animal’s health status, in addition to identifying them individually, through its face.

How does this benefit producers?

This technology aims to allow producers to have greater control of their herd. In addition to identification, they will have access to accurate information about animals, so better farm administration and a better understanding of the behaviour and health of the herd. This information could be stored via computers and/or smartphones. Therefore, an identification and monitoring system allows for several functionalities for smarter and cheaper control.

What are the advantages over conventional models of individual identification, such as ear tags and chips?

Currently, ear tags used to identify animals on farms can be expensive and require the purchase of sticks to read the codes printed on the ear tags itself. This type of identification has some difficulties. For example, it requires each bovine to be tied up in order to place the identifiers on the animal and read the data. These practices cause stress and injuries.

Thus, identification by images has advantages. A system of cameras makes it possible to identify each cow quickly and accurately, avoiding any stress or injury to the animals. Furthermore, there are many possibilities, including storage, processing of feasible data and traceability.

What are the next steps?

As soon as we achieve the best results on deep learning for facial detection, animal classifications and individualisation, we will create an application that supports all these models on farms. This step will require a study to bring all the functionality together and achieve good usability.

Another very important point is that the project continues to receive images of animals to build the dataset, as the more images we have the better for deep learning.

What is the forecast for versions available to producers?

The project will see its first draft in November 2021. It is possible that the first versions for producers will be available with all functionalities in 2022.

Will the system software be free?

We are still defining monetisation strategies. The idea is to offer something affordable to be competitive with other available technologies.

Will foreign producers be able to use it?

Sure! We intend to offer the application to producers around the world!

Are you receiving animal images from other countries?

Definitely, every contribution is welcome! We designed a protocol to receive images via bruno.carvalho@embrapa.br. It is important to follow these instructions in order to submit the images properly.