WO2019116237A1

WO2019116237A1 - Devices and methods for automatically classifying pet motion as chewing or non-chewing

Info

Publication number: WO2019116237A1
Application number: PCT/IB2018/059894
Authority: WO
Inventors: Benjamin Jean Didier Le Reverend; Sandra Lyn; Annie VALUSKA; David Alonso; Hossein MAMAGHANIAN
Original assignee: Nestec SA
Current assignee: Nestec SA
Priority date: 2017-12-13
Filing date: 2018-12-11
Publication date: 2019-06-20
Anticipated expiration: 2020-06-13

Abstract

A device to be worn by a pet can include base hardware containing one or more first inertial sensors and further include an extended node electrically connected to the base hardware and containing one or more second inertial sensors. The one or more first inertial sensors can obtain a first inertial measurement, the one or more second inertial sensors can obtain a second inertial measurement, and the first and second inertial measurements can be associated with a motion by the pet. The device can have a processor that classifies the motion as a chewing event or a non-chewing event based on the first inertial measurement and the second inertial measurement. A method can comprise positioning the device on a pet, for example by positioning the base hardware on the neck and/or head of the pet and positioning the extended node on the chin of the pet.

Description

DEVICES AND METHODS FOR AUTOMATICALLY CLASSIFYING PET MOTION AS

CHEWING OR NON-CHEWING

BACKGROUND

[0001] A variety of miniaturized sensors for motion and movement analysis have recently been developed due to the progress of semiconductor technologies in the last forty years. As a result, there are several commercially available devices to track activity in human beings.

However, these systems cannot be directly used for pets because they only provide pre- processed overall activity values that are pre-calculated and tuned for humans. In this regard, most existing methods for food intake analysis involve answering questionnaires or filling logs on a phone or computer.

[0002] Human chew detection has very few commercially available embedded systems solutions available. For example, a wrist-wearable device detects the movement of the human’s hand to detect how much food is being eaten. This solution cannot be easily adapted for pet food intake monitoring. An alternative approach would be a tooth sensor to detect pet eating habits. However, a device that is non-intrusive would be easier to use on pets.

[0003] In“Automatic food intake detection based on swallowing sounds” to O. Makeyev et al. , Biomed Signal Process Control, 7(6):649-656 (Nov. 2012), the authors presented methods for food intake detection by analyzing the swallowing sounds. These methods cannot be used in a standalone manner because pets have noise sources that make it difficult to accurately detect chew count from sound alone.

[0004] In“Recognition and volume estimation of food intake using a mobile device” to M. Purwe et al., Proc. of Workshop on Applications of Computer Vision (WACV), pages 1 -8, IEEE Press (2009), computer vision techniques were used to detect food intake using a camera- based system. For pet food intake monitoring, such a system is difficult to implement because pets move around and their faces cannot be tracked easily by cameras.

[0005] In“An intelligent food-intake monitoring system using wearable sensors” to J. Liu et al., Proc. of International Conference on Wearable and Implantable Body Sensor Networks, pages 154-160, IEEE Computer Society (2012), the authors presented a device that can be worn on the ear of a person and can be used detect the food intake. The device has a microphone that detects chew sounds and further uses a video camera that records the intake of food from the plate. Using these two data, the device detects the amount of food intake. However, a video-based recording of the plate or bowl of a pet does not provide accurate tracking of food intake. Moreover, installing the device on the ear of a pet such that the view of the plate or the bowl is sufficient would be incredibly difficult. In addition, the microphone for detecting chew sounds is unlikely to give high accuracy for pets because the ambient noise of pets is high.

[0006] In summary, different types of sensors have been used in the recent years to develop a variety of methods to detect chew count of humans, such as using optical systems, inertial or acoustic sensors. However, as detailed above, all these sensors have a number of shortcomings. For example, an optical analysis method can be used to detect the chew count without the need for additional hardware on the user, but this method requires the subject’s face to be clearly visible to the camera. Thus, the applicability is limited to controlled experiments and for short-term usage. An automatic method to obtain chew count from a video recording is even more challenging.

[0007] In addition, the detection of chew counts of pets poses new challenges when compared to that of humans. In particular, pets have a significant number of movements of the head and body that can present difficulties in detecting the chews from inertial sensors. They also produce a large variety of different sound patterns, thereby making it difficult to track the chews by using only an acoustic sensor.

SUMMARY

[0008] The present inventors recognized that a device having a three-axis accelerometer and a microphone could be used to obtain the movements and noise of chewing. However, the accuracy of the movement to distinguish the actual chewing operation from other head movements is not perfect, and thus an orientation sensor (e.g., gyroscope) can be included to enhance this differentiation of head movements in order to provide the different angles of system movement. Therefore, the combination of an accelerometer with this multi-sensor system can provide excellent accuracy for embedded processing and recording of both the accelerations and the angular movements. Consequently, a chewing event can be

distinguished from non-chewing motion.

[0009] Accordingly, in a general embodiment, the present disclosure provides a device configured to be worn by a pet and comprising: base hardware comprising a processor and one or more first inertial sensors; and an extended node electrically connected to the base hardware and comprising one or more second inertial sensors, the one or more first inertial sensors configured to obtain a first inertial measurement associated with a motion by the pet, the one or more second inertial sensors configured to obtain a second inertial measurement associated with the motion by the pet, and the processor configured to classify the motion by the pet as a chewing event or a non-chewing event based on the first inertial measurement and the second inertial measurement.

[0010] An advantage of one or more embodiments provided by the present disclosure is classifying pet chewing apart from other movements with high accuracy despite a pet typically having much more movement of the head and the body than a human. Another advantage of one or more embodiments provided by the present disclosure is to use a reference system to remove ambient movements that are not associated with chewing. Yet another advantage of one or more embodiments provided by the present disclosure is to use extended multi- parametric sensor recordings to allow the development of chewing identification algorithms having low-complexity processing and computational requirements. Still another advantage of one or more embodiments provided by the present disclosure is to at least match the accuracy of video-based systems for classifying pet chewing. Additional features and advantages are described herein and will be apparent from, the following Detailed Description and the Figures.

BRIEF DESCRIPTION OF THE FIGURES

[0011] FIG. 1 is an architectural block diagram of an embodiment of a device disclosed herein.

[0012] FIG. 2 is an illustration of implementation of an embodiment of a device disclosed herein on a pet.

[0013] FIG. 3 is an illustration of another implementation of an embodiment of a device disclosed herein on a pet.

[0014] FIG. 4 contains graphs of a zoomed window for meal duration for a dog named“Abe” in the study disclosed herein.

[0015] FIG. 5 is a graph of number of events in manually-annotated files vs. number of predicted events obtained by the proposed signal processing flow over all meal recordings in the study disclosed herein.

[0016] FIG. 6 is a graph of number of events in manually-annotated files vs. number of predicted events obtained by the proposed signal processing flow over all treat recordings in the study disclosed herein.

[0017] FIG. 7 is a table of results of meal records using the automated-accounting signal processing flow in the study disclosed herein.

[0018] FIG. 8 is a table of results of treat records using the automated-accounting signal processing flow in the study disclosed herein. DETAILED DESCRIPTION

Definitions

[0019] Some definitions are provided hereafter. Nevertheless, definitions may be located in the“Preferred Embodiments” section below, and the above header“Definitions” does not mean that such disclosures in the Preferred Embodiments” section are not definitions.

[0020] As used in this disclosure and the appended claims, the singular forms“a,”“an” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to“a pet” or“the pet” includes two or more pets and“a motion” and“the motion” includes two or more motions. The term“and/or” used in the context of“X and/or Y” should be interpreted as“X,” or“Y,” or“X and Y.” The term“at least one of’ used in the context of“at least one of X or Y” should be similarly interpreted as“X,” or“Y,” or“X and Y.” Where used herein, the term“example,” particularly when followed by a listing of terms, is merely exemplary and illustrative, and should not be deemed to be exclusive or comprehensive.

[0021] As used herein,“about” is understood to refer to numbers in a range of numerals, for example the range of -10% to +10% of the referenced number, preferably within -5% to +5% of the referenced number, more preferably within -1 % to +1 % of the referenced number, most preferably within -0.1 % to +0.1 % of the referenced number. A range that is“between” two values includes those two values. Furthermore, all numerical ranges herein should be understood to include all integers, whole or fractions, within the range. Moreover, these numerical ranges should be construed as providing support for a claim directed to any number or subset of numbers in that range. For example, a disclosure of from 1 to 10 should be construed as supporting a range of from 1 to 8, from 3 to 7, from 1 to 9, from 3.6 to 4.6, from 3.5 to 9.9, and so forth.

[0022] “Substantially the same time” means within 0.1 seconds of each other, preferably with 0.01 seconds of each other, more preferably within 0.001 seconds of each other, most preferably within 0.0001 seconds of each other. “Synchronous” and“synchronously” mean that each measurement in a first set of measurements is obtained at substantially the same time as a counterpart in a second set of measurements. For example, a first set of measurements obtained synchronously with a second set of measurements means that the first set of measurements has a first measurement obtained at substantially the same time as a first measurement in the second set, the first set of measurements has a second measurement obtained at substantially the same time as a second measurement in the second set, the first set of measurements has a third measurement obtained at substantially the same time as a third measurement in the second set, and so on. [0023] As used herein, a“peak” is preferably one or more measurements that are larger than the measurement immediately preceding the one or more measurements and the measurement immediately following the one or more measurements. In an embodiment, a “peak” is one or more measurements that are larger than the measurement immediately preceding the one or more measurements and the measurement immediately following the one or more measurements by an amount greater than a predetermined threshold.

[0024] The term“pet” means any animal which could benefit from or enjoy the compositions provided by the present disclosure. For example, the pet can be an avian, bovine, canine, equine, feline, hircine, lupine, murine, ovine, or porcine animal, but the pet can be any suitable animal. The term“companion animal” means a dog or a cat.

[0025] The terms “food,” “food product” and “food composition” mean a product or composition that is intended for ingestion by an animal and provides at least one nutrient to the animal. The term“pet food” means any food composition intended to be consumed by a pet.

[0026] The devices disclosed herein may lack any element that is not specifically disclosed herein. Thus, a disclosure of an embodiment using the term“comprising” includes a disclosure of embodiments “consisting essentially of” and “consisting of’ the components identified. Similarly, the methods disclosed herein may lack any step that is not specifically disclosed herein. Thus, a disclosure of an embodiment using the term“comprising” includes a disclosure of embodiments “consisting essentially of” and “consisting of” the steps identified. Any embodiment disclosed herein can be combined with any other embodiment disclosed herein.

[0027] By bite is meant a dog has its head in bowl and is actively taking food into its mouth. Each re-opening of the mouth to bring in food is considered a new bite.

[0028] By chew is meant a dog closes its jaw in order to process food already in its mouth; head may or may not still be in the bowl, but no new food is being brought into mouth from bowl. Each individual closure of the jaw should be counted as 1 chew.

Embodiments

[0029] The present inventors recognized that a device having a three-axis accelerometer and a microphone could be used to obtain the movements and noise of chewing. However, the accuracy of the movement to distinguish the actual chewing operation from other head movements is not perfect, and thus an orientation sensor (e.g., gyroscope) can be included to enhance this differentiation of head movements in order to provide the different angles of system movement. Therefore, the combination of an accelerometer with this multi-sensor system can provide excellent accuracy for embedded processing and recording of both the accelerations and the angular movements. Consequently, a chewing event can be distinguished from non-chewing motion.

[0030] Accordingly, in a general embodiment, the present disclosure provides a device configured to be worn by a pet and comprising: base hardware comprising a processor and one or more first inertial sensors; and an extended node electrically connected to the base hardware and comprising one or more second inertial sensors, the one or more first inertial sensors configured to obtain a first inertial measurement associated with a motion by the pet, the one or more second inertial sensors configured to obtain a second inertial measurement associated with the motion by the pet, and the processor configured to classify the motion by the pet as a chewing event or a non-chewing event based on the first inertial measurement and the second inertial measurement.

[0031] In an embodiment, the one or more first inertial sensors comprises a first three-axis accelerometer and a first three-axis gyroscope. The one or more second inertial sensors can comprise a second three-axis accelerometer and a second three-axis gyroscope.

[0032] In an embodiment, the classifying by the processor comprises comparing the second inertial measurement to a baseline of the one or more second inertial sensors.

[0033] In an embodiment, the classifying by the processor comprises comparing the second inertial measurement to the first inertial measurement. The first inertial measurement is preferably obtained at substantially the same time as the second inertial measurement.

[0034] In an embodiment, the base hardware comprises at least one component selected from the group consisting of a local storage, an element providing Bluetooth connectivity, and a battery.

[0035] In an embodiment, during a time period, the one or more first inertial sensors obtain a plurality of first inertial measurements comprising the first inertial measurement synchronously with the one or more second inertial sensors obtaining a plurality of second inertial measurements comprising the second inertial measurement. The extended node can comprise a microphone, the processor can receive sound data from the microphone, and the processor can use the sound data from the microphone to determine at least one of a start of the time period or an end of the time period.

[0036] In an embodiment, the processor can use the plurality of first inertial measurements and the plurality of second inertial measurements to determine a number of chewing events for the time period. The determining of the number of chewing events by the processor can comprise comparing each of the plurality of second inertial measurements to a baseline of the plurality of second inertial measurements. The determining of the number of chewing events by the processor can comprise identifying one or more peaks in the plurality of second inertial measurements, and each of the one or more peaks is formed by one or more of the second inertial measurements. The determining of the number of chewing events can comprise comparing each of the one or more peaks to a corresponding portion of the plurality of first inertial measurements, and the corresponding portion of the plurality of first inertial measurements is one or more of the first inertial measurements that are obtained at substantially the same time as the one or more second inertial measurements that form the peak. Each of the one or more peaks can be classified as a chewing event if the amplitude of the peak exceeds the amplitude of the corresponding portion of the first inertial measurements and classified as a non-chewing event if the amplitude of the peak does not exceed the amplitude of the corresponding portion of the first inertial measurements.

[0037] In an embodiment, the processor can be configured to determine a bite force of each of the one or more peaks that is classified as a chewing event, and the bite force can be based on the amplitude of the peak classified as a chewing event.

[0038] In an embodiment, the base hardware is part of or affixed to a pet collar.

[0039] In another embodiment, the present disclosure provides a method of determining chewing activity of a pet. The method comprises: positioning a device on the pet, the device comprising base hardware comprising a processor and one or more first inertial sensors, the base hardware positioned on at least one position selected from the group consisting of a neck of the pet and a head of the pet, the device further comprising an extended node electrically connected to the base hardware and comprising one or more second inertial sensors, the extended node positioned on a chin of the pet.

[0040] The method comprises administering a food product to the pet; synchronously obtaining first inertial measurements on the one or more first inertial sensors and second inertial measurements on the one or more second inertial sensors during a time period comprising consumption of at least a portion of the food product by the pet; receiving the first inertial measurements and the second inertial measurements on the processor; and identifying one or more peaks in the second inertial measurements on the processor, each of the one or more peaks is formed by one or more of the second inertial measurements.

[0041] The method comprises classifying each of the one or more peaks as a chewing event or a non-chewing event, the processor performs the classifying based on the peak and a corresponding portion of the first inertial measurements, the corresponding portion of the first inertial measurements is one or more of the first inertial measurements that were obtained at substantially the same time as the one or more second inertial measurements that form the peak.

[0042] In an embodiment, at least one of (i) the identifying of the one or more peaks by the processor or (ii) the classifying of each of the one or more peaks as a chewing event or a non chewing event comprises comparing each of the second inertial measurements to a baseline of the second inertial measurements.

[0043] In an embodiment, the method comprises obtaining sound data on a microphone provided by the extended node, and the processor uses the sound data from the microphone to determine at least one of a start of the time period or an end of the time period.

[0044] In an embodiment, the method comprises determining a number of chewing events for the time period, and the processor uses the classifying of each of the one or more peaks as a chewing event or a non-chewing event to determine the number of chewing events for the time period.

[0045] In an embodiment, the classifying of each of the one or more peaks as a chewing event or a non-chewing event comprises comparing each of the one or more peaks to the corresponding portion of the first inertial measurements, and each of the one or more peaks is classified as a chewing event if the amplitude of the peak exceeds the amplitude of the corresponding portion of the first inertial measurements and classified as a non-chewing event if the amplitude of the peak does not exceed the amplitude of the corresponding portion of the first inertial measurements.

[0046] In an embodiment, the method comprises determining a bite force of each of the one or more peaks that is classified as a chewing event, and the processor determines the bite force based on the amplitude of the peak classified as a chewing event.

[0047] In another embodiment, the present disclosure provides a method of analyzing pet food products. The method comprises: positioning a device on a pet, the device comprising base hardware comprising a processor and one or more first inertial sensors, the base hardware positioned on at least one position selected from the group consisting of a neck of the pet and a head of the pet, the device further comprising an extended node electrically connected to the base hardware and comprising one or more second inertial sensors, the extended node positioned on a chin of the pet.

[0048] The method comprises administering the food products to the pet separately at different times; synchronously obtaining first inertial measurements on the one or more first inertial sensors and second inertial measurements on the one or more second inertial sensors during a time period comprising consumption of at least a portion of the food product by the pet; receiving the first inertial measurements and the second inertial measurements on the processor; and identifying one or more peaks in the second inertial measurements on the processor, each of the one or more peaks is formed by one or more of the second inertial measurements, each of the one or more peaks is formed by one or more of the second inertial measurements.

[0049] The method comprises classifying each of the one or more peaks as a chewing event or a non-chewing event for each of the food products, the processor performs the classifying based on the peak and a corresponding portion of the first inertial measurements, the corresponding portion of the first inertial measurements is one or more of the first inertial measurements that were obtained at substantially the same time as the one or more second inertial measurements that form the peak.

[0050] In an embodiment, the method comprises, for each of the food products, determining a number of chewing events for the time period, the processor uses the classifying of each of the one or more peaks as a chewing event or a non-chewing event to determine the number of chewing events for the time period, the method further comprising comparing the number of chewing events for one of the food products to the number of chewing events for another one of the food products.

[0051] In an embodiment, the method comprises, for each of the food products, determining a bite force of each of the one or more peaks that is classified as a chewing event, the processor determines the bite force based on the amplitude of the peak classified as a chewing event, the method further comprising comparing the bite force for one of the food products to the bite force for another one of the food products.

[0052] As generally illustrated by FIG. 1 , an aspect of the present disclosure is a device 10 for automatically classifying pet motion as chewing or non-chewing, for example by automatically detecting one or more chewing events by the pet. The device 10 can comprise base hardware 20 and an extended node 30. The base hardware 20 can comprise one or more first inertial sensors 21 , preferably a combination of a first three-axis accelerometer and a first three-axis gyroscope (e.g., a first three-axis accelerometer and a first three-axis gyroscope on the same chip, such as a MPU6000D chip). The extended node 30 can comprise one or more second inertial sensors 31 , preferably a combination of a second three-axis accelerometer and a second three-axis gyroscope (e.g., a second three-axis accelerometer and a second three- axis gyroscope on the same chip, such as a MPU6000D chip). The one or more first inertial sensors 21 can cooperate with the one or more second inertial sensors 32 to detect chewing motion and remove ambient movements that are not associated with chews, as discussed in greater detail later herein.

[0053] The base hardware 20 (e.g., one or more of its components) can be physically connected and/or electrically connected to the extended node 30 (e.g., one or more of its components), preferably by a wire 40. Optionally, the extended node 30 further comprises a microphone 32. Each of the one or more second inertial sensors 31 and the microphone 32 can be contained by and/or affixed to a housing 36 of the extended node 30.

[0054] FIGS. 2 and 3 generally illustrate use of the device 10 on a pet (e.g., a dog) for real time chewing accounting. The base hardware 20 preferably can be positioned on the forehead or neck of the pet, and/or the extended node 30 preferably can be positioned on the chin of the pet. As shown in FIG. 3, an embodiment of the device 10 is a collar, i.e., any apparatus that circumscribes the neck of the pet and reversibly removable therefrom. For example, the base hardware 20 can be attached to a flexible band of material, a portion of the wire 40 can be positioned on and/or within the band of material, and the remainder of the wire 40 can extend from the band of material to the extended node 30 which is preferably positioned on the chin of the pet.

[0055] In a preferred embodiment, the one or more first inertial sensors 21 can track activity and/or movement of the head and/or the body of the pet, and a processor 22 (e.g., a microcontroller) can use these measurements to compensate the associated measurements from the one or more second inertial sensors 31 . For example, the movements of the pet’s jaw due to chewing events can be captured on the one or more second inertial sensors 31 and not on the one or more first inertial sensors 21 and thus identified by the processor 22 for chewing analysis. Movements that are captured by the one or more first inertial sensors 21 can be classified as non-chewing movement by the processor 22. As another example, inertial data from the one or more second inertial sensors 31 can be classified as chewing event if the inertial data from the one or more second inertial sensors 31 is larger than the corresponding inertial date from the one or more first sensors 21 (e.g., larger in amplitude). The corresponding inertial data is preferably inertial data that was obtained at substantially the same time.

[0056] The classification can be based on comparison of peaks in the inertial data from the one or more second sensors 31 to a baseline of the inertial data from the one or more second sensors 31. In an embodiment, the number of peaks above the baseline is classified as the number of chewing events. In another embodiment, each the peaks above the baseline are then compared to a corresponding portion of the inertial data from the one or more first inertial sensors 21 and classified as a chewing event if the amplitude of the peak exceeds the corresponding portion of the inertial data from the one or more first inertial sensors 21 . Preferably, the portion of the inertial data from the one or more second sensors that forms the peak is obtained at substantially the same time as the comparative portion of the inertial data from the one or more first inertial sensors 21 .

[0057] Preferably the classification is automatic such that no user input is needed for processing and classifying the inertial data from the one or more first inertial sensors 21 and the one or more second inertial sensors 31 to be a chewing event or non-chewing event. Preferably the classification does not use any video data. In this regard, a preferred embodiment of the device 10 does not include any component that obtains video, i.e., preferably none of the components of the device 10 obtain video.

[0058] The base hardware 20 preferably further comprises one or more of local storage 23 (e.g., a memory card), an element that provides Bluetooth connectivity 24, or a battery 25 that supplies power to the other components of the device 10. The local storage 23 can store the inertial data from the one or more first inertial sensors 21 and/or the one or more second inertial sensors 31 and can store any microphone date from the microphone 32 (if present). The Bluetooth element 24 can transmit the inertial data and optionally any sound data from the device 10 to a phone or computer, for example in real-time. Each of the one or more first inertial sensors 21 , the processor 22, the local storage 23, the Bluetooth element 24, and the battery 25 can be contained by and/or affixed to a housing 26 of the base hardware 20.

[0059] As a non-limiting example, the processor 22 can be a microcontroller having multiple Serial Peripheral Interface (SPI) ports (preferably at least three), an audio interface, and a large onboard RAM for processing (e.g., at least about 50 KB RAM, preferably about 100 KB RAM). In some embodiments, the processor 22 has one or more Universal Synchronous/Asynchronous Receiver/Transmitter (USART) and one or more Inter-Integrated Circuit (I2C) ports. The processor 22 can be any component of the device 10 suitable for processing the inertial data from the one or more first inertial sensors 21 and/or the one or more second inertial sensors 31 ; processing any sound data from the microphone 32 (if present); and classifying the inertial data as representative of chewing or non-chewing, and the present disclosure is not limited to a specific embodiment of the processor 22.

[0060] As a non-limiting example, the one or more first inertial sensors 21 and/or the one or more second inertial sensors 31 can comprise a three-axis accelerometer and a three-axis gyroscope with a high sampling rate, e.g., up to 8 KHz for the gyroscope and up to KHz for the accelerometer. The one or more first inertial sensors 21 and the one or more second inertial sensors 31 can each be any component of the device 10 suitable for measuring acceleration and/or orientation, and the present disclosure is not limited to a specific embodiment of the inertial sensors.

[0061] As a non-limiting example, the microphone 32 can be an analog microphone, such as microelectro-mechanical systems (MEMS) microphone having low power and/or low noise. The microphone 32 can be any component of the device 10 suitable for converting sound from the pet into an electrical signal, and the present disclosure is not limited to a specific embodiment of the microphone 32.

[0062] As a non-limiting example, the local storage 23 can be an external memory card that is removable, such as a secure digital (SD) memory card. Preferably the local storage can store at least one month of data. The local storage 23 can be any component of the device 10 suitable for storing inertial data from the one or more first inertial sensors 21 and the one or more second inertial sensors 31 and optionally any sound data from the microphone 32 (if present), and the present disclosure is not limited to a specific embodiment of the local storage 23.

[0063] As a non-limiting example, the battery 25 can be a rechargeable battery, such as a battery rechargeable by a USB connection. The battery 25 can be any source of electric power suitable for the components of the device 10, and the present disclosure is not limited to a specific embodiment of the battery 25. The housing 26 can be configured to allow replacement and/or recharging of the battery 25. In an embodiment, the wire 40 carries power from the battery 25 to the extended node 30 and/or the components of the extended node 30.

[0064] In a non-limiting example, the base hardware 20 is generally a rectangular cuboid and is about 3 cm long in a first dimension (e.g., width), about 4 cm long in a second dimension (e.g., height) and about 8 cm long in a third dimension (e.g., length). In a non-limiting example, the extended node 30 is generally a rectangular cuboid and is about 1 .5 cm long in a first dimension (e.g., width), about 4 cm long in a second dimension (e.g., height) and about 4 cm long in a third dimension (e.g., length). Nevertheless, the device 10 is not limited to any specific size of the base hardware 20 and the extended node 30.

[0065] Another aspect of the present disclosure is a method of automatically classifying pet motion as chewing or non-chewing, for example a method of automatically detecting chewing. The method can comprise positioning a device on the pet; the device can be the device 10 disclosed above or any device capable of performing one or more of the steps of the method set forth below.

[0066] The method can comprise initiating recording of inertial data from one or more first inertial sensors (e.g., a combination of a first three-axis accelerometer and a first three-axis gyroscope) positioned at a first location on a pet (e.g., neck or head) and/or initiating recording of inertial data from one or more second inertial sensors (e.g., a combination of a second three- axis accelerometer and a second three-axis gyroscope) positioned at a second location on the pet different than the first location (e.g., chin).

[0067] For example, the recording can be initiated in response to activation of a device that provides the first and second sets of inertial sensors, such as by powering on the device or providing user input on the device. As another example, the recording can be initiated in response to detection of sound by a microphone (e.g., sound having an intensity above a predetermined threshold), and the microphone can be part of the device providing the first and second sets of inertial sensors, for example part of an extended node that provides the one or more second inertial sensors and is electrically connected to a first node that provides the one or more first inertial sensors.

[0068] In an embodiment, the recording of the inertial data comprises data conversion. For example, the raw inertial data can be converted into binary format.

[0069] The method can comprise processing the inertial data from the first and second sets of inertial sensors. In an embodiment, the processing comprises segmenting the inertial data, for example truncating the inertial data, preferably based on sound data obtained by the microphone. In embodiments where sound data is used, the processing preferably comprises applying a Short-Time Fourier Transform (STFT) on the sound data, optionally followed by decomposing the STFT output (e.g., in both x and y gradients) and reconstructing the image (e.g., by simple integration of the y-axis decomposition part). The STFT can be subject to additional enhancement, for example by convolution of the STFT.

[0070] The processing of the inertial data can comprise using the magnitude of the inertial data from the one or more second inertial sensors to identify one or more events of chewing activity. For example, the baseline of the inertial data from the one or more second inertial sensors can be removed, then the non-chewing activity can be removed, and then each of the peaks in the resulting signal can be identified and classified as chews. Preferably, the inertial data from the one or more first inertial sensors can also be used to identify the non-chewing activity. For example, a portion of the inertial data from the one or more second sensors 31 can not be classified as chewing activity and/or can be classified as non-chewing activity if the amplitude of the inertial data from the one or more first inertial sensors meets or exceeds the amplitude of the corresponding inertial data from the one or more second of inertial sensors (i.e., the inertial data obtained at substantially the same time). [0071] In an embodiment, the recording of the inertial data can comprise storing the inertial data and optionally any sound data from the microphone (if present) on local storage provided by the device, for example on local storage provided by the base hardware. In such an embodiment, the processing of the inertial data can occur subsequent to the recording of the inertial data by the sensors. However, the inertial data and any sound data are preferably processed in real-time (i.e., substantially concurrently with its recording by the sensors), for example by a processor (e.g., a microcontroller) provided by the device (e.g., provided by the base hardware). The inertial data processed by the processor can be transmitted to a second device, for example a phone or a computer (e.g., by Bluetooth from an element provided by the device, preferably by the base hardware).

[0072] The method can comprise using the chewing activity to perform further calculation, for example using the amplitude of the chewing events to calculate the bite force applied while chewing the food and/or using the number of chewing events over the time period to determine chewing rate. In turn, the bite force and/or the chewing rate can be used to investigate the affect of one or more of size, shape or texture of a petfood. For example, the bite force and/or the chewing rate can be used to maximize appropriateness of a petfood for oral health and/or pet appeal, e.g., for a specific pet size or a specific pet breed. As a specific non-limiting example, a plurality of petfoods (e.g., petfoods having different textures) can be separately administered to one or more pets of a particular size and/or a particular breed; the bite force and/or the chewing rate can be determined for each of the petfoods; and the bite force and/or the chewing rate can be used to determine which of the petfoods is most appropriate for the size and/or breed.

EXAMPLE

[0073] By way of example and not limitation, the following non-limiting example study is illustrative of devices and methods for automatically classifying pet motion as chewing or non chewing in one or more embodiments provided by the present disclosure.

[0074] This study developed an off-line Matlab-based signal processing framework for the analysis of chewing cycles using the inventive device. The definition of “off-line” refers to this processing flow not running in real-time on the device but instead as a separate Matlab flow that uses the information gathered from the sensors during the study with dogs.

[0075] This study was conducted with different dog sizes. Each dog was given two types of meals: the main meal and a treat. During the study, each dog was always wearing an embodiment of the inventive device that featured two sets of sensors used to capture the feeding activity (chewing/bite, etc.). The dog activity was also video-recorded and further manually classified into one of the feeding categories by experimenters, and video scoring files were obtained.

[0076] The main goal of the study was to see if a complete flow (i.e. , set of algorithms) could detect the feeding activity only by using the data obtained from the accelerometers, gyroscopes and microphone included in the base hardware and extended node of the device, thus automating the process and removing the need for manual video annotations by experts. To verify the results of the proposed signal processing flow in this study, video scoring files were used.

[0077] As explained below, the obtained average performance of the proposed signal processing flow is > 90%, which proves that feeding activities in dogs can be detected in an automated way with a high level of accuracy with respect to manual annotations from trained operators using video-recordings. Moreover, the automated calculation merely requires a couple of minutes per each dog, in contrast to manual annotations by experts requiring several hours.

Experimental Setup

[0078] The device used in the study is described in detail earlier in this application and thus this section only provides brief description of the main features of the device, specifically the sensors used in the signal processing flow. The tested device had two sets of sensors, namely, accelerometers and gyroscopes (ACC + Gyro). The accelerometer and gyroscope had a high correlated accuracy as they were included in the same chip (reference MPU6000D). One pair was located on the master board, whereas the second pair was located on the slave board. Additionally, one microphone (reference ADMP441 ) was located on the slave board, and this microphone was only used to perform the segmentation of the relevant phases of treat and main meal. The specifications of these sensors are given in the table shown below. The exact use of these different sensors is detailed hereafter.

[0079] The study was conducted on 20 dogs (Abe, Annikan, Apollo, Beam, Doc, Fischer, Floppy, Honest, Jay, Kenworth, Maverick, Peace, Peter, Sausage, Shaggy, Shilo, Shimmer, Sid, Star and Victoria). All recordings were done for each dog individually, and the entire set of experiments was video-recorded. During each recording, each dog was given a main meal and a treat. Video files were checked by experimenters to capture feeding events manually (chews, bite, etc.) by visually reviewing and annotating the observed events in the video frame sequences (“video scoring”). Output files of these video scorings were compared with the automated results of the device. While videos were recorded during the study, each dog wore the inventive device to record the activity during feeding. The base hardware was worn on the dog’s head, whereas the extended node was attached to a collar, close to the dog’s chin. After finishing each experiment, the date of the recording was extracted from the node and sent to the computer.

[0080] The main goal of the proposed signal processing flow to analyze the gathered data was to find the feeding activity only by using the data obtained from the sensors in order to automate the process and remove the need for a manual video annotation. To this end, the output of the video scoring obtained by an expert was used as the standard to which the automated results were compared.

[0081] To apply this flow, the recording time of the device and the video recordings was synchronized for comparison over an identical time frame. This synchronization was achieved by playing a beep signal (single tone) at a specific frequency (3, 156 Hz), which was heard both on the video recording and also by the microphone on the extended node. Beep signals were each played for three seconds. The first beep indicated the start of the main meal, while the following beep indicated its end. Relying on the same type of sound, the third beep was considered to indicate the start of a treat, and the fourth beep indicated its end. However, not all recordings of dogs contain a treat because sometimes dogs did not eat it, which means that only the first two beeps are present in that case and compared in the results of this study.

[0082] The original data from each experiment was provided in a .txt format recorded in the inventive device. There was a subfolder for each dog, and each subfolder contained the accelerometer/gyroscope data obtained from the base hardware, the accelerometer/gyroscope data obtained from the extended node, and the sound data obtained from the extended node.

[0083] Each piece of data from the accelerometer and gyroscope sensor contained six values per line, which represented the three axes of the accelerometer and of the gyroscope. Each value had a resolution of 2 bytes (16 bits) following this sequence: AcceIX, AccelY, AccelZ, GyrX, GyrY, GyrZ. Finally, the data coming obtained with the microphone contained one value per line, and each of them had a resolution of 3 bytes (24 bits).

[0084] The initially recorded data was written in text format. To speed up the processing (to have a fast and an efficient read/write considering the large sizes of the sensor files), a first pre processing step was included in the proposed signal processing flow. Specifically, the ACC/Gyro files were converted into 16-bits signed integers in binary format, whereas sound data were converted into 32-bits signed integers (24-bit binary operators do not exist in Matlab, which accepts either 16 or 32 bits resolutions for data management). These were new binary files.

[0085] The video scoring files were provided in .txt format, and no extra inputs were provided for comparison of the results with the proposed automated signal processing flow. The data in each file was tabulated where columns’ titles are written in the header line. The columns’ titles for each video scoring file were as follows: Onset Time, Offset Time, Not Visible, Non-Feeding Behavior, Actively Chewing, Chew, Bite, and Duration Time. Then, feeding events were indicated by four columns: Non-Feeding Behavior, Actively Chewing, Chew, and Bite. Each row contained the onset and offset of the timing. The duration of feeding events (Bite and Chew) was always fixed to 0.01 seconds. This duration indicated only the presence of an event and not an actual onset nor offset, due to the difficulty in their capture. The event called “Feeding Duration” in the“Actively Chewing” column indicated the whole duration of the feeding event. This line indicated the duration where the onset and offset of the event was annotated by the expert. Then, another relevant event to analyze the accuracy of the automated results count reported in this result was the“Non-Visible” event, which indicated that the expert was not able to clearly recognize the feeding event. For such periods, there was no annotation. Thus, these events were discarded in the final comparison between results obtained from the proposed algorithm and those obtained using the video scoring standard.

[0086] Finally, the complete script that reads and extracts the information from these aforementioned files was included in the provided analysis framework (e.g., Matlab code).

Signal Processing Flow

[0087] The flowchart of the proposed signal processing flow is shown below. Each of the indicated phases (or signal processing blocks) is described hereafter. The application of these blocks will be illustrated in different case studies (i.e. , dogs’ data sets) for clarity.

[0088] In the block“Reading data (ACC, Gyro, microphone),” the data obtained from the device was read. As previously mentioned, this data contained the data already converted into the binary format for the rest of the signal processing flow.

[0089] In the block “Finding beep signals in the sound data,” the internal microphone included in the extended node was used to segment different parts of the ACC/Gyro signals (main meal and treat). This segmentation was performed by exploiting the synchronous recording of the data in the device. As mentioned above, the start and end of each feeding event was indicated by a beep signal. Therefore, the first beep signals (tones at frequency f = 3, 156Hz) in the sound data were used to identify times to extract the events from the ACC/Gyro data files, i.e., with an algorithm which performs this beep extraction from the sound data.

[0090] After segmenting the data (i.e., identifying the relevant periods of data to process) the next step in the flow was the application of a Short-Time Fourier Transform (STFT)-based approach to find the location of the aforementioned events in the ACC/Gyro signals. The STFT was calculated only on the relevant frequency window, considering the fact that locations of tones in the frequency domain are known. Hence, a 1024-point symmetric Blackman window could be used in each processing step to reconstruct the STFT. The processing flow makes all the tones stand out from the original signal with high precision. Noise still needs to be filtered, so an extra step was applied.

[0091] To enhance the STFT analysis, in the processing flow the STFT was decomposed in both x and y gradients (called g_x and g_y). Then, the image was reconstructed from the g_y decomposition part by simple integration. As a result, all the noise were removed, and a clear representation of the signal (without noise) to find tones was obtained. To finally improve the STFT against real-life noise sources (e.g., dog’s body movements), the plane was folded from the frequency of interest to emphasize tones. Finally, the averaging over neighboring frequency can be used to find the events’ peaks. These peaks can be considered as centers of three- second recorded beep signals. After this step, peaks were easily detected in the transformed data. Indeed, based on their positions, parts of ACC/Gyro signals corresponding to the main meal and treat could be clearly segmented.

[0092] In FIG. 4, the complete ACC and Gyro signals for the dog called Abe is shown as an example of this process. The duration of this recording was 580 seconds. Black vertical lines represent beep signals. In the block“Data Processing,” the signal processing flow obtains the events of dog chewing activity. To obtain the events of dog chewing activity, only the ACC/Gyro data obtained from the extended node was needed. In particular, the entire processing in this step was based on the magnitude of the ACC data. After removing its baseline, all non-feeding activities were removed, and peaks in the resulting signal were extracted. These peaks represented the feeding activity.

[0093] Besides the average-based algorithm used in this block of the processing flow, an alternative implementation of events identification relying on a Kalman filter was developed, but this method was deprecated in the final flow due to the high complexity with respect to the idea of targeting an embedded implementation. Moreover, the Kalman filter approach is significantly more complex than the current average-based method and does not achieve significantly better results.

[0094] In the block“Comparing Results,” a number of methods were used to generate graphs that compare the results obtained with the manual scoring files with the automated counting approach.

Results

[0095] This section summarizes the obtained results after applying the signal processing flow to the data from twenty dogs (Abe, Annikan, Apollo, Beam, Doc, Fischer, Floppy, Honest, Jay, Kenworth, Maverick, Peace, Peter, Sausage, Shaggy, Shilo, Shimmer, Sid, Star and Victoria). Among all records, only the one for“Honest” was not usable because of recording inconsistencies in the experimental procedure. Apart from this case, the annotated data for “Annikan,” both for the main meal and for the treat, seemed to have inconsistent labeling in the video-recording file. Hence, the data for“Honest” and“Annikan” was excluded.

[0096] In particular, FIGS. 5 and 6 show the number of events found by the signal processing flow versus the actual number of annotated events identified manually in video files for meal and treat records, respectively. The detailed list of number of events along with the performance results of the algorithm are shown for all the considered dogs in the tables in FIGS. 7 and 8, which distinguishes the mail meal and treat records. However, not all names of dogs appear in these tables because in some cases the treat or meal was not taken by certain dogs.

[0097] Finally, the overall formula used for calculating the performance of the signal processing flow was the following:

[0098] Performance = 100 x (1 - |a - b\la)

[0099] where a represents the number of manually annotated events, and b represents the number of events found by the proposed signal processing flow. Conclusions

[00100] This study describes the complete signal processing flow for detecting in an automated way the feeding activity of dogs. The presented results demonstrate that this processing flow reaches a high level of accuracy, i.e., more than (> 90%) overall, when compared to the manually-annotated (and time-consuming) standard results using video recordings. These results demonstrate that the process of capturing the feeding activity of dogs can be automated and the need for a manual video annotation can be removed, which can have a dramatic improvement in processing speed to analyze the chewing patterns of dogs in experimental tests of different types of food.

[00101] It should be understood that various changes and modifications to the presently preferred embodiments described herein will be apparent to those skilled in the art. Such changes and modifications can be made without departing from the spirit and scope of the present subject matter and without diminishing its intended advantages. It is therefore intended that such changes and modifications be covered by the appended claims.

Claims

CLAIMS The invention is claimed as follows:

1. A device configured to be worn by a pet and comprising:

base hardware comprising a processor and one or more first inertial sensors; and an extended node electrically connected to the base hardware and comprising one or more second inertial sensors, the one or more first inertial sensors configured to obtain a first inertial measurement associated with a motion by the pet, the one or more second inertial sensors configured to obtain a second inertial measurement associated with the motion by the pet, and the processor configured to classify the motion by the pet as a chewing event or a non chewing event based on the first inertial measurement and the second inertial measurement.

2. The device of Claim 1 wherein the one or more first inertial sensors comprises a first three-axis accelerometer and a first three-axis gyroscope.

3. The device of Claim 2 wherein the one or more second inertial sensors comprises a second three-axis accelerometer and a second three-axis gyroscope.

4. The device of Claim 1 wherein the classifying by the processor comprises comparing the second inertial measurement to a baseline of the one or more second inertial sensors.

5. The device of Claim 1 wherein the classifying by the processor comprises comparing the second inertial measurement to the first inertial measurement.

6. The device of Claim 5 wherein the first inertial measurement is obtained at substantially the same time as the second inertial measurement.

7. The device of Claim 1 wherein the base hardware comprises at least one component selected from the group consisting of a local storage, an element providing Bluetooth connectivity, and a battery.

8. The device of Claim 1 wherein, during a time period, the one or more first inertial sensors obtain a plurality of first inertial measurements comprising the first inertial measurement synchronously with the one or more second inertial sensors obtaining a plurality of second inertial measurements comprising the second inertial measurement.

9. The device of Claim 8 wherein the extended node comprises a microphone, the processor receives sound data from the microphone, and the processor uses the sound data from the microphone to determine at least one of a start of the time period or an end of the time period.

10. The device of Claim 8 wherein the processor uses the plurality of first inertial measurements and the plurality of second inertial measurements to determine a number of chewing events for the time period.

1 1 . The device of Claim 10 wherein the determining of the number of chewing events by the processor comprises comparing each of the plurality of second inertial measurements to a baseline of the plurality of second inertial measurements.

12. The device of Claim 10 wherein the determining of the number of chewing events by the processor comprises identifying one or more peaks in the plurality of second inertial measurements, and each of the one or more peaks is formed by one or more of the second inertial measurements.

13. The device of Claim 12 wherein the determining of the number of chewing events comprises comparing each of the one or more peaks to a corresponding portion of the plurality of first inertial measurements, the corresponding portion of the plurality of first inertial measurements is one or more of the first inertial measurements that are obtained at substantially the same time as the one or more second inertial measurements that form the peak.

14. The device of Claim 13 wherein each of the one or more peaks is classified as a chewing event if the amplitude of the peak exceeds the amplitude of the corresponding portion of the first inertial measurements and classified as a non-chewing event if the amplitude of the peak does not exceed the amplitude of the corresponding portion of the first inertial measurements.

15. The device of Claim 12 wherein the processor is configured to determine a bite force of each of the one or more peaks that is classified as a chewing event, and the bite force is based on the amplitude of the peak classified as a chewing event.

16. The device of Claim 1 wherein the base hardware is part of or affixed to a pet collar.

17. A method of determining chewing activity of a pet, the method comprising:

positioning a device on the pet, the device comprising base hardware comprising a processor and one or more first inertial sensors, the base hardware positioned on at least one position selected from the group consisting of a neck of the pet and a head of the pet, the device further comprising an extended node electrically connected to the base hardware and comprising one or more second inertial sensors, the extended node positioned on a chin of the pet;

administering a food product to the pet;

synchronously obtaining first inertial measurements on the one or more first inertial sensors and second inertial measurements on the one or more second inertial sensors during a time period comprising consumption of at least a portion of the food product by the pet;

receiving the first inertial measurements and the second inertial measurements on the processor;

identifying one or more peaks in the second inertial measurements on the processor, each of the one or more peaks is formed by one or more of the second inertial measurements; and

classifying each of the one or more peaks as a chewing event or a non-chewing event, the processor performs the classifying based on the peak and a corresponding portion of the first inertial measurements, the corresponding portion of the first inertial measurements is one or more of the first inertial measurements that were obtained at substantially the same time as the one or more second inertial measurements that form the peak.

18. The method of Claim 17 wherein at least one of (i) the identifying of the one or more peaks by the processor or (ii) the classifying of each of the one or more peaks as a chewing event or a non-chewing event comprises comparing each of the second inertial measurements to a baseline of the second inertial measurements.

19. The method of Claim 17 comprising obtaining sound data on a microphone provided by the extended node, and the processor uses the sound data from the microphone to determine at least one of a start of the time period or an end of the time period.

20. The method of Claim 17 comprising determining a number of chewing events for the time period, and the processor uses the classifying of each of the one or more peaks as a chewing event or a non-chewing event to determine the number of chewing events for the time period.

21 . The method of Claim 17 wherein the classifying of each of the one or more peaks as a chewing event or a non-chewing event comprises comparing each of the one or more peaks to the corresponding portion of the first inertial measurements, and each of the one or more peaks is classified as a chewing event if the amplitude of the peak exceeds the amplitude of the corresponding portion of the first inertial measurements and classified as a non-chewing event if the amplitude of the peak does not exceed the amplitude of the corresponding portion of the first inertial measurements.

22. The method of Claim 17 comprising determining a bite force of each of the one or more peaks that is classified as a chewing event, and the processor determines the bite force based on the amplitude of the peak classified as a chewing event.

23. A method of analyzing pet food products, the method comprising:

positioning a device on a pet, the device comprising base hardware comprising a processor and one or more first inertial sensors, the base hardware positioned on at least one position selected from the group consisting of a neck of the pet and a head of the pet, the device further comprising an extended node electrically connected to the base hardware and comprising one or more second inertial sensors, the extended node positioned on a chin of the pet;

administering the food products to the pet separately at different times;

identifying one or more peaks in the second inertial measurements on the processor, each of the one or more peaks is formed by one or more of the second inertial measurements, each of the one or more peaks is formed by one or more of the second inertial measurements; and classifying each of the one or more peaks as a chewing event or a non-chewing event for each of the food products, the processor performs the classifying based on the peak and a corresponding portion of the first inertial measurements, the corresponding portion of the first inertial measurements is one or more of the first inertial measurements that were obtained at substantially the same time as the one or more second inertial measurements that form the peak.

24. The method of Claim 24 comprising, for each of the food products, determining a number of chewing events for the time period, the processor uses the classifying of each of the one or more peaks as a chewing event or a non-chewing event to determine the number of chewing events for the time period, the method further comprising comparing the number of chewing events for one of the food products to the number of chewing events for another one of the food products.

25. The method of Claim 24 comprising, for each of the food products, determining a bite force of each of the one or more peaks that is classified as a chewing event, the processor determines the bite force based on the amplitude of the peak classified as a chewing event, the method further comprising comparing the bite force for one of the food products to the bite force for another one of the food products.