KR102026183B1 - Method and system for state analysis of pets using sensor technology - Google Patents

Abstract

A method and system for analyzing a pet's behavior using sensor technology and inferring the pet's mood or condition are disclosed. The pet condition analysis method includes receiving sensor data generated from a plurality of sensors including a wearable sensor worn on a pet, the sensor data including 3-axis acceleration data and 3-axis angular velocity data; Analyzing the pet's behavior in the sensor data to generate an action arrangement, the action arrangement including information about the pet's continuous actions for the plurality of time series sensor data; And inferring the mood or condition of the pet corresponding to the action arrangement.

Description

METHOOD AND SYSTEM FOR STATE ANALYSIS OF PETS USING SENSOR TECHNOLOGY}

An embodiment of the present invention relates to a technology for analyzing a pet's condition, and more particularly, to a method and system for analyzing a pet's behavior using sensor technology and inferring a pet's mood or condition based on the pet's behavior. .

Pets usually refer to animals that humans mainly breed for pleasure. Recently, pets have been popularized in the sense of pets in the sense of treating them as friends or companions in life, not as toys for human enjoyment.

Various research efforts for care or communication with these pets are underway. For example, in "Device and method for remotely monitoring animal behavior" of US Patent Publication No. 2010-0302004, a sensor is attached to an animal such as a horse, generates motion data through a sensor unit, and is transmitted to a remote location. Disclosed are a monitoring apparatus and a method for analyzing a motion data to detect a specific motion occurrence and identifying a specific behavior pattern of an animal based on a network determination algorithm prepared in advance.

According to the apparatus and method of the above publication, it is detected and transmitted to the manager by detecting a specific behavior pattern such as a pain situation requiring the intervention of the manager, thereby initiating the effective care of the animal.

However, the above-described prior art identifies a specific behavior pattern of an animal, but since the specific behavior pattern is for a specific situation such as a horse falling down, there is a very limited limitation in identifying the behavior pattern. Therefore, the above-described prior art is difficult to apply to animals such as dogs and cats having various kinds and sizes.

In addition, it is impossible to use the prior art as a method for caring for and communicating with animals such as dogs and cats. As such, there is a demand for a method of communicating with these animals while caring for animals such as dogs and cats, but practically no technology is applicable.

In order to solve the above-described problems of the prior art of the present invention, an object of the present invention is to provide a method and system for inferring the mood or condition of the pet through the analysis of the behavior of the pet based on the sensor technology.

Another object of the present invention is to provide a pet condition analysis method and system that can be used to predict the needs of the pet (needs) based on the inference of the pet's mood or condition, and can be actively utilized in communication with the pet or viewing the pet view To provide.

According to an aspect of the present invention, there is provided a method for analyzing a pet condition, the method comprising: receiving sensor data generated from a plurality of sensors including a wearable sensor worn on a pet; Data and triaxial angular velocity data; Analyzing the pet's behavior in the sensor data to generate an action arrangement, the action arrangement including information about the pet's successive behaviors for a plurality of time series sensor data; And inferring a mood or condition of the pet corresponding to the action arrangement.

Pet condition analysis system according to another aspect of the present invention for solving the above technical problem, the receiver receiving the sensor data generated from a plurality of sensors including a wearable sensor worn on the pet-the sensor data is three-axis acceleration Data and triaxial angular velocity data; An action array generator for analyzing an action of the pet in the sensor data to generate an action array, wherein the action array includes information about the continuous actions of the pet with respect to a plurality of time series sensor data; And a state inference unit that infers the mood or state of the pet corresponding to the action arrangement.

In the case of using the pet condition analysis method and system described above, the mood or condition of the pet can be reliably analyzed based on the pet's behavior inference. In addition, based on the condition analysis of the pet can increase the interaction with the pet and can effectively carry out the pet care.

In addition, the result of the behavior analysis of the pet using the sensor technology or the pet's mood or condition can be informed to the owner, thereby having the advantage of helping the communication between the pet and the person.

1 is a block diagram of the overall configuration of a pet care system including a pet condition analysis system according to an embodiment of the present invention.
FIG. 2 is an exemplary view of a wearable sensor that may be employed in the pet care system of FIG. 1.
3 is a block diagram of a pet condition analysis system according to an embodiment of the present invention.
4 is a block diagram of an action arrangement generator of the pet condition analysis system of FIG. 3.
5 is an exemplary diagram of sensor data that may be employed in the system of FIG. 3.
FIG. 6 is a diagram illustrating a behavior classification process in the pet condition analysis system of FIG. 3.
7 to 9 are diagrams for explaining the pet condition analysis method according to another embodiment of the present invention.
10 is a view for explaining an embodiment of the pet condition analysis method of FIGS.
11 is a view for explaining a pet condition analysis method according to another embodiment of the present invention.
12 is a view for explaining a specific operating principle of the pet condition analysis system according to another embodiment of the present invention.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the drawings, similar reference numerals are used for similar elements.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes any of a plurality of related listed items or a plurality of related listed items.

When a component is referred to as being 'connected' or 'connected' to another component, it may be understood that it may be directly connected or connected to the other component, but it may be understood that other components may exist in the middle. something to do. On the other hand, when a component is referred to as being 'directly connected' or 'directly connected' to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. As used herein, the terms "comprise", "having", and the like are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of the overall configuration of a pet care system including a pet condition analysis system according to an embodiment of the present invention. FIG. 2 is an exemplary view of a wearable sensor that may be employed in the pet care system of FIG. 1.

1 and 2, the pet care system according to the present embodiment includes a device 10, a gateway 30, a mobile terminal 50, and a service providing system 100. The service providing system 100 may be referred to as a monitoring system, a host system, or the like.

In more detail, each component may refer to a wearable device, and may include a sensor worn on the pet 2 or the pet. The sensor may include an acceleration sensor 3, a gyro sensor 5, an amplified pressure sensor, a sound sensor, a camera, and the like. The gyro sensor 5 may be referred to as an angular velocity sensor.

The device 10 may be provided with a microphone 7 capable of recording a pet's sound. The device 10 may have a necklace shape that is worn on the neck of a companion animal such as a puppy, but is not limited thereto. Depending on the implementation may include a microphone (7), a camera (not shown) is installed in the feeder (feeder) for discharging the food at a predetermined time to the dog (dog).

In addition, the device 10 may be equipped with a Bluetooth module (see 10a of FIG. 12) and connected to the gateway 30 or the mobile terminal 50 in a wireless communication manner to transmit and receive signals and data. The aforementioned device 10 may be referred to as a sensor device because it is equipped with a sensor, and may be referred to as a wearable device when the sensor is mounted and worn on the companion animal 2.

The gateway 30 connects various wearable devices or different types of sensors and the service providing system 100. The gateway 30 supports communication between different communication networks or networks using different protocols. Different protocols can be converted as appropriate. The gateway 30 may recognize an electric field of an OSI reference model to absorb another communication network of a transmission method, and may be installed to support different types of devices.

In this embodiment, the gateway 30 may be connected to the device 10 in a Bluetooth wireless communication scheme. The Bluetooth wireless communication scheme may include Bluetooth low energy (BLE). Of course, the gateway 30 may support other wireless local area network schemes, such as beacons and WiFi, depending on the implementation. In addition, the gateway 30 may support a wired network, a wireless network, or a combination thereof. Wired networks may include serial communications, Ethernet, and the like, and wireless networks may include short-range wireless communications networks such as WiFi and bluetooth, Wibro, code division multiple access (CDMA), orthogonal frequency multiple access (FDMA), and LTE ( It may include a broadband wireless network such as long term evolution (LTE), LTE-A (LET-advanced), WiMAX2, and the like.

In the case of a home or premises gateway, the pet user may acquire at least a part of sensor data, for example, image data or sound data, through a user terminal accessible to the gateway 30, and thereby may check the status of the pet in real time. (See S98 in FIG. 12).

The mobile terminal 50 may be connected to the device 10 including the wearable sensor. The mobile terminal 50 may relay signal and data transmission and reception between the device 10 and the service providing system 100. In addition, the mobile terminal 50 may process, display or output a part of the sensor data. The mobile terminal 50 may mount an application for a connection function, a relay function, a data processing function, and the like. The application may be referred to by a name such as 'pet buddy'. The above-described mobile terminal 50 may be a user terminal of the owner of the pet in terms of a device worn by a particular pet.

Data measured at the sensor of the device 10 includes sensor data or activity data. The sensor data may include signals and data such as body temperature, acceleration, angular velocity, sound, and images. In particular, the sensor data may comprise at least acceleration data and angular velocity data. In the following description, it is assumed that the sensor data includes activity data broadly and for convenience of explanation.

The service providing system 100 may include a system environment 110, an application programming interface (API) 130, an analysis server 150, and an SNS server 170. When the service providing system 100 uses the pet condition analysis method and shares information related to the mood or condition of the pet online or through the community, the service providing system 100 may be used as an online service providing system, a community service providing system, or a care service server system. May be referred to.

The service providing system 100 may have a cloud hierarchy. In one example, the service providing system 100 may provide software as a service (SaaS), platform as a service (PaaS), infrastructure as a service (IaaS), or a combination thereof.

SaaS may be referred to as software on demand. It can be configured to host software and related data as a central server or software as a service (SaaS). The user terminal may access a central server or SaaS through a client such as an application or a web browser.

PaaS may be referred to as a cloud computing service that supports cloud computing, which is a kind of internet-based computing. Here, the central server may be installed so that the user terminal can access anywhere (on demand) on configurable computer resources, including, for example, computer networks, servers, storage, applications, services, and the like. PaaS provides a platform for customers or programmers to develop, run, and manage applications, and can be configured to easily borrow the development elements that developers need. Public APIs provided on Internet sites are a kind of PaaS.

IssS turns servers, storage, and networks into virtualized environments so that infrastructure resources can be used as needed. IaaS can be selectively applied with technologies such as server virtualization, gateway virtualization, and user terminal virtualization.

In the above-described service providing system 100, the system environment 110 includes a controller in hardware or a processor and a memory, a predetermined server in software, a multicore, a virtual machine included in the server, At least some of the functional units or components selected from the software-based video / audio capturing, encoding unit, and streaming unit may be provided. System environment 110 may include a cloud computing server.

The API 130 may support sensor data collection and real time streaming. In addition, the API 130 is remote management; Database configuration management; It may be installed to perform status monitoring for logging for remote management, setting change, error handling, data transmission / reception cycle, camera on / off, resolution adjustment, log search, and the like.

That is, the API 130 may be connected to the system environment 110 to transmit and receive signals and data with the gateway 30 or the mobile terminal 50. The API 130 may collect detection signals or data detected by the sensor units of the sensor network connected to the gateway 30 and the mobile terminal 50. In this case, the API 130 may function as an information acquisition unit.

In addition, the API 130 may be a means for allowing a user terminal or a mobile terminal to access through the gateway 30 and interworking with the user terminal or a component corresponding to the means. That is, the API 130 may include at least some functions of the management server of the sensor network including the sensor of the device 10 and the gateway 30 or some components that perform functions corresponding to the functions.

The analysis server 150 may be implemented as a system (software aspect) for implementing a pet condition analysis system or a pet condition analysis method in hardware in the sense of agreement. In this case, the analysis server 150 may include an action array generator and a state inference unit.

In addition, analytics server 150 may have a real-time analytics infrastructure. The real time analytics infrastructure includes a real-time analytics engine and a data stream management system. The real-time analysis engine may include a behavior recognition model, a activity model, and the like.

In the above-described case, the analysis server 150 acquires the sensor data of the device 10 and classifies the features, or an action array generator including the feature classifier and an action of the companion animal based on the classified features. It may be provided with a behavior determination unit for determining the state or a state inference unit for identifying the mood or state of the pet based on the behavior determination unit. The components of the feature classifier, the action array generator, the action discriminator, and the state inference unit are stored in the memory in the form of a software module and may be implemented by a processor connected to the memory to perform a corresponding function. It is not limited.

In addition, the analysis server 150 may include at least some functions of the management server of the sensor network including the sensor of the device 10 and the gateway 30, or some components that perform functions corresponding to the functions. The analysis server 150 may be provided with a data stream management system.

For example, the analysis server 150 may convert sensor data into time series data as a management server and transmit the sensor data to a real time analysis engine. In that case, the real-time analysis engine may analyze the time series data based on the behavior recognition model and provide the result to the pet user or another user (such as a doctor) through the API 130. Here, the analysis result may refer to information or data in which the statuses of the pets are listed over time.

The data stream management system of the analysis server 150 may store time series data by matching them with a series of statuses of the analyzed pets based on the behavior recognition model.

In addition, the analysis server 150 may perform scenario-based computing. Scenario-based computing includes real-time computing including stream processing for abnormality alarms, health care alerts, and the like; Batch computing including background processing for generating data such as every hour, every three hours, every six hours, etc .; And on-demand computing, including client / server processing.

In real-time computing, the sensor data from the pet side is analyzed through the behavior recognition model, the analysis results are provided to the user terminal, and the data analyzed through the behavior recognition model are stored in the database, and the corresponding data and related information are stored in the user terminal. Service to search on. In this case, the user terminal (or an application mounted on the user terminal) can check in advance what the pet, which is the provider of the sensor data, is doing right now or in what state, and in advance so that the user can intuitively understand. The information may be searched, inquired or output in the form of a predetermined graphic, text, video, or a combination thereof.

In batch computing, the sensor data from the pet side is stored in a database, the sensor data stored in the database is analyzed through an emotion recognition model, matched with the sensor data and stored in the database, and the search / query of the application of the user terminal is Accordingly, the analysis result of the corresponding sensor data may be provided to the user terminal. In that case, the user can check what the pet alone at home today feels every hour or for a predetermined period of time.

In the on-demand computing, the sensor data from the pet side is stored in a database, and the sensor data is retrieved according to the execution request signal from the user terminal with respect to the sensor data for a predetermined time or for a certain period of time, and the sensor data retrieved from the activity amount model After analyzing through, the analysis result may be provided to the user terminal. In the above-described case, the user terminal may output an activity amount during the corresponding time or period as a calorie value, and the user may check the calorie value and determine whether or not his pet needs more exercise today.

The SNS server 170 may provide an online service, a community service, a social network service, etc. related to pet status analysis to a user terminal of another user such as a mobile terminal 50, a pet owner's user terminal, a related doctor, and the like. . The SNS server 170 may be installed to be divided into a leader area and a slave area for sharing information between users, or may be divided into a follower and a follower.

In addition, the SNS server 170 may interwork with an existing social network service or social networking service (SNS). Existing social network services include Facebook (Facebook.com), Twitter (Twitter.com), Kakaotalk (kakao.com), Band (band.us) and the like.

According to the above-described configuration, the owner of the pet or the user terminal of the other user accesses the database of the service providing system or the management server of the sensor network, and analyzes the time series data and the behavior of the pet based on the sensor data provided by the service providing system. As a result, the pet's criteria or status information can be obtained and used to care for the pet or communicate with the pet.

In addition, according to the above configuration, it is possible to provide a cloud-based pet buddy infrastructure having a connection environment, such as the Internet of things. In addition, the animal's behavior, emotions, and motor effects can be analyzed from a large amount of data generated by the IoT apparatus including the wearable device, and the analysis result can be used for various services. In addition, the analysis results may be expanded or compared to user data of the pet owner or expert data of the veterinarian. The aforementioned collected data, analysis data, user data, expert data, or a combination thereof may be referred to as big data.

3 is a block diagram of a pet condition analysis system according to an embodiment of the present invention. 4 is a block diagram of an action arrangement generator of the pet condition analysis system of FIG. 3. FIG. 5 is an exemplary diagram of sensor data that may be employed in the system of FIG. 3.

Referring to FIG. 3, the pet state analysis system according to the present exemplary embodiment may include an action arrangement generator 152 and a state inference unit 154. In this case, the pet state analysis system may correspond to the analysis server 150, and the action arrangement generator 152 and the state inference unit 154 may have a software module form or a program form. That is, the pet condition analysis system according to the present embodiment includes an action arrangement generator 152 and a state inference unit 154 and may be mounted and used in various types of control devices or computing devices. In that case, the control device or computing device may be a pet condition analysis system or at least a system that performs its function.

The action array generator 152 analyzes the behavior of the pet from the sensor data obtained through the receiver or the API and generates an action array in which time-series arrangement of the analyzed actions / actions is performed. The action arrangement may include information about the pet's successive actions with respect to the plurality of time series sensor data. Information about successive actions may be information or a collection of a series of actions over time rather than an uninterrupted time sequence.

As shown in FIG. 4, the sensor data may include three-axis acceleration data ACC_X, ACC_Y and ACC_Z and three-axis angular velocity data GYR_X, GYR_Y and GYR_Z. Triaxial acceleration data may be obtained from a wearable sensor worn on a pet. The wearable sensor may include an acceleration sensor and a gyro sensor. The gyro sensor may be a six-axis gyro sensor, but is not limited thereto.

In addition, the action array generator 152 selects, extracts, or determines the analysis result determined or labeled by the acceleration component and the angular velocity component respectively, which are represented at a predetermined window size for the sensor data on the time series. Can be. That is, in FIG. 4, each of the three-axis acceleration data and the three-axis angular velocity data in the unit window section including the specific time point t1 or the specific time point represents specific waveform patterns, and a combination thereof may be set to match the specific behavior of the pet. Can be.

For example, as shown in FIG. 5, when the pet is a dog, the standing state of the pet corresponds to the FFT feature shown in (a), and the behavior in which the pet walks is (b Corresponds to the FFT feature shown in), and the behavior in which the pet is running may correspond to the FFT feature shown in (c).

5 (b) may correspond to a case where an event of a certain level is a certain number of times (eg, one time or less) for a predetermined time, such as a portion indicated by a dotted line E1, as compared with FIG. 5 (a). And (c) of FIG. 5 may correspond to a case where a certain level of events for a predetermined time is a certain number of times (eg, two or more times), such as the portions indicated by the dotted lines E2 and E3. As shown in each graph, the reference level may be different from each other at a predetermined level or more in FIGS. 5B and 5C.

According to the above-described embodiment, the behaviors of pets may be classified into specific FFT features for each behavior or for a predetermined class. Here, the FFT feature may include FFT periodicity. Using these classification criteria, it is possible to distinguish between various states or dynamic behavior of the pet.

Referring back to FIG. 3, the state inference unit 154 infers the mood or state of the pet corresponding to the action arrangement generated by the action arrangement generator 152. The state inference unit 154 may infer the pet's mood or state using a preset action-state table (see 80 of FIG. 8).

The action-state table may include a preset pet mood or state as a column or column item, and a preset pet behavior as a column or column item.

Pet behaviors include lying, lie down, sitting, standing, shivering, shievering, barking, eating, drinking, and jumping. jumping, walking, fast walking, running, shaking, chewing, excreting, kicking or pawing, shaking off, smelling Sniffing, licking, panting, crouch, and the like.

The behavior of the pet can be selectively set for the behaviors applicable by the FFT application. In the present embodiment, basically, lying, sitting, standing, shaking (chilling, shivering), barking, eating, drinking, drinking, jumping, walking Walking, fast walking, running, and shaking are the basic behaviors, but are not limited to these.

The mood or condition of the pet may include pain, hunger, fear, loneliness, enjoyment, comfort, comfort, or a combination thereof. .

The above-described action-status table may be improved by comparing the data of the expert, such as a veterinarian or a zoologist, to determine the mood or condition of the actual pet.

According to the present embodiment, the mood or state of the pet may be analyzed quickly and accurately through the action-state table according to the action arrangement generated by the action arrangement generator 152.

FIG. 6 is a diagram illustrating a behavior classification process in the pet condition analysis system of FIG. 3.

Referring to FIG. 6, the action array generator 152 of the pet condition analysis system according to the present embodiment may include a frequency analyzer 1521, a fast fourier transform (FFT) application unit 1523, and a three-axis summation unit 1525. And an interval adding unit 1527.

The frequency analyzer 1521 analyzes frequency information included in data of the acceleration sensor and the angular velocity sensor. Frequencies within the frequency information may be divided into a plurality of sections according to preset pet conditions or behaviors.

The fast fourier transform (FFT) application unit 1523 may apply a fast Fourier transform to sensing values for each of three axes of the rectangular coordinate system. In addition, the FFT application unit 1523 may perform fast Fourier transform on the sensing values corresponding to each of the three axes of the rectangular coordinate system included in the acceleration and angular velocity sensing values of the sensor data.

The triaxial summation unit 1525 sums the triaxial acceleration data and the triaxial angular velocity data to which the fast Fourier transform is applied in the FFT application unit 1523, respectively. The section adding unit 1527 divides the three-axis acceleration data and the three-axis angular velocity data to which the fast Fourier transform is applied by the FFT application unit 1523 into a plurality of sections, and adds the FFT results. The three-axis adder 1525 and the section adder 1527 may be integrated into a single adder, and each adder may include an acceleration data adder and an angular velocity data adder.

For example, the single adder may extract four feature values by summing three-axis acceleration data divided into four sections, and extract another four feature values by summing three-axis angular velocity data divided by four sections. The section may be set to the main frequency band corresponding to the behavior of the pet.

For example, a lying, sitting or standing section (posture, called a posture section) corresponds to a frequency band of 0.8 kHz or less with 0 to 1 section, and a walking section with more than 0.8 kHz to 2 to 4 section 3.1 Corresponding to the frequency band below ㎐, the fast walking or drinking section corresponds to the frequency band below 6.4 kHz to more than 3.1 kHz in 5 to 8 intervals, and shaking or running The interval may be 9 to 13 intervals and may correspond to a frequency band of greater than 6.4 Hz and less than or equal to 10.1 Hz. The number of sections and the section size may vary depending on the class. The class may correspond to at least one specific behavior of the preset pet.

According to the segmentation of the above section, it is possible to improve the accuracy of the classification of the behavior of the pet. Furthermore, when the first segmentation of the main acts or movements pre-divided with respect to the pets' actions or behaviors and the second segmentation of the main acts or the transformations of the main acts or movements are applied, May drop slightly, but the overall reliability of behavioral analysis can be greatly improved. It does not analyze only the main behaviors or behaviors, but can effectively and relatively improve the reliability of analysis by blocking the missing analysis data to analyze the entire behavior or behavior of the pet. In this case, since the sensor data amount, data throughput, and data processing time increase, it is preferable to apply the sensor to a high speed data processing environment.

In the above-described embodiment, the prediction accuracy index of the behavior classification may be calculated by multiplying the accuracy of the first and second segments for a predetermined time and the percentage of tagged data in the sensor data. In this case, the prediction accuracy index for the behavior classification or behavior analysis of the pet may be provided to the gateway, the mobile terminal, or the user terminal through the SNS server or the app server.

7 to 9 are diagrams for explaining the pet condition analysis method that can be employed in the pet condition analysis system of FIG. 10 is a view for explaining an embodiment of the pet condition analysis method of FIGS.

The pet condition analysis method according to the present embodiment analyzes the pet's action in the sensor data and infers the mood or condition of the pet through the action-state matrix according to the analyzed action. Here, we create an action array with a certain number of actions to get the state from the successive pet actions.

By pointing to the action-to-state matrix and pointing to the action array, you can create state points. Since the generated state points represent the current pet's mood state, it can be used to infer the mood or state of the pet.

Specifically, as shown in FIG. 7, a frequency is analyzed in window sections w1 to w6 from sensor data input in time series to select an action corresponding to each window section. do. The time lengths p1 and p2 between the window sections may be the same or different from each other. Action 70 may include standing (S), walking (W), lying (L), and the like as the behavior of the pet.

Next, a pointing operation is performed with reference to the action-state matrix 80 shown in FIG. 8 for the action arrangement by the selected actions. The pointing operation may involve creating a state point most similar to the arrangement of FFT values in the action array in the direction or arrangement, such as horizontal, vertical, or diagonal.

In other words, as shown in FIG. 9, the walking behavior W of one time in the first behavior A1 section corresponding to the walking section and the second behavior A2 section corresponding to the lying section are shown. In a specific series of actions defined as a group of two lying times (L, L) and three times of standing actions (S, S, S) in the third section (A3) corresponding to the writing section. Based on the pet's mood can be inferred.

For example, as shown in FIG. 10, in the first step, an action array may be generated (S101). The action array Action [6] may be generated as a set of {walking, lying, lying, standing, standing, standing}.

Next, a state point may be created by performing a pointing operation on the action array with reference to the action-state matrix (see 80 of FIG. 8). The state point, StatePoint [6], may be generated as a set of {23, -2, -4, 10, 1, 14}.

Next, the highest state indicated by each feature value in the state point is inferred (S103). The highest state for the state point, MaxState (StatePoint []), may have a particular state index of "0". The status index is a numeric representation of your pet's mood or condition, where 0 is pain, 1 is hunger, 2 is fear, 3 is loneliness, and 4 is pleasure. And 5 may correspond to comfort, respectively.

According to the above series of processes, the pet condition analysis method according to the present embodiment can analyze the current mood or condition of the pet as "Pain" (S104). The result of the analysis may be delivered in real time to the user's mobile terminal according to the setting of the system or the setting of the menu / item in the user account may provide a notification service for the user to care for his pet.

According to the present embodiment, a pet's action and state can be easily and accurately inferred by utilizing various sensor technologies applied to a mobile device. In addition, the pet's mood is identified based on the inferred pet's behavior and condition, and the pet's owner is provided with such identification information or pet's needs prediction information to the pet's owner. This can improve the care and welfare of the pet. In addition, the pet's mood or condition can be provided to the pet's owner to facilitate communication between the pet and the person.

11 is a view for explaining a pet condition analysis method according to another embodiment of the present invention.

The pet condition analysis method according to the present embodiment may be implemented to further use sound data included in the sensor data. Sound data may be collected by a wearable sensor or a microphone worn on the pet (see reference numeral 7 of FIG. 2) or by another microphone installed around the pet.

When sound data is FFT-converted, the sound data reflecting a pet's behavior or state shows a certain pattern. For example, sound data corresponding to barking represents a frequency form as shown in FIG. 11 (a) or (b), and sound data corresponding to growling is shown in FIG. 11 (c). ) Or frequency form as shown in (d). Unique frequency forms of sound data also appear in howling, whining, etc. in addition to barking or roaring. In FIG. 11, each sound data corresponds to data extracted in a window size section of 0.5 second.

In this way, the sound data can be extracted at a window size of a predetermined section, and the mood or state of the pet can be identified based on the extracted sound data. In particular, in the present embodiment, the above-described sensor data and sound data may be combined to more accurately identify or analyze the behavior of the pet. For example, when inferring a pet's behavior from a window size (segment) including a specific time point of the three-axis acceleration data and the three-axis angular velocity data that are input or arranged in time series, it corresponds to the window size range including the specific time point. Based on the sound data, behavior inference in a particular section of the pet may be corrected, whether to apply weights to behavior inference, or the weight / level of weights for behavior inference may be adjusted.

The sound data described above may be mash up data in which a plurality of sound data are mixed. In this case, the analysis server may add a means for data conversion and normalization or a component for performing a function corresponding to the means. It may be provided as. In addition, the sound data may be obtained by analysis through an FFT, a mel-frequency cepstral coefficient (MFCC), or the like, or by automatic recognition through a sound card. Since the recognition criterion may vary depending on the window size or analysis time, the sound data may be extracted and used corresponding to the extraction / analysis time of the sensor data.

In addition, the aforementioned sound data may be recognized through a classifier that uses a k-neighbor algorithm (kNN or k-NN), which is one of pattern recognition methods, as a basic framework. The KNN classifier combines a feature extractor that extracts features from the window section of sound data, a learner that learns the data set (also called a training set) extracted from the feature extractor through KNN, and a KNN class recognized by the learner. The result set generator may be configured to generate a result set, and a probability recognizer configured to recognize a maximum probability of a specific mood or condition of the pet from the result set data generated by the result set generator. The sound data in the feature extractor may be divided through the MFCC. The maximum probability may correspond to the highest frequency class. Here, the class may correspond to the mood or state of the preset pet.

The above-described KNN classifier can greatly improve the accuracy of the final recognition of the pet's mood or condition by extracting the highest frequency class by adding the probability to the prediction class derived through KNN.

12 is a view for explaining a specific operating principle of the pet condition analysis system according to another embodiment of the present invention.

Referring to FIG. 12, the pet condition analysis system according to the present embodiment includes an analysis server 150, and the sensor data collected through the Bluetooth device 10a may be applied to a user terminal application. , UT App., 50a). The pet condition analysis system may further include an app server 190 connected to the analysis server 150 or may be linked with a separate app server 190. The Bluetooth device 10a may be a wearable device worn on a pet. The user terminal application 50a may be referred to simply as a user app.

The operation principle of the pet condition analysis system will be described in more detail. First, the Bluetooth device 10a may periodically wake up and wirelessly broadcast data collected (S121). The data sent for identification of the data may include an identifier such as a universally unique identifier (UUID).

Next, the user app 50a may recognize the Bluetooth device 10a through the identifier, connect to the Bluetooth device 10a according to a predetermined connection procedure, and request sensor data (S122). The sensor data may include three-axis acceleration data and three-axis angular velocity data, and may include sound data, depending on the implementation. The Bluetooth device 10a may provide sensor data to the user app 50a according to a request of the user app 50a (S123).

Next, the user app 50a may periodically transmit data to be analyzed (hereinafter, sensor raw data) to the analysis server 150 (S124). At this time, the sensor raw data has a significant difference in the amount of data depending on the data type and the transmission policy. Therefore, the analysis server 150 may have a predetermined data type and a transmission policy between the user app 50a and / or the analysis server 150. Of course, the data type and the transmission policy may be set in the user app 50a and / or the analysis server 150 in combination with the user interface to enable real-time adjustment.

Next, the analysis server 150 may periodically provide analysis completion pattern information to the app server 190 and the like (S125). The analysis completion pattern information may include information regarding the mood or the state of the pet, and the information may be provided to a user terminal or a social network service through the app server 190 or the like.

On the other hand, the user app 50a may be transferred to the app server 190 periodically or intermittently the data unnecessary for analysis (S126). Data that does not require analysis may include image data, sound data, and the like, and may partially overlap with data for analysis. Data for which such analysis is unnecessary may be relayed or provided to be directly used by the user terminal through the app server 190.

In addition, the user app 50a may request the pet's pattern data from the app server 190 (S127). In this case, the app server 50a may provide the pattern data of the pet to the user app 50a according to the user's request (S128).

According to the present embodiment, the analysis server 150 or the app server 190 may receive sensor data transmitted from the device through the relay of the user app 50a. In this case, the user app 50a may transmit data requiring analysis according to a preset criterion to the analysis server 150 and transmit data not requiring analysis to the app server 190. In addition, the user app 50a may request and acquire the final pattern data representing the mood or state of the pet from the app server 190 and display the result on the screen of the user terminal or output the user terminal.

According to this embodiment described above, the pet state analysis system acquires sensor data of a mobile device located adjacent to the pet in association with the user terminal, and processes a part of the sensor data to determine the mood or state of the pet. The analysis result may be provided to the user terminal.

Although described above with reference to a preferred embodiment of the present invention, those skilled in the art will be variously modified and changed within the scope of the invention without departing from the spirit and scope of the invention described in the claims below I can understand that you can.

Claims (18)

Receiving sensor data generated from a plurality of sensors including a wearable sensor worn on the pet, wherein the sensor data includes 3-axis acceleration data and 3-axis angular velocity data;
Analyzing the pet's behavior in the sensor data to generate an action arrangement, the action arrangement including information about the pet's successive behaviors for a plurality of time series sensor data; And
Inferring the mood or condition of the pet corresponding to the action arrangement;
At least one or more angular velocity data of the three-axis angular velocity data is a predetermined weight is applied to at least some of the predetermined behaviors, pet state analysis method. The method according to claim 1,
The behavior of the pet is lying, sitting, standing, shaking (chilling, shivering), barking, eating, drinking, jumping, jumping, walking A method for analyzing a pet's condition, including walking, fast walking, running, and shaking. The method according to claim 2,
The mood or condition of the pet includes pain, hunger, fear, loneliness, pleasure and comfort. The method according to claim 3,
The inferring is performed by referring to a preset action-state table.
And the action-state table includes the pet's mood or condition and the pet's behavior as rows and columns. delete The method according to claim 1,
The sensor data further includes sound data,
The sound data includes a segmentation on a time series including a time series viewpoint for the three-axis acceleration data or the three-axis angular velocity data. The method according to claim 6,
And after generating the action arrangement, correcting the behavior arrangement of the pet based on the three-axis acceleration data and the three-axis angular velocity data based on the sound data. The method according to claim 1,
In the step of generating the action arrangement, the behavior analysis of the pet is determined by the acceleration component and the angular velocity component respectively appearing at a predetermined window size for the sensor data on the time series. The method according to claim 1,
Generating the action array,
Extracting feature data listed in time series from the sensor data;
Extracting a series of action queue data for a preset action corresponding to an action window section from the feature data; And
Generating a behavior queue corresponding to each of the series of action queue data,
And inferring the mood or condition of the pet in accordance with the action queue. The method according to claim 9,
The extracting of the series of action queue data may include a portion where action window sections of adjacent action queue data overlap each other. The method according to claim 9,
The period of the sensor data is several tens of hertz, and the action window section is set according to a generation order of a preset sensing event regardless of timing. A receiver receiving sensor data generated from a plurality of sensors including a wearable sensor worn on a pet, wherein the sensor data includes 3-axis acceleration data and 3-axis angular velocity data;
An action array generator for analyzing an action of the pet in the sensor data to generate an action array, wherein the action array includes information about the continuous actions of the pet with respect to a plurality of time series sensor data; And
It includes a state inference unit for inferring the mood or state of the pet corresponding to the action arrangement,
The action array generator,
A frequency analyzer for analyzing frequency information included in data of the three-axis acceleration sensor and the three-axis angular velocity sensor;
A fast fourier trasform (FFT) application unit for applying a fast Fourier transform to the sensing values from which the frequency information is analyzed; And
And a summation unit for summing three-axis acceleration data and three-axis angular velocity data to which the fast Fourier transform is applied, respectively. delete The method according to claim 12,
The behavior of the pet is extracted by dividing the periodic characteristics of the FFT in the FFT application unit, pet condition analysis system. The method according to claim 12,
The pet state analysis system further comprises an application programming interface (API) having the receiving unit. The method according to claim 15,
It is disposed between the API and the pet condition analysis system further comprises a system environment or a central controller to control the interworking with the API, the signal and data transmission and reception. The method according to claim 16,
And a social network service (SNS) server connected to the API and sharing the analysis result of the state inference unit. The method according to claim 17,
The app server connected to the system environment, the central controller, the SNS server, or any one thereof may provide the analysis result to a predetermined or related user terminal or an application of the user terminal, or to request a data request from the user terminal. And providing the pet-related pattern data to the user terminal.

Images