KR20230086541A - Apparatus and method for animal speech analysis using artificial intelligence - Google Patents

Abstract

Disclosed are a device and method for analyzing an animal speech using artificial intelligence. The device for analyzing the animal speech according to one embodiment comprises: a camera part comprising a camera and one or more motors to drive the camera, and generating image data through the camera; a speech detection part that generates speech data through a plurality of microphones; and a control part that controls the camera part and the speech detection part, wherein the control part may comprise a first artificial neural network learned to analyze a type and age of an animal based on the image data, and a second artificial neural network learned to analyze a state of the animal based on speech data.

Description

Apparatus and method for animal speech analysis using artificial intelligence}

As a technology for analyzing animal voices, the present invention relates to an apparatus and method for analyzing animal voices using artificial intelligence, in particular.

Recently, voice recognition technology using artificial intelligence has been actively researched, and furthermore, studies are being actively conducted to detect emotions and states of animals by analyzing voices of non-human animals.

However, the most important part in analyzing the state of an animal is to obtain learning data in which emotions and states are classified by acquiring voice data for each of animals of various types and ages. However, since animals cannot be forcibly exposed to dangerous situations in order to acquire voice data for a specific state, it is difficult to acquire learning data for this.

In the case of Korean Patent Publication No. 10-2019-0126552, a method for analyzing an animal's voice is disclosed, but when there is no animal's voice data for a specific emotional state, a specific method for supplementing this is not presented.

An object of the present invention is to provide a device and method for analyzing animal voices using artificial intelligence.

According to one aspect, an animal voice analysis apparatus includes a camera and one or more motors for driving the camera, and includes a camera unit generating image data through the camera; a voice detector generating voice data through a plurality of microphones; and a control unit for controlling a camera unit and a voice detection unit, wherein the control unit has a first artificial neural network learned to analyze the type and age of an animal based on image data and a first artificial neural network learned to analyze a state of an animal based on voice data. 2 may include artificial neural networks.

The control unit analyzes the location where the corresponding voice occurred when a voice of a predetermined level or higher is detected based on the voice data received from the voice detector, and controls one or more driving motors so that the camera can capture the location based on the analyzed location. can do.

When a voice of a predetermined level or higher is detected based on the voice data received from the voice detector, the control unit analyzes the location where the corresponding voice is generated, and generates a beamforming vector to be applied to a plurality of microphones based on the analyzed location. can

The controller may input information about the type and age of the animal analyzed through the first artificial neural network into the second artificial neural network.

The control unit may include a third artificial neural network that generates synthetic learning data for combinations of animal types, ages, and states not included in the basic training data based on the basic training data labeled with the animal types, ages, and states. there is.

The control unit groups basic learning data based on animal type, age, and condition to generate one or more voice data groups, and voice characteristic common data for each voice data group based on voice data included in one or more voice data groups. can create

The control unit detects a missing combination that is not included in the voice data group among combinations classified based on animal type, age, and state, and has a state different from the state of the missing combination among combinations for the type of animal corresponding to the missing combination. A combination is extracted, and a combination having common voice feature data for the age corresponding to the missing combination is extracted from among the extracted combinations to generate a first reference combination, and the same age and condition as the age and condition of the animal corresponding to the missing combination The type of another animal having common voice feature data corresponding to the state is detected, and a combination having common voice feature data corresponding to the first reference combination is extracted from among the common voice feature data corresponding to the type of the detected animal to obtain a second A reference combination may be generated, and voice difference characteristic data may be generated based on common voice feature data included in the first reference combination and common voice feature data included in the second reference combination.

The third artificial neural network receives common voice feature data and voice difference characteristic data corresponding to the same age and state as the age and state of the animal corresponding to the missing combination among the common voice feature data corresponding to the type of detected animal, and performs basic learning Synthetic learning data can be created for combinations of animal types, ages, and conditions not included in the data.

The second artificial neural network may be trained based on basic training data and synthetic training data labeled with the type, age, and state of the animal.

If the combination of the analyzed voice data is not included in the basic training data, the control unit includes the analyzed voice data in the basic training data and deletes synthetic training data corresponding to the combination of the analyzed voice data to set the training data. can be updated.

The controller may re-learn the second artificial neural network based on the updated training data set.

According to one aspect, an animal voice analysis method includes generating image data through a camera driven by one or more motors; generating voice data through a plurality of microphones; Analyzing the type and age of the animal on the image data using a first artificial neural network trained to analyze the type and age of the animal based on the image data; and analyzing a state of the animal corresponding to the voice data using a second artificial neural network trained to analyze the state of the animal based on the voice data.

The animal voice analysis method includes the steps of analyzing a location where a corresponding voice occurred when a voice of a predetermined level or higher is detected based on voice data, and controlling one or more driving motors so that a camera can capture the location based on the analyzed location. can include more.

The animal voice analysis method further includes the steps of analyzing a location where the voice is generated when a voice of a predetermined level or higher is detected based on voice data, and generating a beamforming vector to be applied to a plurality of microphones based on the analyzed location. can include

The second artificial neural network may receive information about the type and age of the animal analyzed through the first artificial neural network and analyze the state of the animal.

The animal voice analysis method uses a third artificial neural network to generate synthetic learning data for combinations of animal types, ages, and states that are not included in the basic training data based on basic learning data labeled with animal types, ages, and states. A generating step may be further included.

In the step of generating synthetic learning data, one or more voice data groups are generated by grouping the basic training data based on the type, age, and state of the animal, and each voice data is generated based on the voice data included in the one or more voice data groups. Voice feature common data for each group may be generated.

The step of generating synthetic learning data detects missing combinations that are not included in the voice data group among combinations classified based on animal type, age, and state, and selects missing combinations among combinations for the type of animal corresponding to the missing combination. A combination having a different state from the state is extracted, and a combination having common voice feature data for an age corresponding to the missing combination is extracted from among the extracted combinations to generate a first reference combination, and an animal age corresponding to the missing combination is extracted. and detecting a type of another animal having common voice feature data corresponding to the same age and state as the state, and having common voice feature data corresponding to the first reference combination among common voice feature data corresponding to the type of the detected animal. The combination may be extracted to generate a second reference combination, and voice difference characteristic data may be generated based on common voice feature data included in the first reference combination and common voice feature data included in the second reference combination.

The third artificial neural network receives common voice feature data and voice difference characteristic data corresponding to the same age and state as the age and state of the animal corresponding to the missing combination among the common voice feature data corresponding to the type of detected animal, and performs basic learning Synthetic learning data can be created for combinations of animal types, ages, and conditions not included in the data.

The second artificial neural network may be trained based on basic training data and synthetic training data labeled with the type, age, and state of the animal.

The animal voice analysis method includes the analyzed voice data in the basic training data when the combination of the analyzed voice data is not included in the basic training data, and deletes the synthetic training data corresponding to the analyzed voice data combination. Updating the training data set may be further included.

In the updating of the training data set, the second artificial neural network may be retrained based on the updated training data set.

Artificial intelligence can be used to accurately analyze animal emotions and health conditions. In addition, the analysis performance of artificial intelligence can be improved by using synthesized voice data generated based on animal voice data.

1 is a configuration diagram of an animal voice analysis device according to an exemplary embodiment.
2 and 3 are exemplary diagrams for explaining an operation of an apparatus for analyzing animal voices according to an exemplary embodiment.
4 is a flowchart illustrating a method for analyzing animal voices according to an exemplary embodiment.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In describing the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, the definition should be made based on the contents throughout this specification.

Hereinafter, embodiments of an apparatus and method for analyzing animal voice will be described in detail with reference to drawings.

1 is a configuration diagram of an animal voice analysis device according to an exemplary embodiment.

According to an embodiment, the animal voice analysis device 100 includes a camera and one or more motors for driving the camera, and the camera unit 110 generates image data through the camera and voice data through a plurality of microphones. It may include a voice detecting unit 120 that generates voice and a controller 130 that controls the camera unit 110 and the voice detecting unit 120 .

Referring to FIG. 2 , the camera unit 110 may generate image data by photographing animals in a predetermined direction, and the voice detector 120 may generate audio data of animals using a plurality of microphones.

According to an embodiment, the controller 130 includes a first artificial neural network trained to analyze the type and age of an animal based on image data and a second artificial neural network trained to analyze a state of an animal based on voice data. can do.

According to an example, the first artificial neural network may be an object recognition model, and may be trained to detect an animal on image data and analyze the type and age of the detected animal. For example, the type of animal may be classified into canines and felines, or classified based on breed within the same canine family such as Chihuahua and Poodle. For example, age may be divided into two groups, such as adult dog/adult cat (O) and baby dog/baby cat (Y).

According to an embodiment, the controller 130 may analyze a location where the corresponding voice occurred when a voice of a predetermined level or higher is detected based on the voice data received from the voice detector 120 .

For example, the animal's voice may have different levels depending on whether a specific event has occurred. For example, animals may emit loud vocalizations when unusual events such as threat, hunger, or pain occur. Accordingly, the controller 130 may filter out a case in which a voice of a predetermined level or higher is detected.

For example, the controller 130 may calculate the location where the voice was generated using a time difference between voices input through a plurality of microphones.

According to an embodiment, the controller 130 may control one or more driving motors so that the camera can capture the location based on the analyzed location. For example, when an animal emits a sound of a predetermined level or higher, the controller 130 may determine the location of the animal and control a motor for driving the camera so that the camera can take pictures centering on the location. For example, as shown in FIG. 2 , a camera may capture a specific location or a specific angle.

According to an embodiment, the controller 130 may generate beamforming vectors to be applied to a plurality of microphones based on the analyzed positions. For example, when an animal emits a sound of a predetermined level or higher, the controller 130 may determine the location of the animal and generate a beamforming vector for intensively acquiring the sound of the corresponding location by a plurality of microphones. For example, as shown in FIG. 2 , the microphone may form beamforming to acquire sound in a predetermined direction.

According to an embodiment, the controller 130 may input information about the type and age of animals analyzed through the first artificial neural network to the second artificial neural network. For example, the controller 130 may determine the type and age of animals included in the image data through the first artificial neural network. For example, the animal can be a Chihuahua/puppy. Thereafter, the controller 130 may input information analyzed as Chihuahua/baby dog to the second artificial neural network. In this case, the second artificial neural network may analyze the state of the animal by analyzing the animal's voice data based on the received Chihuahua/baby dog information.

According to an embodiment, the controller 130 generates synthetic learning data for combinations of animal types, ages, and states that are not included in the basic training data based on basic training data labeled with animal types, ages, and states. It may include a third artificial neural network that does.

Referring to FIG. 3 , learning data may be classified based on animal type, age, and condition. In this case, learning data obtained from an actual animal and labeled may be used as basic learning data. However, it may be difficult to obtain voice data for a specific state, such as a state in which pain is felt, or voice data of an animal of a low age. Accordingly, basic learning data of a specific combination may not exist in a combination classified by animal type-age-state. For example, basic learning data corresponding to a combination of A-O-3 and C-O-1 among type-age-state combinations may not exist.

In this case, the third artificial neural network may generate synthetic training data to supplement non-existent basic training data using basic synthetic data under a specific condition.

According to an embodiment, the controller 130 may generate one or more voice data groups by grouping basic learning data based on the animal's type, age, and condition.

For example, basic learning data may be grouped on the basis of type-age-state, and each cell marked with 'O' and 'X' in FIG. 3 may represent a group.

According to an embodiment, the controller 130 may generate voice feature common data for each voice data group based on voice data included in one or more voice data groups. For example, the controller 130 may extract voice features by analyzing voice data corresponding to each cell in FIG. 3 for each group. For example, features such as frequency, interval, and spectral density of voice may be extracted, and voice feature common data may be generated based on the feature.

According to an embodiment, the controller 130 may detect missing combinations that are not included in the voice data group among combinations classified based on animal types, ages, and states. For example, the controller 130 may detect the combination 'A-O-3' corresponding to the cell indicated by B1 in FIG. 3 as a missing combination.

According to an embodiment, the control unit 130 extracts a combination having a state different from the state of the missing combination among animal types corresponding to the missing combination, and determines the age corresponding to the missing combination among the extracted combinations. A first reference combination may be generated by extracting a combination having common voice feature data.

For example, if the combination 'A-O-3' is a missing combination, the type of animal corresponding to the combination becomes A, and the combinations having a state different from the state of the missing combination are 'A-O-1' and 'A-Y-1'. ', 'A-O-2', 'A-Y-2'. Thereafter, the controller 130 may extract 'A-O-1' and 'A-O-2', which are combinations having common voice feature data for age corresponding to the missing combination among the extracted combinations, and use this combination as a first criterion. It can be created with combination (B2).

According to an embodiment, the controller 130 may detect the type of another animal having common voice feature data corresponding to the same age and state as the age and state of the animal corresponding to the missing combination. For example, the controller 130 may detect a type having common voice characteristic data among combinations of '*-O-3'. Referring to FIG. 3 , corresponding types may be 'B' and 'C'.

Thereafter, the controller 130 may generate a second reference combination by extracting a combination having common voice feature data corresponding to the first reference combination among common voice feature data corresponding to the type of the detected animal. For example, as shown in FIG. 3 , the controller 130 may detect 'B-O-1', 'B-O-2', and 'C-O-2' to generate the second reference combination B4.

According to an embodiment, the controller 130 may generate voice difference feature data based on common voice feature data included in the first reference combination and common voice feature data included in the second reference combination. For example, the controller 130 may generate voice difference feature data by comparing common voice feature data corresponding to B2 and common voice feature data corresponding to B3. For example, the controller 130 may generate voice difference characteristic data by comparing 'A-O-1' with 'B-O-1'.

According to an embodiment, the third artificial neural network includes common voice feature data and voice difference characteristics corresponding to the same age and state as the age and state of the animal corresponding to the missing combination among the common voice feature data corresponding to the type of the detected animal. Data can be input and synthetic learning data can be created for combinations of animal types, ages, and conditions that are not included in the basic learning data.

For example, the third artificial neural network corresponds to voice feature common data B4 corresponding to the same age and state as the age and state of the animal corresponding to the missing combination among the voice feature common data corresponding to the detected animal type. B-O-3' can be selected. Then, synthetic learning data for 'A-O-3' corresponding to the missing combination B1 is generated by applying the voice difference characteristic data generated using 'B-O-1' and 'A-O-1' to 'B-O-3' can do.

According to an embodiment, the second artificial neural network may be trained based on basic training data and synthetic training data labeled with the type, age, and state of the animal. In other words, the second artificial neural network may be trained using both learning data generated based on voice data obtained from a real animal and learning data generated through the third artificial neural network.

According to an embodiment, if the combination of the analyzed voice data is a combination not included in the basic training data, the controller 130 includes the analyzed voice data in the basic training data, and provides a response corresponding to the combination of the analyzed voice data. You can update the training data set by deleting the synthetic training data.

For example, voice data analyzed by the controller 130 may be analyzed as type A/age O/state 3. In other words, the voice data analyzed by the controller 130 may be data corresponding to the missing combination corresponding to combination B1.

In this case, the controller 130 may determine the analyzed voice data as the basic learning data of the 'A-O-3' combination. In addition, the control unit 130 may delete synthesized learning data previously generated through the third artificial neural network and replace the corresponding data with analyzed voice data.

According to an embodiment, the controller 130 may relearn the second artificial neural network based on the updated training data set. That is, the controller 130 can improve the performance of the second artificial neural network by re-learning the second artificial neural network using voice data, which is more accurate data than synthesized learning data.

4 is a flowchart illustrating a method for analyzing animal voices according to an exemplary embodiment.

According to an embodiment, the animal voice analysis apparatus may generate image data through a camera driven by one or more motors (410), and may generate voice data through a plurality of microphones (420).

According to an embodiment, the animal voice analysis apparatus may analyze the type and age of an animal on the image data by using a first artificial neural network trained to analyze the type and age of an animal based on the image data (430). According to an example, the first artificial neural network may be an object recognition model, and may be trained to detect an animal on image data and analyze the type and age of the detected animal.

According to an embodiment, the animal voice analysis apparatus may analyze the state of the animal corresponding to the voice data by using the second artificial neural network learned to analyze the state of the animal based on the voice data (440).

According to an embodiment, the animal voice analysis apparatus may input information about the type and age of the animal analyzed through the first artificial neural network to the second artificial neural network. For example, the animal voice analysis apparatus may determine the type and age of animals included in the image data through the first artificial neural network. For example, the animal can be a Chihuahua/puppy. Thereafter, the animal voice analysis apparatus may input information analyzed as Chihuahua/baby dog to the second artificial neural network. In this case, the second artificial neural network may analyze the state of the animal by analyzing the animal's voice data based on the received Chihuahua/baby dog information.

According to an embodiment, the apparatus for analyzing animal voices generates synthetic learning data for combinations of animal types, ages, and states that are not included in the basic training data based on basic training data labeled with animal types, ages, and states. It may include a third artificial neural network that does.

Among the embodiments of FIG. 4 , matters overlapping those described with reference to FIGS. 1 to 3 are omitted. Accordingly, the embodiment described with reference to FIGS. 1 to 3 can be equally applied to the animal voice analysis method described with reference to FIG. 4 .

An aspect of the present invention may be implemented as computer readable code on a computer readable recording medium. Codes and code segments implementing the above program can be easily inferred by a computer programmer in the art. A computer-readable recording medium may include all types of recording devices storing data that can be read by a computer system. Examples of computer-readable recording media may include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, and the like. In addition, the computer-readable recording medium may be distributed among computer systems connected through a network, and may be written and executed as computer-readable codes in a distributed manner.

So far, the present invention has been looked at mainly with its preferred embodiments. Those skilled in the art to which the present invention pertains will be able to understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the scope of the present invention should be construed to include various embodiments within the scope equivalent to those described in the claims without being limited to the above-described embodiments.

100: animal voice analysis device
110: camera unit
120: voice detector
130: control unit

Claims (22)

a camera unit including a camera and one or more motors for driving the camera and generating image data through the camera;
a voice detector generating voice data through a plurality of microphones; and
A control unit for controlling the camera unit and the voice detection unit,
The control unit
A first artificial neural network trained to analyze the type and age of animals based on the image data; and
An animal voice analysis apparatus comprising a second artificial neural network learned to analyze a state of an animal based on the voice data.
According to claim 1,
The control unit
When a voice of a predetermined level or higher is detected based on the voice data received from the voice detector, a location where the corresponding voice is generated is analyzed;
Animal voice analysis apparatus for controlling the one or more drive motors so that the camera can capture the location based on the analyzed location.
According to claim 1,
The control unit
When a voice of a predetermined level or higher is detected based on the voice data received from the voice detector, a location where the corresponding voice is generated is analyzed;
Animal voice analysis apparatus for generating a beamforming vector to be applied to the plurality of microphones based on the analyzed position.
According to claim 1,
The control unit
An animal voice analysis apparatus for inputting information on the type and age of an animal analyzed through the first artificial neural network to the second artificial neural network.
According to claim 1,
The control unit
Based on the basic training data labeled with the type, age and condition of the animal, a third artificial neural network for generating synthetic learning data for combinations of the type, age and condition of animals not included in the basic training data, animal voice analysis device.
According to claim 5,
The control unit
Generate one or more voice data groups by grouping the basic training data based on animal type, age, and condition;
An animal voice analysis apparatus for generating common voice feature data for each voice data group based on voice data included in the one or more voice data groups.
According to claim 6,
The control unit
Detect missing combinations that are not included in the negative data group among combinations classified based on animal type, age, and condition,
A combination having a state different from that of the missing combination is extracted from among combinations of types of animals corresponding to the missing combination, and a combination having common voice feature data for an age corresponding to the missing combination among the extracted combinations. Extracting to generate a first reference combination,
Detecting the type of another animal having common voice feature data corresponding to the same age and state as the age and state of the animal corresponding to the missing combination, and among the common voice feature data corresponding to the type of the detected animal, the first generating a second reference combination by extracting a combination having voice feature common data corresponding to the reference combination;
and generating voice difference feature data based on common voice feature data included in the first reference combination and common voice feature data included in the second reference combination.
According to claim 7,
The third artificial neural network
Among the common voice feature data corresponding to the type of the detected animal, common voice feature data corresponding to the same age and state as the age and state of the animal corresponding to the missing combination and the voice difference feature data are received as input, and the basic learning data An animal voice analysis device that generates synthetic learning data for combinations of types, ages, and states of animals that are not included in the
According to claim 5,
The second artificial neural network
An animal speech analysis device trained on the basis of basic training data and synthetic training data labeled with the type, age, and condition of the animal.
According to claim 5,
The control unit
When the combination of the analyzed voice data is a combination not included in the basic training data, the analyzed voice data is included in the basic training data, and the synthesized training data corresponding to the analyzed voice data combination is deleted for learning. An animal voice analysis device that updates a data set.
According to claim 10,
The control unit
Animal voice analysis apparatus for re-learning the second artificial neural network based on the updated learning data set.
generating image data through a camera driven by one or more motors;
generating voice data through a plurality of microphones;
analyzing the type and age of the animal on the image data using a first artificial neural network trained to analyze the type and age of the animal based on the image data; and
and analyzing a state of the animal corresponding to the voice data using a second artificial neural network learned to analyze the state of the animal based on the voice data.
According to claim 12,
When a voice of a predetermined level or higher is detected based on the voice data, analyzing a location where the corresponding voice is generated, and controlling the one or more driving motors so that the camera can capture the location based on the analyzed location. Further comprising, animal voice analysis method.
According to claim 12,
When a voice of a predetermined size or more is detected based on the voice data, analyzing a location where the corresponding voice is generated, and generating a beamforming vector to be applied to the plurality of microphones based on the analyzed location Further comprising , animal voice analysis method.
According to claim 12,
The method of analyzing animal voice, wherein the second artificial neural network receives information on the type and age of the animal analyzed through the first artificial neural network and analyzes the state of the animal.
According to claim 12,
Generating synthetic learning data for combinations of animal types, ages, and states not included in the basic learning data based on basic training data labeled with animal types, ages, and states using a third artificial neural network. Further comprising, animal voice analysis method.
17. The method of claim 16,
The step of generating the synthetic learning data is
Generate one or more voice data groups by grouping the basic training data based on animal type, age, and condition;
The animal voice analysis method of generating common voice feature data for each voice data group based on voice data included in the one or more voice data groups.
18. The method of claim 17,
The step of generating the synthetic learning data is
Detect missing combinations that are not included in the negative data group among combinations classified based on animal type, age, and condition,
A combination having a state different from that of the missing combination is extracted from among combinations of types of animals corresponding to the missing combination, and a combination having common voice feature data for an age corresponding to the missing combination among the extracted combinations. Extracting to generate a first reference combination,
Detecting the type of another animal having common voice feature data corresponding to the same age and state as the age and state of the animal corresponding to the missing combination, and among the common voice feature data corresponding to the type of the detected animal, the first generating a second reference combination by extracting a combination having voice feature common data corresponding to the reference combination;
and generating voice difference feature data based on common voice feature data included in the first reference combination and common voice feature data included in the second reference combination.
According to claim 18,
The third artificial neural network
Among the common voice feature data corresponding to the type of the detected animal, common voice feature data corresponding to the same age and state as the age and state of the animal corresponding to the missing combination and the voice difference feature data are received as input, and the basic learning data An animal voice analysis method that generates synthetic learning data for a combination of animal types, ages, and conditions not included in.
17. The method of claim 16,
The second artificial neural network
A method for analyzing animal voices, trained on the basis of primary training data and synthetic training data labeled by animal type, age, and condition.
17. The method of claim 16,
When the combination of the analyzed voice data is a combination not included in the basic training data, the analyzed voice data is included in the basic training data, and the synthesized training data corresponding to the analyzed voice data combination is deleted for learning. A method for analyzing animal speech, further comprising updating the data set.
According to claim 21,
Updating the training data set
The animal voice analysis method of re-learning the second artificial neural network based on the updated learning data set.

Images