KR102128612B1 - A method and system for enhancing the accuracy of circumstantial judgment using audiovisual detection - Google Patents

Abstract

The present invention relates to a method and a system for improving the accuracy of situation determination through audio-visual detection, and a method for improving the accuracy of situation determination through audio-visual detection according to an embodiment of the present invention includes a situation determination result through an audio-visual detection unit Generating a Bayesian network with a root node as a root node, and estimating a priori probability of the root node based on statistical information learned from context data collected through the audio-visual sensor. , Referring to the conditional probability of at least one or more child nodes for the root node based on the information input from the user and the domain region information, and the posterior probability of the a priori probability of the root node based on the referenced conditional probability (aposteriori probaility).

Description

A METHOD AND SYSTEM FOR ENHANCING THE ACCURACY OF CIRCUMSTANTIAL JUDGMENT USING AUDIOVISUAL DETECTION}

The present invention relates to a method and a system for improving the accuracy of situational judgment through audiovisual detection, and more specifically, by using a Bayesian network, combining audiovisual detection results and domain expert knowledge to reduce misrecognition and improve situational accuracy It relates to a method and system.

Recently, the development of machine learning methods such as deep learning is increasing the recognition performance of video and audio signals to the level of human vision and hearing. However, even if these methods alone guarantee a cognitive performance comparable to that of a person, there is a limit to reducing misperception by only a single vision or modality of hearing.

For example, in the case of forest fire monitoring, if it is judged that it is a fire by analyzing only the video of CCTV, it is inevitable to misjudge the smoke, factory smoke, and fog of heating chimneys of houses. Therefore, in order to reduce the misjudgment, it is necessary to make a final judgment by observing other variables necessary for decision making, in addition to vision or hearing, in order to make accurate decisions like humans.

However, the variables necessary for such decision making and the causal relationship in which these variables contribute to decision making are often difficult and difficult to obtain. That is, there is a limit in constructing a data driven intelligent system through learning by accumulating a lot of data corresponding to these variables, and it is inevitable to collect and utilize uncertain knowledge of the domain expert.

Therefore, it is necessary to reduce the misrecognition and improve the accuracy of recognition by combining the qualitative causal relationship between environmental variables that do not have much data and results recognized through audio-visual stimulation.

Republic of Korea Registered Patent Publication No. 10-1675692 (2012.09.12)

The present invention is intended to solve the above-described problems, and when it is not easy to secure data, the accuracy of situation determination through audio-visual detection is improved by using domain expert knowledge and environment variables corresponding to information in the area. It is an object to provide a method and a system that can be performed.

In addition, an object of the present invention is to implement a total artificial intelligence system that can easily combine a result of a judgment on a situation perceived using a Bayesian network and its cause and provide a more accurate recognition result.

A method of improving the accuracy of situation determination through audio-visual detection according to an embodiment of the present invention includes: generating a Bayesian network using a context determination result through the audio-visual detection unit as a root node; Estimating a priori probability of the root node based on statistical information learned from context data collected through the sensing unit, at least one or more child nodes for the root node from information input from the user and information in the domain region It may include a step of referring to the conditional probability of (child node) and an inference step of updating the a priori probability of the root node to an aposteriori probability based on the referenced conditional probability.

The child node according to an embodiment of the present invention is formed based on the cause of the situation determination result through the audio-visual detection unit, and the cause for the situation determination result through the audio-visual detection is derived from information input from the user and domain domain information. This may be possible.

Based on the referenced conditional probability according to an embodiment of the present invention, the preliminary step of updating the a priori probability of the root node to a posterior probability is a cause situation corresponding to any one of the at least one child node When this occurs, further comprising the step of detecting the cause situation, and updating the a priori probability of the root node with a posterior probability through the conditional probability of the child node corresponding to the detected cause situation to adjust the degree of belief in the root node (belief). This may be possible.

A system for improving the accuracy of situation determination through audio-visual sensing according to an embodiment of the present invention collects context data through at least one of visual or auditory sensing, and performs machine learning based on the collected context data. An audiovisual detection unit that learns statistical information and judges the situation, a domain data collection unit that collects information input from a user and information of a domain area, and a situational determination that generates a Bayesian network using a context determination result through the audiovisual detection unit as a root node Estimating a priori probability of the root node based on statistical information learned through the network generation unit and the audio-visual detection unit, and referring to the conditional probability of at least one or more child nodes for the root node from information input from the user and information in the domain region And a context determination network inference unit for updating the a priori probability of the root node with a posterior probability based on the probability reference unit and the estimated conditional probability.

The child node according to an embodiment of the present invention is formed based on the cause of the situation determination result through the audio-visual detection unit, and the cause for the situation determination result through the audio-visual detection is derived from information input from the user and domain domain information. This may be possible.

When a cause situation corresponding to any one of the at least one child node according to an embodiment of the present invention occurs, further comprising a cause situation detection unit that detects the cause situation, and corresponds to the cause situation detected by the situation determination network reasoning unit Through the conditional probability of the child node, the a priori probability of the root node may be updated with a posterior probability to control the degree of belief in the root node.

On the other hand, according to an embodiment of the present invention, it is possible to provide a recording medium readable by a computer recording a program for executing the above-described method on a computer.

According to a method and system for improving the accuracy of situation determination through audio-visual detection provided as an embodiment of the present invention, when it is impossible to secure the accuracy of situation determination using only audio-visual data, it corresponds to knowledge of a domain expert and information of the area By using environment variables, etc., it is possible to improve the accuracy of situation determination through the audio-visual detection system, and also improve the reliability of the situation.

In addition, according to a method and system for improving the accuracy of situation determination through audio-visual sensing provided as an embodiment of the present invention, it is possible to easily combine the result of the determination with respect to the situation recognized using the Bayesian network and its causes The performance of a conventional audio-visual sensing/recognition system can be further improved.

1 is a flowchart illustrating an example of a method of improving the accuracy of situation determination through audio-visual sensing according to an embodiment of the present invention.
2 is a flowchart illustrating another example of a method of improving the accuracy of situation determination through audio-visual sensing according to an embodiment of the present invention.
3 is a block diagram showing an example of a Bayesian network using a determination result for a fire detection situation according to an embodiment of the present invention as a root node.
4 is a table showing an example of conditional probability of child nodes with respect to a determination result of a fire detection situation according to an embodiment of the present invention.
5 is a block diagram showing an example of a system for improving the accuracy of situation determination through audio-visual sensing according to an embodiment of the present invention.

Terms used in the present specification will be briefly described, and the present invention will be described in detail.

The terminology used in the present invention has been selected, while considering the functions in the present invention, general terms that are currently widely used have been selected, but this may vary according to the intention or precedent of a person skilled in the art or the appearance of new technologies. In addition, in certain cases, some terms are arbitrarily selected by the applicant, and in this case, their meanings will be described in detail in the description of the applicable invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the entire contents of the present invention, not a simple term name.

When a certain part of the specification "includes" a certain component, this means that other components may be further included rather than excluding other components unless otherwise specified. In addition, terms such as "... unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software. .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating an example of a method of improving the accuracy of situation determination through audio-visual detection according to an embodiment of the present invention, and FIG. 2 is an accuracy of situation determination through audio-visual detection according to an embodiment of the present invention It is a flow chart showing another example of how to improve.

Referring to FIG. 1, a method of improving the accuracy of situation determination through audio-visual detection according to an embodiment of the present invention is a Bayesian network using a situation determination result through the audio-visual detection unit 100 as a root node Generating (Bayesian Network) (S100), Estimating a priori probability (a priori probability) of the root node based on statistical information learned from context data collected through the audio-visual sensor 100 (S200), A step (S300) of referring to the conditional probability of at least one child node for the root node from the information input from the user and the domain region information, and the posterior probability of the a priori probability of the root node based on the referenced conditional probability It may include a reasoning step (S400) for updating to (aposteriori probability).

The Bayesian network refers to a graph in which random variables included in decision-making are represented by nodes, and the causal relationship between these random variables as a directed edge. In this graph, each node can be a continuous or discrete random variable, and each node contains a conditional probability distribution. For example, the Bayesian network for two events (spring cooler and rain) where the grass can get wet can be shown in Table 1 below. In [Table 1] below, each node is a binary random variable and the directional edge represents a causal relationship, and each node may include a conditional probability.

[Table 1]

Information input from a user according to an embodiment of the present invention may be knowledge of a domain expert. The domain referred to herein refers to a range of knowledge recognized by a name in terms of a control area or a range of knowledge, and refers to a field that a user intends to judge through audio-visual sensing. For example, if it is desired to determine whether a fire has occurred through audio-visual detection, the domain expert may be a fire expert related to the fire, and information input from the user may be a cause of the fire that may be obtained from the knowledge of the fire expert.

The child node according to an embodiment of the present invention is formed based on the cause of the situation determination result through the audio-visual detection unit 100, and the cause for the situation determination result through audio-visual detection is the information input from the user and the domain area It may be possible to derive from the information. In other words, when knowledge of a domain expert and information of a domain area are collected through the domain data collection unit 200, a cause for a situation to be determined can be derived from the collected information, and conditional probability for each cause can also be referred to. Can. Through this, it is possible to configure a child node based on each of the causes derived when the Bayesian network is generated using the situation determination result through the audio-visual detection unit 100 as a root node and the conditional probability for the cause.

A priori probability refers to the probability of an event that is known without obtaining additional information about a specific event, and according to an embodiment of the present invention, statistics learned from context data using machine learning such as deep learning Based on the information, the a priori probability of the root node can be estimated, and the conditional probability of the child nodes can be referenced from domain expert knowledge and domain domain information.

Referring to FIG. 2, as a preliminary step of updating an a priori probability of a root node to a posterior probability based on a referenced conditional probability according to an embodiment of the present invention, any one of the at least one child node When a cause situation corresponding to one occurs, further comprising the step of detecting the cause situation (S350), and updating the a priori probability of the root node with a posterior probability through the conditional probability of the child node corresponding to the detected cause situation. It may be possible to control the degree of belief in (belief).

At this time, the degree of belief means that if any evidence is given in the inference process of the Bayesian network, the conditional probability or marginal probability of each node changes according to the evidence, which is called the degree of belief. As evidence is collected, it can be updated with new values.

For example, in [Table 1], if the probability of rain was 0.2 and the probability of rain was 0.8 as a priori probability by the deep learning recognition device including the visual sensor, based on the measured evidence of other nodes, You can update the probability that it is raining or not raining. If evidence was obtained by measuring that the grass was wet, P(R=T|G=T) would increase to 0.3577, supporting a greater level of belief in the hypothesis that it was raining.

In other words, if a cause situation occurs for each child node according to an embodiment of the present invention and it is detected or confirmed, the a priori probability of the root node can be increased or decreased from the conditional probability of the node where the cause situation occurred. And through this update process, the accuracy of judgment can be improved.

3 is a block diagram showing an example of a Bayesian network using a determination result for a fire detection situation according to an embodiment of the present invention as a root node, and FIG. 4 is a determination of a fire detection situation according to an embodiment of the present invention This table shows an example of conditional probability of child nodes for a result.

For example, if it is desired to determine whether a fire has occurred through audio-visual detection, a result of determining whether a fire is obtained from visual data using machine learning such as deep learning is set as the root node F, and child nodes for the root node F As a field, a Bayesian network as shown in FIG. 3 may be generated in consideration of a weather node W, a humidity node H, a fire vulnerable node B, and a smoke humidification area (eg, a factory chimney) node A. The set child nodes and conditional probability for each child node can be extracted from the knowledge of a fire expert (domain expert) who has experienced a lot of fire or can be derived and referenced from fire occurrence and fire related information. Referring to FIG. 4, an example of a conditional probability table for each of these child nodes can be confirmed.

For example, the probability of occurrence of a fire (a priori probability) estimated based on statistical information learned using machine learning such as deep learning is P(F)=0.87, and each fact or multiple facts for each node When this cause is detected through the situation detecting unit 600, based on this, the probability of a fire (a priori probability) is updated with a posterior probability, so that the value of P(F)=0.87 can be increased or decreased. When the probability increases or decreases in this way, it is possible to reduce miss-detection or reduce the possibility of false detection in determining whether a final fire has occurred.

5 is a block diagram illustrating an example of a system 1000 for improving the accuracy of situation determination through audio-visual sensing according to an embodiment of the present invention.

Referring to FIG. 5, the system 1000 for improving the accuracy of situation determination through audio-visual sensing according to an embodiment of the present invention collects context data through at least one of visual sensing and auditory sensing, and collects Audio-visual detection unit 100 for learning statistical information through machine learning based on contextual data and determining the situation, a domain data collection unit 200 for collecting information input from a user and domain domain information, and an audio-visual detection unit ( 100) estimates the a priori probability of the root node based on the statistical information learned through the context determination network generation unit 300 and the audio-visual detection unit 100 that generates a Bayesian network using the context determination result as a root node, Update the a priori probability of the root node with a posterior probability based on the probability reference unit 400 referring to the conditional probability of at least one or more child nodes for the root node from the information input from the user and the information in the domain region, and the referenced conditional probability It may include a network judgment unit 500 to determine the situation.

For example, in the case of a system for determining whether a fire has occurred, the audio-visual detection unit 100 may be a visual detection-based fire alarm device including a CCTV for fire detection. The Bayesian network shown in FIG. 3 may be generated by the context determination network generation unit 300 based on the context data collected from the fire alarm device and the learned statistical information.

The child node according to an embodiment of the present invention is formed based on the cause of the situation determination result through the audio-visual detection unit 100, and the cause for the situation determination result through audio-visual detection is the information input from the user and the domain area It may be possible to derive from the information.

Referring to FIG. 5, when a cause situation corresponding to any one of the at least one child node according to an embodiment of the present invention occurs, the cause situation detecting unit 600 for detecting the cause situation further includes a situation determination network The a priori probability of the root node may be updated with a posterior probability through the conditional probability of the child node corresponding to the cause situation detected by the inference unit 500 to control the degree of belief in the root node.

The cause situation detecting unit 600 according to an embodiment of the present invention may be formed to correspond to a cause situation corresponding to each child node of the Bayesian network. That is, in order to update the a priori probability of the root node with a posterior probability, it is necessary to detect or check whether a cause situation corresponding to each child node has occurred, so it is included in the cause situation detection unit 600 to correspond to the number of generated child nodes. Sensor can be formed. For example, referring to FIG. 3, when a weather node, a humidity node, a fire vulnerable node, and a smoke humidification area node are formed as a child node for a fire occurrence determination result, each child is included in the cause situation detection unit 600. A weather sensor, a humidity sensor, a fire vulnerability determination sensor, and a smoke humidification area determination sensor may be included to detect the cause of the node.

With respect to the system according to an embodiment of the present invention, the contents of the above-described method may be applied. Therefore, with respect to the system, the description of the same contents as the contents of the above-described method is omitted.

On the other hand, according to an embodiment of the present invention, it is possible to provide a recording medium readable by a computer recording a program for executing the above-described method on a computer. In other words, the above-described method may be written in a program executable on a computer, and may be implemented in a general-purpose digital computer that operates the program using a computer-readable medium. In addition, the structure of data used in the above-described method may be recorded on a computer-readable medium through various means. A recording medium that records an executable computer program or code for performing various methods of the present invention should not be understood as including temporary objects such as carrier waves or signals. The computer-readable medium may include a storage medium such as a magnetic storage medium (eg, ROM, floppy disk, hard disk, etc.), optical read media (eg, CD-ROM, DVD, etc.).

The above description of the present invention is for illustration only, and a person having ordinary knowledge in the technical field to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted to be included in the scope of the present invention. .

100: audio-visual detection unit 200: domain data collection unit
300: situation determination network generation unit 400: probability reference unit
500: situation determination network reasoning unit 600: cause situation detection unit
700: database
1000: system to improve the accuracy of audiovisual detection

Claims (7)

In the method of improving the accuracy of the situation judgment through audio-visual detection,
Generating a Bayesian network in which the context determination result through the audio-visual detection unit is a root node in the context determination network generation unit;
Estimating a priori probability of the root node based on statistical information learned from context data collected by the audio-visual sensing unit in a probability reference unit;
Referring to the conditional probability of at least one child node for the root node from information input from a user and information of a domain region in the probability reference unit; And
And a reasoning step of updating the a priori probability of the root node to a posterior probability based on the referenced conditional probability in the context determination network reasoning unit.
According to claim 1,
The child node is formed based on a cause for a situation determination result through the audio-visual detection unit,
A method for improving the accuracy of situation determination through audio-visual detection, wherein the cause for the situation determination result through the audio-visual detection can be derived from information input from the user and information in a domain area.
According to claim 1,
As a pre-step of the reasoning step of updating the a priori probability of the root node with a posterior probability based on the referenced conditional probability, if a cause situation corresponding to any one of the at least one child node occurs, the cause Further comprising the step of detecting the cause situation in the situation detection unit,
The condition determination network inference unit updates the a priori probability of the root node with a posterior probability through the conditional probability of the child node corresponding to the detected cause situation, so that it is possible to adjust the degree of belief in the root node. How to improve the accuracy of situation judgment through audio-visual detection.
In the system for improving the accuracy of the situation judgment through audio-visual detection,
An audio-visual sensing unit for collecting context data through at least one of visual or auditory sensing, learning statistical information through machine learning based on the collected context data, and determining the context;
A domain data collection unit collecting information input from a user and information of a domain area;
A context determination network generation unit that generates a Bayesian network using a context determination result through the audio-visual detection unit as a root node;
Estimating the a priori probability of the root node based on statistical information learned through the audio-visual sensing unit, and referring to conditional probability of at least one or more child nodes for the root node from information input from the user and information in a domain region Probability reference; And
And a context determination network inference unit for updating the a priori probability of the root node with a posterior probability based on the referenced conditional probability.
The method of claim 4,
The child node is formed based on a cause for a situation determination result through the audio-visual detection unit,
A system for improving the accuracy of situation determination through audio-visual detection, wherein the cause for the situation determination result through the audio-visual detection can be derived from information input from the user and domain domain information.
The method of claim 4,
When the cause situation corresponding to any one of the at least one child node occurs, further comprising a cause situation detection unit for detecting the cause situation,
The situation determination network inference unit updates the a priori probability of the root node to a posterior probability through the conditional probability of the child node corresponding to the sensed cause situation, thereby adjusting the degree of belief in the root node. System to improve the accuracy of situation judgment through detection.
A computer-readable recording medium on which a program for implementing the method of any one of claims 1 to 3 is recorded.

Images