KR102399121B1 - Early screening system and method for high-risk groups of suicide using artificial intelligence (AI)-based video and audio information - Google Patents

Abstract

The present invention relates to a system and method for early screening of a high-risk group using artificial intelligence (AI)-based video and audio information, comprising a subcomputer installed with a 3D camera for acquiring valid data by photographing the behavior of an interviewer for a high-risk group a measurement device including an information acquisition device and a measurement server connected to the information acquisition device wirelessly or by wire to store valid data according to an input image; an analysis module for separating each data according to a face, an expression, and an action based on the valid data applied by being connected to the measuring device by wire or wirelessly; An analysis server that analyzes the psychological state of an interviewer in a high-risk group by matching the separated image applied from the analysis module to pre-stored standardized data, and displays the results; characterized in that it includes; application of commercialized vision-detecting technology By linking with the existing motion recognition technology and motion-related algorithms (technology), it is possible to build a system with a more powerful evaluation function, and by applying the algorithm for the psychological neuroscientific standard data table, the psychological It is applicable to the field of IoT interworking technology using applicable sensor devices in the field of emotional and psychotherapy that can be applied as a basic evaluation program for problems. It has the effect of objectively evaluating the psychological state by analyzing and judging through the behavioral standardized data written by scientific experts.

Description

Early screening system and method for high-risk groups of suicide using artificial intelligence (AI)-based video and audio information}

The present invention relates to a system and method for early screening of high-risk groups of suicide using artificial intelligence (AI)-based video and audio information. It relates to a system and method for early screening of high-risk groups of suicide using artificial intelligence (AI)-based video and audio information that enables objective evaluation of psychological states by analyzing and judging (or calculating) through behavioral standardized data prepared by experts. .

In general, objective and quantitative criteria for suicidal behavior, including suicidal thoughts and suicidal attempts, are unclear. Therefore, the management points of suicide prevention centers in the region are ambiguous, and it is difficult to closely manage them. In addition, each local government is implementing various projects to improve the suicide field rating of the Ministry of Public Administration and Security's local safety index, but the effectiveness is insignificant, and efforts are being made to switch the counseling and management system to a non-face-to-face focus in the post-corona era.

In addition, since the person who commits suicide has very limited communication with society, communication with family, lovers, or friends is a great strength. However, it is common to make extreme choices after hiding emotional expressions such as depression, loss, and frustration, and these sudden emotional changes often lead to suicidal behavior.

As such, the psychological state of the suicide victims may be unstable, and this psychological state is not accurately monitored, causing problems in many local governments.

As described above, as a prior document for understanding the psychological state based on information technology, the 'mental state judgment system and method using sensing equipment' of Republic of Korea Patent No. 10-1689021 (2016.12.23. Announcement) is the subject of counseling or interviewing. measuring at least one of motion information, biosignal information, and voice information of the subject using at least one sensing device; Filtering at least one state type information corresponding to the current state of the subject by using the measurement data of the at least one sensing device based on a plurality of state type information classifiable by the measurement data for each measurement factor ; and matching the filtered at least one state type information with a psychological classification table to extract and provide at least one psychological state possessed by the examinee, wherein the psychological classification table includes a plurality of heterogeneous psychological states. In contrast, among the plurality of state type information, corresponding state type information corresponding to each of the psychological states is individually mapped, and the motion information of the examinee includes facial expression information extracted from the face region and gesture information extracted from the body region of the examinee. including, wherein the biosignal information of the examinee includes at least one of a pulse, body temperature, blood pressure, and EEG signals of the examinee, and the step of filtering the at least one state type information includes: A plurality of state type information that can be classified by State type information is individually stored, and the plurality of first motion state type information is a plurality of behavior type information that can be classified in response to a muscle part in which a change in motion is detected in the face region, and the at least one psychological state is extracted. and providing, in the case of a measurement factor of the motion information, matching the filtered first motion state type and the second motion state type with the psychological classification table, respectively, to obtain a psychological state corresponding to the facial expression and the gesture Separately, by comparing the emotional level difference between the psychological state matched to the first motion state type and the psychological state matched to the second motion state type, if the emotional level difference is equal to or greater than a threshold, the second motion state type The mapped mental state is erased, and only the mental state matched to the first motion state type is extracted and provided, and the plurality of state type information for the voice information includes: trembling of the subject's voice, a sudden increase or decrease in voice strength , time elapsed for answering questions, specific preset It is characterized in that each includes state type information corresponding to a plurality of types selected from types corresponding to repeated statements of words.

According to the above-mentioned prior literature, the psychological state judging system using sensing equipment and its method, the psychological state corresponding to the behavioral pattern of the examinee can be derived more objectively by monitoring and analyzing the subject's behavioral patterns in real time during counseling or interview. And there is an advantage that can increase the identification accuracy of the psychological state.

However, the above prior literature has a problem in that it is not possible to objectively evaluate the interviewer psychologically and neuroscientifically for each type of question.

Republic of Korea Patent Registration No. 10-1689021 (2016.12.23. Announcement)

The present invention can objectively evaluate the psychological state by acquiring the behavioral pattern of a real-time subject through a 3D camera and analyzing and judging the behavioral pattern standardized data written by a psychology and neuroscience expert in order to improve the problems of the prior literature It aims to provide a system and method for early screening of high-risk groups of suicide using artificial intelligence (AI)-based video and audio information.

In addition, in order to provide an objective identification method, the present invention provides an artificial intelligence (AI)-based video and audio information that enables the psychological and neuroscientific objective evaluation of the interviewer for each type of question through specific motion analysis technology. The purpose of this study is to provide a system and method for early screening of high-risk groups for suicide.

The present invention is a means for achieving the above object,

An information acquisition device including a sub-computer equipped with a 3D camera to acquire valid data by photographing the behavior of the interviewer, and a measurement server connected to the information acquisition device wirelessly or by wire to store valid data according to the input image measuring device; an analysis module for separating each data according to a face, an expression, and an action based on the valid data applied by being connected to the measuring device by wire or wirelessly; An analysis server that analyzes the psychological state of the interviewer and displays the results by matching the separated image applied from the analysis module to the standardized data stored in advance; using artificial intelligence (AI) based video and audio information comprising: Provides an early screening system for high-risk groups of suicide.

The analysis module of the present invention includes: a face analysis module for judging and figuring out features by threshold classification based on the average difference of pixel values for black and white regions; a motion recognition module composed of action units to analyze the activity state of a specific muscle; A voice recognition module that extracts features from a voice, specifies the extracted minimum value (Minimum pitch (Hz)) and the maximum value (Maximum pitch (Hz)), and extracts 12 features from within the specified voice waveform; characterized by including; do it with

The face analysis module of the present invention is calculated by subtracting the sum of the brightness of the black part from the sum of the brightness of the image pixels in the white part, and it is calculated according to whether this value is greater than or less than the threshold value assigned to the feature when searching for images and images. It is characterized in that it is assumed to be the target object to be grasped.

The motion recognition module of the present invention provides a state of occurrence or non-occurrence including under the forehead, pulling the tip of the lip, under the tip of the lip, closing the lips, sucking the lips, and blinking the eyes, and raising the center of the forehead, raising the outside of the forehead, under the forehead, above the eyelids, the cheeks It is characterized by recognizing the state of occurrence, including upper, nose frown, upper lip, lip pull, dimple, lower lip, upper lower jaw, lip extension, lip split, and lower chin.

Extraction of 12 features from within the voice waveform of the present invention is the extraction of the high/low, slope, and high/low changes of the waveform, the number of syllables per second measured, the CID ratio, the number of pitches per utterance, the number of words per utterance, the number of content words per utterance , phonemic, semantic, subsequent utterance start time, inverted words, interprojection, repeated words, and modified words are characterized in that they are extracted.

Analysis server of the present invention, based on the valid data supplied from the analysis module, provides a history in the form of a graph according to the change of time for emotions, provides the corresponding video linkage for each emotion according to the history, and provides analysis data according to behavior recognition analysis characterized in that

The present invention is another means for achieving the above object,

a motion detection and voice recognition step of detecting a motion by detecting a facial expression, a head and a hand movement through an image or image input from a measurement device, and acquiring an image or image; A specific area and voice feature extraction step of extracting a specific area from the valid data applied through the motion detection and voice recognition step to understand the emotional state of the interviewer; and an analysis step of determining and analyzing the psychology of the face electron by matching it with pre-stored standardized data based on the area data applied through the specific area and voice feature extraction step; and an early screening method for high-risk groups of suicide using voice information.

The motion detection and voice recognition step of the present invention includes, through the 3D camera of the measuring device, the presence or absence of occurrence including under the forehead, pulling the tip of the lip, under the tip of the lip, closing the lips, sucking the lips, and blinking the eyes, raising the center of the forehead, the forehead Recognizes and measures the degree of occurrence including external elevation, under the forehead, over the eyelids, over the cheeks, nose wrinkle, upper lip, lip pull, dimple, lower lip, upper lower jaw, lip extension, split lip, and lower chin It is characterized in that it is stored on the server.

In the specific area, voice feature extraction step of the present invention, the face is rotated so that the two eyes are exactly horizontal using eye detection to transform the face to face the front, and then the face is reduced so that the distance between the two eyes is always the same. geometric transformation and trimming steps of cutting out the background, hair, forehead, ears, and chin from the face image; a histogram equalization step of equalizing the histogram by dividing the left and right sides of the face to obtain standardized brightness and contrast values because the left and right faces may look completely different depending on lighting; a smoothing step of making the image smooth and clear the boundary line by using a Bilateral Filter to reduce pixel noise that may be increased in the histogram equalization step; A mask step of extracting only the face facing the front by covering the smoothed image with an oval mask; A voice waveform recognition step of detecting a voice waveform of a minimum value and a maximum value from the voice input through the microphone; and a waveform feature extraction step of extracting 12 features from the audio waveform extracted through the speech waveform recognition step (S25).

The waveform feature extraction step of the present invention includes: extracting the high-low, slope, and high-low variation of the waveform, the number of syllables per second measured, the CID ratio, the number of pitches per utterance, the number of words per utterance, the number of content words per utterance, the phoneme It is characterized in that it is a semantic word, subsequent utterance start time, inverted word, liver projection, repeated word, and modified word.

In the analysis step of the present invention, the valid data obtained through the motion detection and speech recognition step (S10) is obtained through the geometric transformation and smoothing step of the specific region, the speech feature extraction step, the histogram equalization step, the smoothing step, and the mask step. Extract and display objective information and interpretation data by comparing and matching the data extracted from 12 features within the face image and voice waveform with the standard data table, and comparing the data according to the matched image and voice waveform features with the patient group and normal group data characterized in that

The present invention can build a more powerful evaluation function system by interworking with existing motion recognition technology and motion-related algorithms (techniques) by applying the commercialized vision-detecting technology, and it is a psychological and neuroscientific standard data table. By applying the algorithm of

The present invention has the effect of objectively evaluating the psychological state by acquiring the behavioral pattern of a real-time subject through a 3D camera and analyzing and judging the behavioral pattern standardized data written by a psychology and neuroscience expert.

In addition, the present invention can be effectively used in various fields, and in particular, by providing an objective identification method by motion analysis technology, there is an effect that the interviewer's psychological and neuroscientific objective evaluation for each type of question is effective. .

1 is a schematic diagram illustrating an early screening system for high-risk groups of suicide using artificial intelligence (AI)-based video and audio information of the present invention.
2 is a flowchart illustrating an early screening method for a high-risk group using artificial intelligence (AI)-based video and audio information of the present invention.
FIG. 3 is a diagram illustrating an extraction step according to the specific region and voice feature extraction step of FIG. 2 .
4 is a view showing a state in which the position of the hand is determined for action analysis in the motion detection and voice recognition step shown in FIG. 2 .
5 is a view showing the state of the process of cutting out the background, head, forehead, and chin from the face through the geometric transformation step of the face in the specific region and voice feature extraction step shown in FIG. 3 step by step;
FIG. 6 is a diagram illustrating a state in which an image is obtained by equally dividing a face through a histogram equalization step in a specific region and voice feature extraction step shown in FIG. 3 .
7 is a view showing a state in which a face is extracted by applying a mask to the face through the mask step in the specific region and voice feature extraction step shown in FIG. 3 .

Hereinafter, it should be noted that the technical terms used in the present invention are only used to describe specific embodiments, and are not intended to limit the present invention, and the technical terms used in the present invention have a different meaning in particular in the present invention. Unless defined, it should be interpreted in the meaning generally understood by those of ordinary skill in the art to which the present invention pertains, and should not be interpreted in an overly comprehensive meaning or in an excessively reduced meaning.

In addition, when the technical term used in the present invention is an incorrect technical term that does not accurately express the spirit of the present invention, it should be understood by being replaced with a technical term that can be correctly understood by those skilled in the art. In addition, the general terms used in the present invention should be interpreted according to the definition in the dictionary or according to the context before and after, and should not be interpreted in an excessively reduced meaning.

In addition, the singular expression used in the present invention includes a plural expression unless the context clearly indicates otherwise. It should not be construed as necessarily including all of several steps, and it should be construed that some components or some steps may not be included, or additional components or steps may be further included.

A system and method for early screening of high-risk groups of suicide using artificial intelligence (AI)-based video and audio information according to the present invention will be described.

The system for early screening of high-risk groups using artificial intelligence (AI)-based video and audio information of the present invention includes a subcomputer 111 installed with a 3D camera 112 for acquiring valid data by photographing the behavior of the interviewer. a measurement device 100 including a measurement server 120 connected to the acquisition device 110 and the information acquisition device 110 wirelessly or by wire to store valid data according to an input image; an analysis module 200 by separating each data according to a face, an expression, and an action based on the valid data applied from the measurement device 100; and an analysis server 300 that analyzes the psychological state of the interviewer by matching the image separated by the analysis module 200 with pre-stored data and displays the results.

The system and method for early screening of high-risk groups using artificial intelligence (AI)-based video and audio information of the present invention having the above characteristics will be described in detail with reference to the accompanying drawings.

When described in detail with reference to FIG. 1 , the system for early screening of a high-risk group using artificial intelligence (AI)-based video and audio information of the present invention includes a measurement device 100 , an analysis module 200 , and an analysis server 300 . include

The measurement device 100 is connected to the information acquisition device 110 and the information acquisition device 110 including a sub-computer 111 installed with a 3D camera for acquiring valid data by photographing the interviewer’s behavior wirelessly or by wire. and a measurement server 120 for storing valid data according to the input image.

The analysis module 200 is connected to the measurement device 100 by wire or wirelessly so as to classify and analyze each data according to the face, expression, and behavior based on the applied valid data, the face analysis module 210, It includes a motion recognition module 220 and a face analysis module 210, and analyzes the image data classified and analyzed by the face analysis module 210, the motion recognition module 220, and the face analysis module 210 ( 300) is approved.

The face analysis module 210 is calculated by subtracting the sum of the brightness of the black part from the sum of the brightness of the image pixels in the white part, and depending on whether this value is greater than or less than the threshold value assigned to the feature when searching for images and images Assume that it is the object to be grasped.

The face analysis module 210 is an image feature used for image recognition, SIFT (Scale-Invariant Feature Transform) using feature points, HOG (Histogram of Oriented Gradient) using template matching, and LBP (Local Binary Pattern) ) and MCT (Modified Census Transform), among others, the Haar-like Feature, which is mainly used together with Adaboost (Adaptive Boosting), basically measures the difference in brightness between regions in the image. use it

In addition, according to the basic features according to the shape, a plurality of these are combined to extract the features of the object at various positions and sizes.

The feature value of each basic feature is calculated by subtracting the sum of the brightness of the black part from the sum of the brightness of the image pixels in the white part of the square, and this value is the threshold value assigned to the feature when searching for images and images. Depending on whether it is larger or not, it is assumed to be the object to be grasped.

Here, it is important to identify meaningful features in order to be sure that the object is an object, which is the same as distinguishing between cases where similar values are displayed in the recognition object in a specific area, but random values are generated when it is not the object.

In the case of face recognition using a Haar-like feature, for example, the eye area is darker than the surrounding area, so this feature is a meaningful feature for human face detection.

And the selection of these meaningful features is done through a machine learning algorithm such as a boosting algorithm such as Adaboost. As such, the Har-like feature is relatively suitable for application because it has a characteristic difference in brightness in the case of a human face because it uses the difference in brightness of the unit area while having geometric information of the object by the basic feature.

The motion recognition module 220 includes a state of occurrence or non-occurrence including under the forehead, pulling the tip of the lip, under the tip of the lip, dry lips, sucking the lips, and blinking the eyes, raising the center of the forehead, raising the outside of the forehead, under the forehead, above the eyelids, Recognize the occurrence status including cheeks, nose frown, upper lip, lip pull, dimple, lower lip, lower chin, lip extension, split lip, and lower chin.

The voice recognition module 230 extracts a feature from the voice input through the microphone, and designates the extracted minimum value (Minimum pitch (Hz)) and the maximum value (Maximum pitch (Hz)).

The voice recognition module 230 extracts the 12 features extracted from the voice waveform, including the high and low slope of the voice waveform, the number of changes in the waveform, the number of syllables per second measured, the CID ratio, the number of pitches per utterance, and the number of words per utterance. , the number of content words per utterance, phonemic verbs, semantic verbs, subsequent utterance start time, inverted words, interprojection, repeated words, modified words, etc. are extracted.

The analysis server 300 analyzes the psychological state of the interviewer and displays the results by matching the separated image applied from the analysis module 200 to the pre-stored standardized data.

In addition, the analysis server 300 compares and analyzes the standard data table 310 and the valid data, and the analysis and results for the compared and analyzed valid data are graphs of "expression analysis" and "behavioral analysis" based on the analysis results. is automatically generated, and according to each item in the graph, the graph information for the item in the line graph is clearly displayed.

The analysis server 300 digitizes the 12 features extracted from the voice waveform specified by the voice recognition module 230 , and compares and analyzes the quantified data with pre-stored data of the patient group and the normal group.

Early detection of high suicide risk group through emotional recognition analysis to determine psychological state using artificial intelligence (AI)-based video and audio information performed in the early screening system for high suicide risk group of the present invention configured as described above The selection method is explained.

The method for early screening of high-risk groups of suicide using artificial intelligence (AI)-based image and audio information performed in the system for early screening of high-risk groups of the present invention includes facial expressions, head and hand movements through images input from the information acquisition device 110 . a motion detection and voice recognition step (S10) of detecting a motion and acquiring an image or an image; A specific area, voice feature extraction step (S20) of extracting a specific area from the valid data applied through the motion detection and voice recognition step (S10) to understand the emotional state of the interviewer; and an analysis step (S30) of determining and analyzing the psychology of the face electron by matching it with pre-stored standardized data based on the area data applied through the specific area and voice feature extraction step (S20).

The specific region extraction step (S20) includes a geometric transformation and smoothing step (S21), an equalization step (S22), a smoothing step (S23), and a mask step (S24).

The geometric transformation and trimming step (S21) is a step of transforming the face to face the front. This is because the face recognition algorithm can find and recognize accurate feature points only when the faces are well aligned. In other words. It seems to revive the aligned face image when detecting the face, but since the process of reviving the face is not accurate, the geometric transformation and trimming step (S21) is performed to obtain an accurate face image.

Also, for more accurate alignment, eye detection is used to rotate the face so that both eyes are exactly horizontal. Also, reduce the face so that the distance between the two eyes is always the same. Afterwards, the background, head, forehead, ears, and chin are cut out from the face image to obtain an image of a specific part.

The equalization step (S22) is greatly affected by lighting during face recognition. In particular, depending on the lighting, the left and right faces may look completely different. Therefore, in order to obtain more standardized brightness and contrast values, the histogram is equalized by dividing the left and right sides of the face.

And if the histogram is equalized by dividing the left and right sides, a boundary is created. Therefore, histogram equalization is performed from left to center or from right to center. Then, the three images are combined, up to the image in which the entire face is histogram equalized.

In the smoothing step (S23), if the histogram equalization is performed, pixel noise of the image may increase. A smoothing process is needed to reduce this pixel noise. In this step, a Bilateral Filter is used. The Bilateral Filter makes the image smooth and sharpens the boundary line.

In the mask step (S24), the background image has already been removed in the geometric transformation and trimming step (S21), but a process of putting on an oval mask is performed to prevent problems in the recognition process due to the shadow of the corner part of the image. . In this process, only the face facing the front is extracted by drawing an ellipse on the smoothed image.

As for the facial image extracted through the above steps, the image features used for image recognition include SIFT or Naive Bayes classifier using feature points, HOG using template matching, and LBP and MCT, among others. The Haar-like feature, which is mainly used together with the Naive Bayes classifier and Adaboost (Adaptive Boosting), basically uses the difference in brightness between the region and the region in the image.

The Naive Bayes classifier is based on the very strong assumption that all dimensions are independent of each other for multidimensional feature vectors, and each dimension is also generally assumed to be a one-dimensional Gaussian probability distribution, and only the mean and variance are estimated for the entire training data.

When an input vector 'x' is given with respect to a given classification 'c', a posterior probability is expressed as Equation (1).

The likelihood of each dimension is assumed to be a normal distribution, and is calculated as in Equation (2).

The learning process requires only the mean and variance of the given learning material. In addition, since all dimensions are assumed independently in Equation (1), implementation is very simple and there are few factors affecting performance, so it is used to obtain baseline performance of emotion recognition.

And the Adaboost algorithm is a method of generating a strong classifier by combining many weak classifiers based on a simple hypothesis. A weak classifier is a classifier with similar results to random guessing. Samples are recognized using several weak classifiers, weights are decreased for correctly recognized samples, and weights are increased for samples that are not recognized and reflected in the next weak classifier. That is, the filters with the largest weight are used first, and the non-face images are filtered out as much as possible first, and then the filters with the next largest weight are used in order. Ultimately, the strong classifier is composed of the pressure regulation of the weak classifiers generated in each stage.

In addition, the process of making one strong classifier is called boosting, and the strong classifiers are configured in a cascade form to finally create an Adaboost face detector. The boosting algorithm consists of three steps as shown in the figure below.

First, data is collected by dividing the face image into positive samples, non-face images, and negative samples. Among the Haar features obtained based on this data, iteratively until a satisfactory face detection rate is obtained, a small number of values that can distinguish faces from non-faces are found. Finally, a strong classifier is constructed by collecting the Haar features, that is, the weak classifiers.

Thereafter, the process of constructing the Adaboost high-speed face detector consisting of several stages by connecting the strong classifiers in a cascade form consists of four stages as follows.

First, the number of all steps and the face detection rate in each step are predetermined. It starts with a low detection rate in the starting stage, but increases the detection rate toward the latter stage. In this case, the face detection rate in each stage affects the number of weak classifiers constituting the strong classifiers in that stage.

Second, one strong classifier is trained while adding Haar features, that is, weak classifiers, until a specified face detection rate is reached in each step. When one stage is completed in this way, the strong classifier of the next stage (the third) is trained while repeating the previous process. At this time, the non-face image that participated in the training in the previous step is excluded and a new non-face image is added to train.

And in the case of a face image, only data that could not be distinguished in the previous step participate in the training in the next step. Repeat the above process repeatedly until the desired number of steps and face detection rate are obtained. In this way, the final (fourth) form of Adaboost for cascade face detection is configured.

In addition, according to the basic features according to the shape, a plurality of these are combined to extract the features of the object at various positions and sizes.

The feature value of each basic feature is calculated by subtracting the sum of the brightness of the black part from the sum of the brightness of the image pixels in the white part of the rectangle. Depending on whether it is larger or not, it is assumed to be the object to be grasped.

Here, it is important to identify meaningful features in order to be sure that the object is an object, which is the same as distinguishing cases in which similar values are displayed in the recognition target in a specific area, but random values are generated in the case of non-target objects.

Also, in the case of face recognition using a Haar-like feature, if the eye is taken as an example, the eye area is darker than the surrounding area, so this feature becomes a meaningful feature for human face detection.

And the selection of these meaningful features is done through a machine learning algorithm such as a boosting algorithm such as Adaboost. As such, the Har-like feature is relatively suitable for application because it has a characteristic difference in brightness in the case of a human face because it uses the difference in brightness of the unit area while having geometric information of the object by the basic feature.

In addition, SVM (Support Vector Machine) is used to analyze the facial expression feature points of the suicide risk group.

The SVM (Support Vector Machine) is a technique for classifying data through supervised learning, and when setting a decision boundary that divides a group and a group for data groups for which the classification result is known, a decision boundary (Hyper Plane), a decision boundary and learning We use the concept of margin between data.

The analysis server compares and analyzes the standard data table 310 and valid data, and selects each item from the pie chart graph while the graphs of “expression analysis” and “behavioral analysis” are automatically generated based on the analysis results as shown in FIG. 8 Then, the graph information for the item selected in the linear graph on the right is displayed more clearly.

The present invention can build a more powerful evaluation function system by interworking with existing motion recognition technology and motion-related algorithms (techniques) by applying a commercially available vision-detecting technology, as well as psychological and neuroscientific standard data By applying the algorithm to the table, it can be applied as a basic evaluation program for mental problems that are emerging as a social problem.

In addition, it is possible to objectively evaluate the psychological state by acquiring the behavioral patterns of the interviewer in the high-risk group in real time through the 3D camera, and analyzing and judging the behavioral pattern standardized data written by psychology and neuroscience experts.

Moreover, it can be effectively used in various fields, and in particular, by providing an objective identification method by motion analysis technology, the interviewer's psychological and neuroscientific objective evaluation for each type of question can be made.

While this specification contains numerous specific implementation details, these are not to be construed as limitations on the scope of any invention or claim, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. should be understood

On the other hand, the embodiments of the present invention disclosed in the present specification and drawings are merely presented as specific examples to aid understanding, and are not intended to limit the scope of the present invention. It will be apparent to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical idea of the present invention can be implemented in addition to the embodiments disclosed herein.

100: measurement device 110: information acquisition device
111: subcomputer 112: 3D camera
120: measurement server (delete) 200: analysis module
210: face analysis module 220: motion recognition module
230: voice recognition module 300: analysis server
310: standard data table S10: motion detection and voice recognition step
S20: specific region, voice feature extraction step S21: geometric transformation and refinement step
S22: Histogram equalization step S23: Smoothing step
S24: mask step S25: voice waveform recognition step
S26: Waveform feature extraction step S30: Analysis step

Claims (11)

The information acquisition device 110 including a subcomputer installed with a 3D camera for acquiring valid data by photographing the behavior of an interviewer in the high-risk group for suicide, and the information acquisition device 110 wirelessly or wiredly connected to effective data according to the input image a measurement device 100 including a measurement server 120 for storing;
By dividing the threshold by the average difference of the pixel values for the black and white area, each data according to the face, expression, and behavior can be separated based on valid data connected to the measurement device 100 by wire or wirelessly A face analysis module 210 for determining and grasping features, a motion recognition module 220 comprising action units to analyze the activity state of a specific muscle, extracting features from a voice, and the extracted minimum pitch (Hz)) and a maximum value (Maximum pitch (Hz)), the analysis module 200 including a voice recognition module 230 for extracting 12 features within the specified voice waveform;
By matching the separated image applied from the analysis module 200 to the standardized data stored in advance, the standard data table 310 and valid data are compared and analyzed so that the psychological state of the interviewer can be analyzed and the results expressed. Analysis and results of the analyzed valid data automatically generate graphs of “expression analysis” and “behavioral analysis” based on the analysis results, and the graph information about the items is clearly displayed in the linear graph according to each item in the graph. , an analysis server 300 that digitizes the features extracted by 12 within the voice waveform specified by the voice recognition module 230, and compares and analyzes the quantified data with pre-stored data of the patient group and the normal group;
An early screening system for high-risk groups of suicide using artificial intelligence (AI)-based video and audio information, characterized in that it includes a. delete According to claim 1,
The face analysis module 210,
It is calculated by subtracting the sum of the brightness of the black part from the sum of the brightness of the image pixels in the white part, and when searching for images and images, it is assumed that this value is the object to be identified depending on whether it is greater than or less than the threshold value assigned to the feature. An early screening system for high-risk groups of suicide using artificial intelligence (AI)-based video and audio information. According to claim 1,
The motion recognition module 220,
Conditions of occurrence including under the forehead, pulling the tip of the lip, under the tip of the lip, dry lips, sucking the lips, and blinking eyes;
Degree of occurrence including forehead lift, external forehead lift, under forehead, over eyelids, over cheeks, frown, over upper lip, lip pull, dimple, under lip, over lower jaw, elongated lip, split lip, under chin An early screening system for high-risk groups of suicide using artificial intelligence (AI)-based video and audio information, characterized by recognizing the state. According to claim 1,
Extraction of 12 features in the voice waveform,
Speech waveform pitch, slope, waveform variation feature extraction, number of syllables per second, CID ratio, number of pitches per utterance, number of words per utterance, number of content words per utterance, phoneme dialect, semantic language, subsequent utterance start time, inversion An early screening system for high-risk groups of suicide using artificial intelligence (AI)-based video and audio information, characterized in that they are words, liver projections, repeated words, and modified words. According to claim 1,
The analysis server 300,
Based on the valid data supplied from the analysis module 200, providing a history in the form of a graph according to time change for emotions, providing a corresponding video linkage for each emotion according to the history, and providing analysis data according to behavior recognition analysis An early screening system for high-risk groups of suicide using artificial intelligence (AI)-based video and audio information. Motion detection that detects facial expressions, head and hand movements through the video or image input from the measurement device 100, detects motion, acquires an image or image, and detects the minimum and maximum waveforms from the voice in the analysis module and a voice recognition step (S10);
A specific area of extracting a specific area from the valid data applied through the motion detection and voice recognition step (S10) to identify the emotional state of the interviewer, and extracting a characteristic from the voice waveform in the analysis module, a speech feature extraction step (S20); and
An analysis step (S30) of determining and analyzing the psychology of the face electron in the analysis server 300 by matching it with the standardized data stored in advance based on the area data applied through the specific area, the voice feature extraction step (S20);
A method for early screening of high-risk groups of suicide using artificial intelligence (AI)-based video and audio information performed in a system for early screening of high-risk groups of suicide, characterized in that it includes a. 8. The method of claim 7,
The motion detection and voice recognition step (S10),
Through the 3D camera of the measuring device, the status of occurrence including under the forehead, pulling the tip of the lip, under the tip of the lip, dry lips, sucking the lips, and blinking the eyes,
Degree of occurrence including forehead lift, external forehead lift, under forehead, over eyelids, over cheeks, frown, over upper lip, lip pull, dimple, under lip, over lower jaw, elongated lip, split lip, under chin Early detection of high-risk groups using artificial intelligence (AI)-based video and audio information performed in a high-risk group early screening system that recognizes the state and stores the voice waveforms according to the minimum and maximum values from the input voice in the measurement server selection method. 8. The method of claim 7,
The specific region, the voice feature extraction step (S20),
In order to transform the face to face forward, it uses eye detection to rotate the face so that the two eyes are exactly horizontal, then reduces the face so that the distance between the two eyes is always the same. , a geometric transformation and trimming step (S21) of cutting the jaw;
a histogram equalization step (S22) of dividing the left and right sides of the face to equalize the histogram in order to obtain standardized brightness and contrast values because the left and right faces may look like completely different faces depending on the lighting;
a smoothing step (S23) of smoothing the image using a Bilateral Filter to reduce pixel noise that may be increased in the histogram equalization step (S22) and clearing the boundary line (S23);
A mask step (S24) of extracting only the face facing the front by covering the image smoothed in the smoothing step (S23) with an oval mask;
A voice waveform recognition step (S25) of detecting a voice waveform of a minimum value and a maximum value from the voice input through the microphone;
a waveform feature extraction step (S26) of extracting 12 features from the voice waveform extracted through the voice waveform recognition step (S25);
A method for early screening of high-risk groups of suicide using artificial intelligence (AI)-based video and audio information performed in a system for early screening of high-risk groups of suicide, characterized in that it includes a. 10. The method of claim 9,
The waveform feature extraction step (S26) is,
Speech waveform pitch, slope, waveform variation feature extraction, number of syllables per second, CID ratio, number of pitches per utterance, number of words per utterance, number of content words per utterance, phoneme dialect, semantic language, subsequent utterance start time, inversion A method for early screening of high-risk groups of suicide using artificial intelligence (AI)-based video and audio information performed in an early screening system for high-risk groups, characterized in that they are words, liver projections, repeated words, and modified words. 8. The method of claim 7,
The analysis step (S30) is,
A specific area, geometric transformation and smoothing step (S21) of the speech feature extraction step (S20), the histogram equalization step (S22), the smoothing step (S23), the mask of the valid data obtained through the motion detection and speech recognition step (S10) Compare and match the data extracted from 12 features in the face image and voice waveform obtained through step S24 with the standard data table 310,
Artificial intelligence (AI)-based video and audio performed in an early detection system for high-risk groups of suicide, characterized in that the data according to the characteristics of the matched image and audio waveform are compared with the data of the patient group and the normal group to extract and display objective information and interpretation data An early screening method for high-risk groups of suicide using information.

Images