KR20220018230A - Method for monitoring authentication based on video call using neural network and system for the method - Google Patents

Abstract

As a method for monitoring authentication based on a video call using a neural network, the present invention may comprise: a step of receiving a facial image or an identification image of a user; a step of receiving the consultation data of a consultant for a non-face-to-face authentication during a video call; a step of calculating a prediction result for an existence probability per identification element included in the received facial image or identification image through the neural network; a step of comparing the calculated prediction result with the consultation data of the consultant; and a step of requesting inspection of the consultation data according to the comparison result. According to the present invention, an authentication rate of the non-face-to-face authentication using the non-face-to-face-based video call and a quality of the consultation work can be improved by monitoring the consultation contents in real-time while the consultation work of the consultant is in progress.

Description

Video call-based authentication monitoring method and system using a neural network {Method for monitoring authentication based on video call using neural network and system for the method}

The present invention relates to a method for monitoring authentication based on a video call using a video call neural network, and to a method for monitoring the consulting work of video call counselors through a neural network.

In recent years, thanks to the development of mobile communication terminals, particularly smartphones and mobile communication technologies, functions that were previously performed offline can be conveniently and conveniently used anytime, anywhere through mobile applications.

In particular, in recent years, in terms of improving user convenience, a technology for non-face-to-face processing of financial business through applications has been developed and provided. done in a way

The quality of the authentication of face-to-face authentication using such a video call is inevitably dependent on the agent who has to check whether the person is the same person directly through the video. Therefore, in order to maintain accurate identification and high quality of counseling, the service subject providing non-face-to-face services needs to establish a process for separately evaluating or managing counselors.

However, since the consultation process of video call is difficult to quantitatively evaluate during the process, it is difficult to objectively evaluate the quality. Indirect methods of evaluation are being used.

Therefore, there is a need for a method to increase the service quality of the non-face-to-face method by quantitatively monitoring the counselor's video call counseling behavior and objectively evaluating the quality.

An object of the present invention is to propose a method for evaluating and monitoring the contents of a video call consultation of a counselor through a neural network.

A video call-based authentication monitoring method according to the present invention for solving the above technical problem comprises the steps of: receiving a user's face image or ID image; receiving counseling data of a counselor for non-face-to-face authentication during the video call; calculating a prediction result for each identification element included in the received facial image or ID image through a neural network; comparing the calculated prediction result with the counselor's counseling data; and requesting the examination of the consultation data according to the comparison result.

Preferably, the identification element is a predetermined item determined for non-face-to-face authentication of the user or feature information on an image divided by a scale for each item.

It is preferable that the neural network is trained using an image labeled based on the verified counseling data as learning data.

The counseling data may be handwritten meta-information of a counselor regarding the existence of identification elements included in the face image or the ID image.

examining the consultation data and the prediction result according to the verification request; and when it is determined that the prediction result calculated through the neural network is inaccurate as a result of the inspection, learning the neural network using the facial image or the ID image labeled based on the counseling data as training data.

A consultation server for performing authentication monitoring based on a video call using a neural network according to the present invention for solving the above technical problem receives the user's face image or ID image, and the counselor's consultation data for non-face-to-face authentication during the video call communication department; a prediction result calculation unit for calculating a prediction result for the existence probability of each identification element included in the received facial image or ID image through a neural network; a comparison unit comparing the calculated prediction result with the counselor's counseling data; and an inspection request unit for requesting inspection of the consultation data according to the comparison result.

Preferably, the identification element is a predetermined item determined for non-face-to-face authentication of the user or feature information on an image divided by a scale for each item.

It is preferable that the neural network is trained using an image labeled based on the verified counseling data as learning data.

The counseling data may be handwritten meta-information of a counselor regarding the existence of identification elements included in the face image or the ID image.

an inspection unit that inspects the consultation data and the prediction result according to the inspection request; And when it is determined that the prediction result calculated through the neural network is inaccurate as a result of the inspection, a learning unit for learning the neural network using the face image or the ID image labeled based on the counseling data as learning data.

According to the present invention, the authentication rate of non-face-to-face authentication using non-face-to-face video call and the quality of counseling service can be improved by monitoring the counseling contents in real time while the counselor's counseling service is in progress.

In addition, according to the present invention, both the efficiency and accuracy of non-face-to-face authentication can be increased by reducing the number of inspection targets that require re-examination by using a neural network, and performing verification of a more difficult procedure only for the inspection target.

In addition, the present invention can more accurately evaluate the counseling quality of a counselor by objectifying the authentication process as an identification element expressed as a quantitative indicator using a neural network.

In addition, the prediction performance of the neural network can be continuously improved by additionally learning the neural network by comparing the output of the neural network during the counseling service with the actual agent's judgment.

1 is an exemplary diagram illustrating a counseling monitoring system according to an embodiment of the present invention.
2 is a flowchart illustrating a video call-based authentication monitoring method using a neural network of a counseling server according to an embodiment of the present invention.
3 is a diagram illustrating counseling data according to an embodiment of the present invention.
4 is a diagram showing the configuration of a neural network according to an embodiment of the present invention.
5 is a diagram illustrating an implementation example through a plurality of neural networks according to an embodiment of the present invention.
6 is an exemplary diagram illustrating a counseling monitoring system according to an embodiment of the present invention.
7 is a block diagram showing the configuration of a counseling server according to an embodiment of the present invention.

The following is merely illustrative of the principles of the invention. Therefore, those skilled in the art can devise various devices that, although not explicitly described or shown herein, embody the principles of the invention and are included in the spirit and scope of the invention. In addition, it should be understood that all conditional terms and examples listed herein are, in principle, expressly intended only for the purpose of understanding the inventive concept, and are not limited to the specifically enumerated embodiments and states. .

The above-described objects, features and advantages will become more apparent through the following detailed description in relation to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the invention pertains will be able to easily practice the technical idea of the invention. .

In addition, in the description of the invention, if it is determined that a detailed description of a known technology related to the invention may unnecessarily obscure the gist of the invention, the detailed description thereof will be omitted. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary diagram illustrating a counseling monitoring system according to an embodiment of the present invention.

Referring to FIG. 1 , the consultation monitoring system 1000 determines whether or not to authenticate the user using information input from the user 10 and the user 10 who perform self-authentication using the user terminal 200 , and It may be composed of a counselor 20 who performs face-to-face counseling, and a counseling server 100 that assists and monitors counseling tasks of the counselor 20 .

The user 10 may perform a video call for user authentication using the user terminal 200 , and may directly photograph and transmit the ID image 14 of the user 10 to the consultation server 100 .

The counselor 20 accesses the consultation server 100 and compares the ID image 14 transmitted from the user 10 with the face image 12 of the user 10 on the video call to determine whether to authenticate the person. . In addition, the counselor 20 may follow a process for performing various financial-related tasks, such as opening a bankbook, a loan, and issuance of a card, according to the identity verification result.

In this case, the counselor 20 may prepare the counseling data 22 as a result of identity verification during the counseling service for identity authentication and transmit it to the counseling server 100 . The consultation data 22 may be meta information about user characteristic information that can be a standard for identification of the user and others. For example, handwritten input information of the counselor 20 regarding whether to wear glasses, a hair style, etc. may be included.

Counseling server 100 calculates a prediction result on whether feature information on the face image 12 or ID image 14 received from the user 10 through a neural network is included, and the prediction result output through the neural network By comparing the counseling data 22 created by the counselor 20, the counselor 20's counseling work can be monitored.

Hereinafter, a video call-based authentication monitoring method using a neural network of the counseling server 100 will be described in detail with reference to FIG. 2 .

2 is a flowchart illustrating a video call-based authentication monitoring method using a neural network of the counseling server 100 according to an embodiment of the present invention.

Referring to FIG. 2 , the counseling server 100 may receive a face image 12 of the user 10 acquired in a video call or an ID image 14 taken by the user 10 ( S100 ). Here, the facial image 12 is image data including at least a part of the face of the user 10 obtained in a video call, and may be received in the form of continuously photographed image frames.

The ID image 14 may be a partial image obtained by separately extracting only the part of the ID photo printed on the ID from among the entire or all photographed images of the ID taken by the user 10 for self-authentication.

Alternatively, the consultation server 100 receives the ID image 14 captured and stored by the user 10 through past self-authentication or membership registration from a separate database, or is delivered from a server of an external organization that issues and manages other ID cards. It is also possible to take and use it.

Next, the counseling server 100 may receive the counselor's counseling data 22 for video call-based non-face-to-face authentication from the counselor 20 (S200). As described above, the consultation data 22 is meta information input by the counselor 20 while conducting a consultation service through a video call with the user 10 , and is characteristic information included in the face image 12 or ID image 14 . including the results of judgment on the existence of

More specifically, the consultation data 22 will be further described with reference to FIG. 3 .

3 is a diagram illustrating counseling data 22 according to an embodiment of the present invention.

Referring to FIG. 3 , in the consultation data 22 , the presence or absence of feature information in the ID image 14 may be defined as meta information about the ID status 22-1.

In addition, the presence or absence of feature information on the face image 12 may be defined as meta information about the video call state 22-2.

That is, the consultation data 22 is characteristic information that may be included in the image, and includes a result of determining whether an identification element is present according to the agent's naked eye. Therefore, the consultation data 22 may be composed of the state of the ID image 14 for each image, feature information that may be included in the face image 12 and a state value for the feature information.

Specifically, the identification elements are information included in a received image, and may be predefined as characteristic information that an agent can visually identify and quantitatively determine.

For example, as an identification element that can be included in the ID image 14, it is an identification factor for the status of the ID card itself, whether the focus of the ID image is blurred, whether a corrected photo is used, whether or not there is a hologram, whether there is a light reflection, 20 years or more Alternatively, it may be defined as information on the issuance of ID cards for more than 10 years, the presence or absence of a sticker attached to the ID card, the presence or absence of cracks due to long-term use, and the presence or absence of cracks.

In addition, as a video call state that may be included in the facial image 12 , environmental information such as ambient brightness, whether indoors or outdoors, etc. may also be included as identification elements.

In addition, as the characteristics of both the identification image 14 and the facial image 12, whether the hair covers the ears, the eyebrows, the forehead, such as the presence of glasses, the presence of a hat, the presence or absence of makeup or hairstyle, which are identification factors for the face itself It may be composed of factors that can be quantitatively determined according to a set scale such as whether to cover or the length of the hair.

In addition, the consultation data 22 may be implemented to be able to qualitatively input specific matters or processing results generated while consulting with the user 10 in a consultation memo.

However, in this embodiment, based on the counseling data 22 of the counselor 20 input as a quantitative identification factor, the quality of the agent 20's self-authentication service can be monitored using a neural network.

To this end, the counseling data 22 created by the counselor 20 may be transmitted to the counseling server 100 . The counseling server 100 can check the accuracy of the counselor's judgment by comparing the received counseling data 22 with the output value of the neural network, and can further determine whether an additional check for the agent's identity verification is necessary.

In addition, the counseling server 100 may reuse the labeling information corrected according to the inspection result as learning data of the neural network.

The counseling server 100 calculates a prediction result for the existence probability of each identification element included in the facial image 12 or ID image 14 received through the neural network (S300).

Specifically, the counseling server 100 uses a neural network trained to calculate the presence or absence of identification elements included in the face image 12 or the ID image 14 or whether it corresponds to a quantitatively divided scale using a neural network trained to produce a prediction result. A probability value for an identification element included in the face image 12 or the ID image 14 may be output.

Here, the neural network may be trained using a face image 120 or an ID image 14 labeled as containing an arbitrarily generated identification element, or based on the verified counseling data 22 of the agent 20 . It is also possible to configure the labeled labeling data and the face image 12 or ID image 14 used for actual counseling as a learning data set and use it for learning.

Specifically, the neural network numerically calculates and outputs a prediction result indicating the existence probability of an identification element included in the input user's face image 12 or ID image 14 . Therefore, the prediction result may be defined as a predicted probability value for the presence or absence of features such as a hologram, light reflection, and wearing glasses as an identification element in the face image 12 or ID image 14.

Hereinafter, the structure of the neural network according to the present embodiment will be described in more detail with reference to FIG. 4 .

4 is a diagram showing the configuration of a neural network according to an embodiment of the present invention.

Referring to FIG. 4 , the neural network 30 according to the present embodiment may be configured as a Convolution Neural Network (CNN) model composed of layers that perform a plurality of convolution operations.

When the facial image 12 or ID image 14 is input to the learned neural network 30, the characteristic values according to the unique shape or color indicating the identification element included in the image while passing through the layers inside the neural network 30 are It can be emphasized through convolution.

Various feature values included in the image data as identification elements for each face image 12 or ID image 14 are output in the form of a new feature map through operation with a filter determined for each convolution layer, The final feature map generated through the iterative operation for each layer may be input as a fully-connected layer and flattened. The difference between the flattened characteristic information and the reference characteristic information defined for each identification element may be calculated, and the existence probability of the identification element may be output as the prediction result 32 according to the calculated difference.

The training of the neural network 30 may be performed with a training data set classified for each identification element as labeling data including image data for the face and a result of determining whether an identification element exists. For example, a neural network can be trained using a plurality of image data labeled as having shadows, reflections, holograms, etc. as an identification element for the ID image, respectively, as training data, and a face wearing glasses as an identification element for the face image. Learning can be performed using an image, a facial image having a hair style with a parting.

The learned neural network 30 may determine the existence of an identification element with respect to the input face image 12 or ID image 14 , and provide a prediction probability value for each identification element as a prediction result 32 .

In addition, the neural network 30 according to this embodiment distinguishes the face image 12 or the ID image 14 and through the labeled training data to include each identification element, the face image 12 or ID image ( 14) It can be learned to predict the existence of an identification element as one identification target with the feature information included in it.

Through this, it is also possible to distinguishly output the determination result of the unique characteristic element included in the face image 12 and the determination result of the identification card image 14 .

It is also possible to use the identification element (facial feature information) that is common in the face image 12 and the ID image 14 as a monitoring basis for determining whether the same person is the same.

For example, in the case of Fig. 4 (a), when the facial image 12 is input to the neural network 30, the probability of wearing glasses among the identification elements present in the facial image 12 is 0.1, and the probability of existence of the part is 0.8 A prediction result 32 may be calculated with .

Also, in the case of (b), when the ID image 14 is input to the neural network 30, the probability of wearing glasses among the identification elements existing in the ID image 14 is 0.2, and the probability of existence of a parting is 0.6. If it is calculated, it can be commonly judged that glasses are not worn and the parting is present.

Furthermore, in another embodiment, the counseling server 100 may be configured to output a prediction result with the existence probability of the identification element using each neural network 30 learned for each identification element.

5 is a diagram illustrating an implementation example through a plurality of neural networks 30 according to an embodiment of the present invention.

Referring to FIG. 5 , the counseling server 100 uses neural networks 30-1, 30-2, and 30-3 corresponding to each identification element of the facial image 12 (or ID image 14) in parallel. It can be configured to output the prediction result 32 as a result.

For example, the counseling server 100 may use each of the neural networks 30 - 1 , 30 - 2 , and 30 - 3 learned to individually determine whether glasses are worn, parted, or hat is worn. Accordingly, each of the neural networks 30 - 1 , 30 - 2 , and 30 - 3 can calculate a more accurate prediction result for each identification element with respect to one face image 12 .

In this case, each of the neural networks 30 - 1 , 30 - 2 , and 30 - 3 may be trained using learning data labeled according to the presence or absence of a corresponding identification element or a divided scale in order to calculate a specific prediction result.

For example, it is possible to learn by labeling the presence or absence of parting in feature elements related to hairstyles, and it is also possible to quantitatively classify the length of hair into three stages, short, medium, and long, to learn and output the learned result. It is possible.

Furthermore, in the present embodiment, the facial image 12 input through the video call may be continuously input in a plurality of frames. In this case, the prediction result for each frame is output through the neural network, and the maximum or average value of the prediction result is calculated. It can be calculated and used statistically.

Then, the counseling server 100 may compare the calculated prediction result with the counselor's counseling data (S400). Specifically, when the probability of the identification element included in the prediction result exceeds the threshold, the consultation server 100 determines that the identification element exists in the ID image 14 or the face image 12, and the counselor 20 ) can be compared with the consultation data 22 to calculate a comparison result. Here, the comparison result is a value for whether the prediction result and the counseling data do not match, and a result may be generated for each item of the identification element.

For example, according to the output of the neural network, when the prediction result on whether to wear glasses is 60% or more, but it is not described as wearing glasses on the consultation data 22, a result may be generated as a non-matching item. .

In this case, the threshold value is a standard value of the probability of existence of the identification element included in the prediction result, and may be determined differently for each identification element, the type of ID card, the year of issue of the ID image, the type of user terminal used for photographing the ID, It may be a variable value dynamically determined by the type of user terminal used for the video call, the quality state of the video call, and the like.

Through the above process, the counseling server 100 may compare the prediction result determined through the neural network with the counseling data of the counselor 20 , and calculate the degree of agreement between the prediction result and the counseling data as a comparison result.

Next, the counseling server 100 determines whether to check the corresponding counseling data according to the calculated comparison result ( S500 ).

Specifically, the consultation server 100 may transmit a request for examination of the consultation data to the examination terminal of the examiner 40 when the comparison result meets the examination requirements. Here, the inspection criterion is a criterion for determining whether to inspect, and the number or degree of non-matching items may be determined.

In addition, the inspection criterion may be set to a variable value determined according to the performance evaluation result of the neural network. That is, the lower the performance evaluation of the neural network, the more strict the inspection criteria can be determined. For example, if an inspection target is determined based on the number of non-matching items between the prediction result of the neural network and the counseling data, as the performance evaluation result of the neural network improves, the number of permissible non-matching items in the counseling data to be inspected increases It is also possible to reduce step by step. That is, if the performance evaluation result is improved by continuous learning of the neural network using the inspection result, the inspection standard is updated accordingly.

That is, if the number of non-matching items is determined to be one as the initial inspection standard, if even one other item is included in the counseling data and the prediction result, the counseling data is determined as the inspection target. If this 80% or more is output with a high probability from the neural network, but it is recorded in the counseling data 22 as no division, the counseling server 100 determines that there is a problem in the counseling task or the neural network of the counselor 20, and the examiner (40) ) may request an additional inspection process.

Consultation server 100 may request inspection from the examiner 40 when the number of non-matching items on the comparison result of the counseling data 22 is equal to or greater than the number of non-matching items as the inspection standard.

Hereinafter, a detailed inspection process will be described with reference to FIG. 6 .

Referring to FIG. 6 , when the counseling data 22 received from the counselor 20 and the prediction result are different from each other, the counseling server 100 may request the examiner 40 to review. Consultation server 100 transmits inspection request information to the inspection terminal of the inspector 40, the inspection terminal receives the consultation data 22, the prediction result of the neural network, a facial image, an ID image, and by the inspector 40 As a result of the inspection performed, the inspection result data may be transmitted to the consultation server 100 .

When there is a request for inspection from the consultation server 100, the examiner 40 compares the prediction result based on the face image 12 and the ID image 14 with the consultation data 22 of the agent 20 with each other. can be compared

As a result of examining the consultation data 22 and the prediction result of the neural network through the inspection process, if it is determined that the agent 20 is erroneously judged, request attention from the agent 20 or take follow-up measures to determine the identity verification process. reliability can be ensured.

Conversely, when the prediction result calculated through the neural network 30 is found to be inaccurate, the inspector 40 may transmit the inspection result data to the counseling server 100 .

Here, the inspection result data is the result of the inspector 40 directly inspecting the consultation service, and includes the contents of the erroneously judged part among the prediction results of the neural network, and the judgment according to the inspector 40 on the consultation data 22 The values may include labeled identification images 14 or facial images 12 .

The consultation server 100 receives the inspection result data from the inspector 40, and the consultation server 100 generates learning data based on the inspection result data or uses the inspection result data itself as learning data to the neural network 30. You can learn more.

As described above, the counseling server 11 according to the present invention can objectively evaluate the video call counseling process of the counselor 20 by using the existence probability of the identification element, which is an index according to the quantitatively quantified probability of the neural network.

In addition, by monitoring the consultation service of the video call counselor 20 in real time, the authentication rate of non-face-to-face authentication using video call and the quality of consultation work can be increased, and predictive performance through re-learning the neural network by reflecting the inspection result can increase

Hereinafter, the configuration of the consultation server 100 will be described with reference to FIG. 7 .

7 is a block diagram showing the configuration of the consultation server 100 according to an embodiment of the present invention.

Referring to FIG. 7 , the counseling server 100 may include a communication unit 110 , a prediction result calculation unit 120 , a comparison unit 130 , an inspection request unit 140 , and a learning unit 150 .

The communication unit 110 may transmit/receive data required for identity authentication from the user 10 or the counselor 20 .

Specifically, the communication unit 110 may receive the user's face image 12 or the ID image 14 obtained in the video call from the user 10 .

Also, the communication unit 110 may receive the counselor 20's consultation data 22 for self-authentication of the video call from the counselor 20 .

In addition, the communication unit 110 may transmit an inspection request to the inspector 40 to inspect the consultation data 22 of the counselor 20, and may re-receive the inspection result data as the inspection result from the inspector 40 .

The prediction result calculation unit 120 may calculate a prediction result for the existence probability of each identification element included in the face image 12 or the ID image 14 received through the neural network 30 .

Specifically, the prediction result calculation unit 120 may calculate the prediction result using the neural network 30 trained to calculate the prediction result according to the presence or absence of an identification element included in the face image or ID image. Here, the identification element may be a quantitative determination item for the presence of feature information on the face determined for non-face-to-face authentication of the user 10 .

The neural network uses image data labeled for each judgment item as learning data, and can be trained using pre-verified counseling data of an actual agent or data re-labeled through inspection as learning data.

The comparison unit 130 may calculate a comparison result by comparing the prediction result calculated by the prediction result calculation unit 120 with the counseling data of the counselor 20 with each other.

Specifically, when the probability of the identification element included in the prediction result exceeds the threshold, the comparison unit 130 determines that the identification element exists in the ID image 14 or the face image 12, and If the probability of the included identification element is less than or equal to the threshold, it is determined that the identification element does not exist in the ID image 14 or the facial image 12, and compared with the counseling data of the agent 20 based on the prediction result of the neural network. Thus, a comparison result can be calculated.

The inspection request unit 140 may determine whether to inspect the consultation data according to the comparison result of the comparison unit 130 . The inspection request unit 140 determines that there is no problem in the consulting work of the corresponding agent 20 when the comparison result matches both the consultation data and the prediction result according to the output of the neural network, and outputs the identity authentication result without going through an additional inspection process can make it

On the other hand, if there is a non-matching item as a result of the comparison, the inspection request may be performed through the communication unit 110 to request the inspection to the inspector to inspect the consultation data of the corresponding counselor 20 .

In addition, the learning unit 150 acquires new label information based on the inspection result data or the corresponding counseling data when it is determined that the prediction result calculated through the neural network is inaccurate as a result of the inspection by the inspector 40 and learns the labeled image The neural network can be trained with data.

As described above, according to the present invention described above, it is possible to reduce the inspection cost by reducing the number of inspection targets required for inspection by using a neural network.

In addition, the present invention is efficient because it is possible to accumulate learning data in the course of the counselor's consultation work without the process of separately producing the learning data required for learning, and the inference performance of the neural network can be further improved by setting the learning environment and the application environment the same. can

Furthermore, various embodiments described herein may be implemented in a computer-readable recording medium using, for example, software, hardware, or a combination thereof.

According to the hardware implementation, the embodiments described herein include ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays, It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions. The described embodiments may be implemented in the control module itself.

According to the software implementation, embodiments such as the procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. The software code may be implemented as a software application written in a suitable programming language. The software code may be stored in the memory module and executed by the control module.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains may make various modifications, changes and substitutions within the scope without departing from the essential characteristics of the present invention. will be.

Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are for explaining, not limiting, the technical spirit of the present invention, and the scope of the technical spirit of the present invention is not limited by these embodiments and the accompanying drawings . The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

10: user 20: agent
30: neural network 100: consultation server
110: communication unit 120: prediction result calculation unit
130: comparison unit 140: inspection request unit
150: study

Claims (11)

In a video call-based authentication monitoring method using a neural network,
receiving a face image or an identification image of the user;
receiving counseling data of a counselor for non-face-to-face authentication during the video call;
calculating a prediction result for each identification element included in the received facial image or ID image through a neural network;
comparing the calculated prediction result with the counselor's counseling data; and
A video call-based authentication monitoring method using a neural network, comprising the step of requesting inspection of the counseling data according to the comparison result. The method of claim 1,
The identification element is a predetermined item determined for non-face-to-face authentication of the user or feature information on an image divided by an item-by-item scale. 3. The method of claim 2,
The neural network is a video call-based authentication monitoring method using a neural network, characterized in that it is learned by using an image labeled based on the verified counseling data as learning data. The method of claim 1,
The counseling data is a video call-based authentication monitoring method using a neural network, characterized in that the counselor's handwritten meta information on the existence of identification elements included in the face image or the ID image. The method of claim 1,
examining the consultation data and the prediction result according to the verification request; and
When it is determined that the prediction result calculated through the neural network is inaccurate as a result of the inspection, the method further comprising the step of learning the neural network using the face image or the identification image labeled based on the counseling data as training data A video call-based authentication monitoring method using a neural network. In the counseling server for performing authentication monitoring based on video call using a neural network,
a communication unit for receiving the user's face image or ID image, and counselor's consultation data for non-face-to-face authentication during the video call;
a prediction result calculation unit for calculating a prediction result for the existence probability of each identification element included in the received facial image or ID image through a neural network;
a comparison unit comparing the calculated prediction result with the counselor's counseling data; and
Consultation server including an inspection requesting unit for requesting inspection of the consultation data according to the comparison result. 7. The method of claim 6,
The identification element is a predetermined item determined for non-face-to-face authentication of the user or feature information on an image divided by an item-by-item scale. 7. The method of claim 6,
The neural network is a counseling server, characterized in that it is learned using the image labeled based on the verified counseling data as learning data. 7. The method of claim 6,
The counseling server, characterized in that the counseling data is a counselor's handwritten meta information on the existence of identification elements included in the face image or the ID image. 7. The method of claim 6,
an inspection unit that inspects the consultation data and the prediction result according to the inspection request; and
When it is determined that the prediction result calculated through the neural network is inaccurate as a result of the inspection, a learning unit for learning the neural network using the facial image or the ID image labeled based on the consultation data as learning data It characterized in that it further comprises consulting server. A computer-readable recording medium storing a program for performing a video call-based authentication monitoring method using a neural network according to any one of claims 1 to 5.

Images