WO2021068485A1 - 多方视频的用户身份验证方法、装置及计算机设备 - Google Patents

多方视频的用户身份验证方法、装置及计算机设备 Download PDF

Info

Publication number
WO2021068485A1
WO2021068485A1 PCT/CN2020/087025 CN2020087025W WO2021068485A1 WO 2021068485 A1 WO2021068485 A1 WO 2021068485A1 CN 2020087025 W CN2020087025 W CN 2020087025W WO 2021068485 A1 WO2021068485 A1 WO 2021068485A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
information
requesting
current
feature
Prior art date
Application number
PCT/CN2020/087025
Other languages
English (en)
French (fr)
Inventor
齐燕
Original Assignee
深圳壹账通智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳壹账通智能科技有限公司 filed Critical 深圳壹账通智能科技有限公司
Publication of WO2021068485A1 publication Critical patent/WO2021068485A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes

Definitions

  • This application relates to the field of image recognition technology, and in particular to a method, device, computer equipment and storage medium for user identity verification of multi-party video.
  • the embodiments of the present application provide a method, device, computer equipment and storage medium for user identity verification of multi-party video, aiming to solve the problem that the authenticity of the identity of each participant cannot be verified when performing multi-party video in the prior art, and thus cannot be guaranteed. It is the participants themselves who come to participate in the multi-party video conference, which leads to the problem of low video security.
  • an embodiment of the present application provides a method for user identity verification of multi-party video.
  • the method includes: if a video connection consent request corresponding to the current entry number sent by the requesting terminal is detected, a corresponding video connection consent request.
  • the requester establishes a connection; performs face recognition based on the current image sent by the requester to obtain the corresponding user identification information; if it is detected that the location information acquisition instruction has been sent to the requester, the requester receives the location information sent by the server The instruction corresponds to the current positioning information pushed; it is determined whether at least one assisting end corresponding to the requesting end has not established a connection with the server within the preset first time threshold; if at least one assisting end corresponding to the requesting end has not been established with the server Connection, obtain the video information corresponding to the requesting end at the current moment, compose a video feature sequence according to the value corresponding to each information in the video information, and input the video feature sequence into a pre-trained convolutional neural network to obtain the The
  • an embodiment of the present application provides a multi-party video user identity verification device, which includes:
  • the connection establishment unit is configured to, if a video connection approval request corresponding to the current entry number sent by the requesting terminal is detected, establish a connection with the requesting terminal corresponding to the video connection approval request;
  • the identity recognition unit is used to perform face recognition according to the current image sent by the requesting end to obtain corresponding user identity recognition information
  • the positioning unit is configured to, if it is detected that the positioning information acquisition instruction has been sent to the requesting end, receive the current positioning information pushed by the requesting end according to the positioning information acquisition instruction sent by the server;
  • the connection judging unit is configured to judge whether at least one assisting end corresponding to the requesting end has not established a connection with the server within the preset first time threshold;
  • the video scene acquisition unit is configured to, if at least one assisting terminal corresponding to the requesting terminal has not established a connection with the server, acquiring the video information corresponding to the requesting terminal at the current moment, and composing the video feature according to the value corresponding to each information in the video information Sequence, input the video feature sequence to a pre-trained convolutional neural network to obtain a video scene classification result corresponding to the video feature sequence; wherein, the video information includes the time parameter corresponding to the current moment and the current location of the requester Information, background color information of the video scene; and
  • the audio data sending unit is used to obtain the background music library corresponding to the classification result of the video scene, and randomly select the audio data of one piece of music to send to the requesting terminal or the assisting terminal.
  • an embodiment of the present application provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and running on the processor, and the processor executes the computer
  • the program implements a user identity verification method.
  • the method includes, if a video connection approval request corresponding to the current entry number sent by the requesting end is detected, establishing a connection with the requesting end corresponding to the video connection approval request; Face recognition is performed on the current image to obtain the corresponding user identification information; if it is detected that the positioning information acquisition instruction has been sent to the requesting end, the requesting end receives the current positioning information corresponding to the push according to the positioning information acquisition instruction sent by the server; Whether at least one assisting end corresponding to the requesting end has not established a connection with the server within the preset first time threshold; if at least one assisting end corresponding to the requesting end has not established a connection with the server, obtain the corresponding information of the requesting end at the current moment Video information, forming a video feature sequence according to
  • the embodiments of the present application also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes a A user identity verification method for multi-party video, the method includes, if a video connection approval request corresponding to the current entry number sent by the requesting terminal is detected, establishing a connection with the requesting terminal corresponding to the video connection approval request; and according to the current image sent by the requesting terminal Perform face recognition to obtain the corresponding user identification information; if it is detected that the positioning information acquisition instruction has been sent to the requesting end, the requesting end will receive the current positioning information pushed corresponding to the positioning information acquisition instruction sent by the server; Within the first time threshold of whether there is at least one assisting end corresponding to the requesting end that has not established a connection with the server; if at least one assisting end corresponding to the requesting end has not established a connection with the server, obtain the video information corresponding to the requesting end at the current moment
  • the embodiments of the present application provide a user identity verification method, device, computer equipment, and storage medium for multi-party video. Real-time verification of the authenticity of the participants’ identities when performing multi-party video, ensures that the participants themselves participate in the multi-party video conference, and can also randomly play the background music during the waiting period according to the video scene classification results during the video waiting period, which improves the video performance Data security.
  • FIG. 1 is a schematic diagram of an application scenario of a user identity verification method for a multi-party video provided by an embodiment of the application;
  • FIG. 2 is a schematic flowchart of a user identity verification method for a multi-party video provided by an embodiment of the application
  • FIG. 3 is a schematic diagram of another process of a method for user identity verification of a multi-party video provided by an embodiment of the application;
  • FIG. 4 is a schematic diagram of the display area distribution of the user interaction interface of the server in the user identity verification method for multi-party video provided by an embodiment of the application;
  • FIG. 5 is a schematic diagram of a sub-flow of a method for user identity verification of a multi-party video provided by an embodiment of the application;
  • FIG. 6 is a schematic diagram of another sub-flow of the method for user identity verification of multi-party video provided by an embodiment of the application;
  • FIG. 7 is a schematic block diagram of a user identity verification device for multi-party video provided by an embodiment of the application.
  • FIG. 8 is another schematic block diagram of a user identity verification device for multi-party video provided by an embodiment of the application.
  • FIG. 9 is a schematic block diagram of a subunit of the user identity verification device for multi-party video provided by an embodiment of the application.
  • FIG. 10 is a schematic block diagram of another subunit of the user identity verification device for multi-party video provided by an embodiment of the application.
  • FIG. 11 is a schematic block diagram of a computer device provided by an embodiment of the application.
  • Figure 1 is a schematic diagram of an application scenario of a multiparty video user identity verification method provided by an embodiment of this application
  • Figure 2 is a schematic flowchart of a multiparty video user identity verification method provided by an embodiment of this application.
  • the user identity verification method of multi-party video is applied to the server, and the method is executed by application software installed in the server.
  • the method includes steps S110 to S160.
  • the first is the server, which corresponds to the terminal operated by the reviewer (such as a desktop computer), which is used to receive the video data of the requesting end and/or the assisting end in a multi-party video scenario, and to authenticate the corresponding users of the requesting end and/or the assisting end, And according to the video data, the background music is automatically generated during the waiting period of the multi-party video.
  • the reviewer such as a desktop computer
  • the second is the requester, which corresponds to the terminal (such as a smart phone or a Ping An computer) operated by the requester (also known as the applicant), which is used to send the requester's application information to the server, and can send the requester's real-time video Wait for the data to be sent to the server.
  • the terminal such as a smart phone or a Ping An computer operated by the requester (also known as the applicant) operated by the requester (also known as the applicant), which is used to send the requester's application information to the server, and can send the requester's real-time video Wait for the data to be sent to the server.
  • the third is the assistance terminal, which corresponds to the terminal (such as a smart phone or a safe computer) set by the requester to assist the personnel to send the identity information of the assistance personnel to the server, and can send the assistance personnel’s real-time video data to the server server.
  • the terminal such as a smart phone or a safe computer
  • the server When the server detects the current entry number entered by the operator, it enters the waiting state of waiting for the requester to access. Specifically, after the server detects the current entry number entered by the operator, the server sends a video connection request to the requesting end corresponding to the current entry number, and the requesting end sends a video connection approval request to the server according to the video connection request. If a video connection approval request corresponding to the current entry number sent by the requesting terminal is detected, a connection is established with the requesting terminal corresponding to the video connection approval request, and the server performs video communication with the requesting terminal at this time.
  • the specific use scenario of this application is set as the multi-person video interview scenario of the credit business
  • 1-2 credit enhancers such as the applicant’s spouse or Friends, these credit enhancers use the assistant terminal to participate in the multi-person video interview) to assist in participating in the video interview.
  • application information including loan application information, borrower information, mortgage information, and other loan information
  • an order and a unique order number corresponding to the order are generated, and the order number will be stored in the server.
  • the server If the server does not establish a connection with the requester within the preset response time threshold (such as any time value within 20-30s), the server automatically sends a prompt message that "customer service is not online" to the requester. If the server establishes a connection with the requesting end within the response time threshold, the server conducts video communication with the requesting end.
  • the preset response time threshold such as any time value within 20-30s
  • the method further includes:
  • S112 Obtain assistance terminal information according to the application information corresponding to the video connection request;
  • the applicant’s video data corresponding to the requester is displayed in the upper left corner area of the user interaction interface as shown in Figure 4 (ie, the applicant’s display area).
  • the video data of the reviewer is displayed in the lower right corner of the user interaction interface as shown in FIG. 4 (that is, the reviewer display area).
  • the upper right and lower left corners of the user interaction interface are the credit enhancer display areas, that is, when the assisting user participates in the multi-faceted review, it is displayed in the upper right and lower left corners of the user interaction interface for the credit enhancer display Area (for example, the upper right corner of the user interaction interface is the display area of Credit Enhancer 1, and the lower left corner is the display area of Credit Enhancer 2).
  • the user interaction interface is also provided with an application information display area for displaying application information.
  • the credit enhancer corresponding to the assisting end needs to be notified in time to go online in time to participate in the multi-party video.
  • the server in order for the server to accurately notify the assisting end, it needs to first obtain the assisting end information (mainly to obtain the telephone number or user account information of the assisting end) according to the application information uploaded by the requesting end corresponding to the video connection request, and then the server according to the assisting end The information sends the assistance video connection request to the corresponding assistance terminal. In this way, when the applicant corresponding to the requesting terminal and the reviewer corresponding to the server are both online, the assistant terminal will be notified to go online in time.
  • S120 Perform face recognition according to the current image sent by the requesting end to obtain corresponding user identification information.
  • step S120 includes:
  • the server sends a face recognition request to the requester.
  • the requester sends a face recognition consent request to the server.
  • the server detects the face recognition consent request corresponding to the requesting end, it acquires the current image at the time corresponding to the face recognition approval request (that is, the requesting end collects the current image and uploads it to the server for face recognition). Then compare the feature vector corresponding to the current image with the feature template stored in the face database, if the feature template stored in the face database contains the same feature as the picture feature vector corresponding to the current image Template to obtain the corresponding user identification information.
  • the method further includes:
  • S1222 Obtain a picture feature vector corresponding to the preprocessed picture through a convolutional neural network model.
  • the image preprocessing of the face is based on the face detection result, the image is processed and finally serves the process of feature extraction. Due to various conditions and random interference, the original image obtained by the server cannot be used directly. It must be pre-processed in the early stage of image processing such as gray-scale correction and noise filtering.
  • the preprocessing process mainly includes light compensation, gray scale transformation, histogram equalization, normalization, geometric correction, filtering and sharpening of the face image.
  • the feature vector of a picture When obtaining the feature vector of a picture, first obtain the pixel matrix corresponding to the preprocessed picture, and then use the pixel matrix corresponding to the preprocessed picture as the input of the input layer in the convolutional neural network model to obtain multiple feature maps.
  • the feature map is input into the pooling layer, and the one-dimensional row vector corresponding to the maximum value corresponding to each feature map is obtained. Finally, the one-dimensional row vector corresponding to the maximum value corresponding to each feature map is input to the fully connected layer, and the obtained and preprocessed The image feature vector corresponding to the image.
  • the feature templates stored in the face database store the feature vectors corresponding to the massive amount of face images that have been collected, that is, each person’s face corresponds to a unique feature vector. With these massive feature templates as data After the foundation, it can be used to determine one or more people corresponding to the preprocessed picture, so as to realize face recognition.
  • the obtained user identification information can be the user's ID number. Since each citizen's ID number is unique, it can be used as its unique identification code. When the user identity information of the applicant is consistent with the corresponding user identity information in the application information, it can be ensured that the applicant is himself participating in the multi-party video.
  • the server when the server needs to obtain the positioning information of the requesting end, the server first triggers a positioning information obtaining instruction, and then the server sends the positioning information obtaining instruction to the requesting end, and the requesting end sends the positioning information to the requesting end after obtaining the current positioning information.
  • the current location information is sent to the server, and finally the server receives the current location information sent by the requester, that is, the auditor can monitor the applicant's location information in real time, that is, the address information included in the application information provided by the applicant can be verified again Is it wrong.
  • the current location information sent by the requesting terminal is displayed in the location information display area in the user interaction interface as shown in FIG. 4.
  • S140 Determine whether at least one assisting terminal corresponding to the requesting terminal has not established a connection with the server within a preset first time threshold.
  • the server after the server successfully establishes a connection with the requesting end, the server sends an online request of the assisting end to one or more assisting ends corresponding to the requesting end.
  • the assisting terminal responds to the assisting terminal's online request and goes online in time at the first time threshold (for example, setting the first time threshold to 5-10s), all the assisting terminals go online in time to participate in multi-party video.
  • the requesting terminal and the server The process of waiting for the assistance terminal to go online is relatively short, and there is no need to wait for processing.
  • the assistant terminal corresponding to the requesting terminal does not establish a connection with the server, and the video data corresponding to the assistant terminal is displayed in the credit enhancer display area preset in FIG. 4.
  • the current location information corresponding to the assistance terminal can also be displayed in the location information display area set in Figure 4.
  • the information display area displays the geographic location locations of the applicant, credit enhancer 1 and credit enhancer 2 in the form of points in the electronic map, so as to achieve the display effect of the geographic distribution map of personnel and intuitively display the current geographic location of each personnel.
  • At least one assisting terminal corresponding to the requesting terminal has not established a connection with the server, obtain the video information corresponding to the requesting terminal at the current moment, form a video feature sequence according to the values corresponding to each information in the video information, and combine the The video feature sequence is input to the pre-trained convolutional neural network to obtain the video scene classification result corresponding to the video feature sequence; wherein, the video information includes the time parameter corresponding to the current moment, the current positioning information of the requesting end, and the video scene classification result. Background color information.
  • the applicant and the assisting personnel are required to participate in the multi-party video interview.
  • the server detects that after the requesting end is successfully connected, there is still at least one assisting end that is not connected to the server, it can wait for the connection In the interval, the server automatically obtains the video scene classification result according to the corresponding video information of the requesting end at the current moment, and the video scene classification result determines a waiting music to be played as the background music during the connection between the requesting end and the server waiting for the assisting end.
  • step S150 includes:
  • the current time is 12 o'clock
  • the latitude and longitude corresponding to the current positioning information is in the East 8
  • the background color information generally includes three parameter values of R, G, and B
  • the RGB parameter value is divided by 256 to obtain a third value sequence composed of three values;
  • the RGB parameter value of the background color information is (128, 128, 128,), and the third value sequence is 128/256 128/256 128/256, that is, 0, 5 0, 5 0, 5.
  • the video feature sequence is obtained as [1/2 1/3 1/2 1/2 1/2].
  • the method before step S150, the method further includes:
  • the video scene classification result corresponding to each video feature sequence in the training set is pre-labeled.
  • the video scene classification result can be labeled with a value of 1-10, where 1 represents a cheerful scene, 2 represents a serious scene, and so on.
  • the video scene classification result corresponding to the video feature sequence is obtained, it is necessary to randomly obtain a piece of music from the background music library corresponding to the video scene classification result and send it to the requesting end as the currently pushed audio file. Or assist the end.
  • the video scene classification result is 1 (representing a cheerful scene)
  • the corresponding background music library is Music Library 1 (the cheerful style music stored in it)
  • the audio data of one of the music is randomly selected from the music library 1 and sent To the requesting end or the assisting end, as the waiting music of the requesting or assisting end.
  • step S160 the method further includes:
  • a prompt message for enabling multi-party video is sent to the requesting end and the assisting end.
  • the applicant at the requesting end or the assisting personnel corresponding to at least one of the assisting ends connected to the server has listened to the audio for a duration equal to the second time threshold (for example, set to 30-120 seconds).
  • start the multi-party video interview as soon as possible. You can send the prompt message to start the multi-video interview to the requester and the assisting end of the connected server, prompting to start the video conference of the multi-party video interview in the absence of an assisting end to reduce waiting time .
  • step S160 the method further includes:
  • the target video data of the preset duration corresponding to the requesting end is acquired, and the target video data is preprocessed by the streamer method to obtain the target picture set corresponding to the target video data.
  • the server clicks the "Emotion Detection" button on the user interaction interface as shown in Figure 4 it can obtain and request The end corresponds to the target video data of the preset duration.
  • the applicant’s micro-expression is identified by the optical flow method to determine whether there is fraud.
  • optical flow expresses the change of the image, contains the information of the target's movement, and can be used to determine the target's movement.
  • the three elements of optical flow one is the motion velocity field, which is a necessary condition for the formation of optical flow; the second is the part with optical characteristics such as gray-scale pixels, which can carry motion information; the third is the imaging projection from the scene to the The image plane can thus be observed.
  • optical flow is based on points. Specifically, let (u, v) be the optical flow of the image point (x, y), then (x, y, u, v) is called the optical flow point.
  • the collection of all optical flow points is called the optical flow field.
  • a corresponding image motion field, or image velocity field is formed on the image plane.
  • the optical flow field corresponds to the sports field.
  • the image can be dynamically analyzed. If there is no moving target in the image, the optical flow vector changes continuously throughout the image area. When there is a moving object in the image (when the user has a micro-expression, the face will move, which is equivalent to a moving object), there is relative movement between the target and the background. The velocity vector formed by the moving object must be different from the background velocity vector, so that the position of the moving object can be calculated.
  • Preprocessing is performed by the optical flow method to obtain a target picture set corresponding to the target video data. The optical flow method is used for preprocessing, and a target picture set composed of pictures with micro-expressions in the target video data is obtained.
  • the target picture set can be pushed to the corresponding receiving end (this receiving end can be another cloud server for micro-expression detection, or it can be a sub-module for micro-expression detection set in the server) Perform micro-expression analysis to determine whether the applicant is likely to be fraudulent in the process of video communication with the server.
  • This method realizes the real-time verification of the authenticity of the participants’ identities when performing multi-party video, ensuring that the participants themselves participate in the multi-party video conference, and can also randomly play the background music during the waiting period according to the video scene classification results during the video waiting period. Data security of the video.
  • FIG. 7 is a schematic block diagram of a user identity verification apparatus for a multi-party video provided in an embodiment of the present application.
  • the user identity verification device 100 of the multi-party video may be configured in a server.
  • the user identity verification device 100 for multi-party video includes a connection establishment unit 110, an identity recognition unit 120, a positioning unit 130, a connection judgment unit 140, a video scene acquisition unit 150, and an audio data sending unit 160.
  • the connection establishment unit 110 is configured to, if a video connection approval request corresponding to the current entry number sent by the requesting terminal is detected, establish a connection with the requesting terminal corresponding to the video connection approval request.
  • the server when the server detects the current entry number entered by the operator, it enters the waiting state of waiting for the requesting end to access. Specifically, after the server detects the current entry number entered by the operator, the server sends a video connection request to the requesting end corresponding to the current entry number, and the requesting end sends a video connection approval request to the server according to the video connection request. If a video connection approval request corresponding to the current entry number sent by the requesting terminal is detected, a connection is established with the requesting terminal corresponding to the video connection approval request, and the server performs video communication with the requesting terminal at this time.
  • the specific use scenario of this application is set as the multi-person video interview scenario of the credit business
  • 1-2 credit enhancers such as the applicant’s spouse or Friends, these credit enhancers use the assistant terminal to participate in the multi-person video interview) to assist in participating in the video interview.
  • application information including loan application information, borrower information, mortgage information, and other loan information
  • an order and a unique order number corresponding to the order are generated, and the order number will be stored in the server.
  • the server If the server does not establish a connection with the requester within the preset response time threshold (such as any time value within 20-30s), the server automatically sends a prompt message that "customer service is not online" to the requester. If the server establishes a connection with the requesting end within the response time threshold, the server conducts video communication with the requesting end.
  • the preset response time threshold such as any time value within 20-30s
  • the user identity verification apparatus 100 for multi-party video further includes:
  • Applicant video display unit 111 configured to display the video data corresponding to the requesting terminal in a preset applicant display area
  • the assistance terminal information obtaining unit 112 is configured to obtain assistance terminal information according to the application information corresponding to the video connection request;
  • the assistance terminal connection sending unit 113 is configured to send an assistance video connection request to the assistance terminal corresponding to the assistance terminal information.
  • the applicant’s video data corresponding to the requester is displayed in the upper left corner area of the user interaction interface as shown in Figure 4 (ie, the applicant’s display area).
  • the video data of the reviewer is displayed in the lower right corner of the user interaction interface as shown in FIG. 4 (that is, the reviewer display area).
  • the upper right and lower left corners of the user interaction interface are the credit enhancer display areas, that is, when the assisting user participates in the multi-faceted review, it is displayed in the upper right and lower left corners of the user interaction interface for the credit enhancer display Area (for example, the upper right corner of the user interaction interface is the display area of Credit Enhancer 1, and the lower left corner is the display area of Credit Enhancer 2).
  • the user interaction interface is also provided with an application information display area for displaying application information.
  • the credit enhancer corresponding to the assisting end needs to be notified in time to go online in time to participate in the multi-party video.
  • the server in order for the server to accurately notify the assisting end, it needs to first obtain the assisting end information (mainly to obtain the telephone number or user account information of the assisting end) according to the application information uploaded by the requesting end corresponding to the video connection request, and then the server according to the assisting end The information sends the assistance video connection request to the corresponding assistance terminal. In this way, when the applicant corresponding to the requesting terminal and the reviewer corresponding to the server are both online, the assistant terminal will be notified to go online in time.
  • the identity recognition unit 120 is configured to perform face recognition according to the current image sent by the requesting end to obtain corresponding user identity recognition information.
  • the identity recognition unit 120 includes:
  • the current image acquisition unit 121 is configured to, if a face recognition consent request corresponding to the requesting end is detected, acquire a current image at the time corresponding to the face recognition consent request;
  • the comparison unit 122 is configured to compare the feature vector corresponding to the current image with the feature template stored in the face database to determine whether there is a feature template stored in the face database corresponding to the current image Feature templates with the same feature vector of the pictures;
  • the first processing unit 123 is configured to obtain corresponding user identification information if there is a feature template that is the same as the image feature vector corresponding to the current image among the feature templates stored in the face database;
  • the second processing unit 124 is configured to, if there is no feature template that is the same as the feature vector of the picture corresponding to the current image among the feature templates stored in the face database, perform a prompt to add the current user identification information.
  • the server sends a face recognition request to the requester.
  • the requester sends a face recognition consent request to the server.
  • the server detects the face recognition consent request corresponding to the requesting end, it acquires the current image at the time corresponding to the face recognition approval request (that is, the requesting end collects the current image and uploads it to the server for face recognition). Then compare the feature vector corresponding to the current image with the feature template stored in the face database, if the feature template stored in the face database contains the same feature as the picture feature vector corresponding to the current image Template to obtain the corresponding user identification information.
  • the identity recognition unit 120 further includes:
  • the preprocessing unit 1221 is configured to perform grayscale correction and noise filtering on the current image to obtain a preprocessed picture
  • the feature vector obtaining unit 1222 is configured to obtain a picture feature vector corresponding to the preprocessed picture through a convolutional neural network model.
  • the image preprocessing of the face is based on the face detection result, the image is processed and finally serves the process of feature extraction. Due to various conditions and random interference, the original image obtained by the server cannot be used directly. It must be pre-processed in the early stage of image processing such as gray-scale correction and noise filtering.
  • the preprocessing process mainly includes light compensation, gray scale transformation, histogram equalization, normalization, geometric correction, filtering and sharpening of the face image.
  • the feature vector of a picture When obtaining the feature vector of a picture, first obtain the pixel matrix corresponding to the preprocessed picture, and then use the pixel matrix corresponding to the preprocessed picture as the input of the input layer in the convolutional neural network model to obtain multiple feature maps.
  • the feature map is input into the pooling layer, and the one-dimensional row vector corresponding to the maximum value corresponding to each feature map is obtained. Finally, the one-dimensional row vector corresponding to the maximum value corresponding to each feature map is input to the fully connected layer, and the obtained and preprocessed The image feature vector corresponding to the image.
  • the feature templates stored in the face database store the feature vectors corresponding to the massive amount of face images that have been collected, that is, each person’s face corresponds to a unique feature vector. With these massive feature templates as data After the foundation, it can be used to determine one or more people corresponding to the preprocessed picture, so as to realize face recognition.
  • the obtained user identification information can be the user's ID number. Since each citizen's ID number is unique, it can be used as its unique identification code. When the user identity information of the applicant is consistent with the corresponding user identity information in the application information, it can be ensured that the applicant is himself participating in the multi-party video.
  • the positioning unit 130 is configured to, if it is detected that the positioning information obtaining instruction has been sent to the requesting end, receive the current positioning information pushed by the requesting end according to the positioning information obtaining instruction sent by the server.
  • the server when the server needs to obtain the positioning information of the requesting end, the server first triggers a positioning information obtaining instruction, and then the server sends the positioning information obtaining instruction to the requesting end, and the requesting end sends the positioning information to the requesting end after obtaining the current positioning information.
  • the current location information is sent to the server, and finally the server receives the current location information sent by the requester, that is, the auditor can monitor the applicant's location information in real time, that is, the address information included in the application information provided by the applicant can be verified again Is it wrong.
  • the current location information sent by the requesting terminal is displayed in the location information display area in the user interaction interface as shown in FIG. 4.
  • the connection determining unit 140 is configured to determine whether at least one assisting terminal corresponding to the requesting terminal has not established a connection with the server within the preset first time threshold.
  • the server after the server successfully establishes a connection with the requesting end, the server sends an online request of the assisting end to one or more assisting ends corresponding to the requesting end.
  • the assisting terminal responds to the assisting terminal's online request and goes online in time at the first time threshold (for example, setting the first time threshold to 5-10s), all the assisting terminals go online in time to participate in multi-party video.
  • the requesting terminal and the server The process of waiting for the assistance terminal to go online is relatively short, and there is no need to wait for processing.
  • the assistant terminal corresponding to the requesting terminal does not establish a connection with the server, and the video data corresponding to the assistant terminal is displayed in the credit enhancer display area preset in FIG. 4.
  • the current location information corresponding to the assistance terminal can also be displayed in the location information display area set in Figure 4.
  • the information display area displays the geographic location locations of the applicant, credit enhancer 1 and credit enhancer 2 in the form of points in the electronic map, so as to achieve the display effect of the geographic distribution map of personnel and intuitively display the current geographic location of each personnel.
  • the video scene acquiring unit 150 is configured to, if at least one assisting terminal corresponding to the requesting terminal has not established a connection with the server, acquiring the video information corresponding to the requesting terminal at the current moment, and composing the video according to the value corresponding to each information in the video information Feature sequence, input the video feature sequence to a pre-trained convolutional neural network to obtain a video scene classification result corresponding to the video feature sequence; wherein, the video information includes the time parameter corresponding to the current moment, and the current Positioning information, background color information of the video scene.
  • the applicant and the assisting personnel are required to participate in the multi-party video interview.
  • the server detects that after the requesting end is successfully connected, there is still at least one assisting end that is not connected to the server, it can wait for the connection In the interval, the server automatically obtains the video scene classification result according to the corresponding video information of the requesting end at the current moment, and the video scene classification result determines a waiting music to be played as the background music during the connection between the requesting end and the server waiting for the assisting end.
  • the video scene acquiring unit 150 includes:
  • the first value calculation unit 151 is configured to obtain the time parameter corresponding to the current moment in the video information, divide the time parameter by 24 for normalization, and obtain the first value;
  • the second value calculation unit 152 is configured to obtain the current location information in the video information, obtain a time zone number corresponding to the current location information, and divide the time zone number by 24 for normalization to obtain the second value. value;
  • the third value sequence obtaining unit 153 is configured to obtain background color information of the video scene in the video information, obtain RGB parameter values corresponding to the background color information, and divide the RGB parameter values by 256 to obtain the first Three-value sequence;
  • the value concatenation unit 154 is configured to concatenate the first value, second value, and third value sequence to obtain a video feature sequence.
  • the current time is 12 o'clock
  • the latitude and longitude corresponding to the current positioning information is in the East 8
  • the background color information generally includes three parameter values of R, G, and B
  • the RGB parameter value is divided by 256 to obtain a third value sequence composed of three values;
  • the RGB parameter value of the background color information is (128, 128, 128,), and the third value sequence is 128/256 128/256 128/256, that is, 0, 5 0, 5 0, 5.
  • the video feature sequence is obtained as [1/2 1/3 1/2 1/2 1/2].
  • the user identity verification device 100 for multi-party video further includes:
  • the model training unit is used to take each video feature sequence in the training set as the input of the convolutional neural network to be trained, and use the corresponding video scene classification result as the output of the convolutional neural network to be trained, and to the convolutional neural network to be trained Perform training to obtain a convolutional neural network for classifying video scenes;
  • the video scene classification result corresponding to each video feature sequence in the training set is pre-labeled.
  • the video scene classification result can be labeled with a value of 1-10, where 1 represents a cheerful scene, 2 represents a serious scene, and so on.
  • the video scene acquiring unit 160 is configured to acquire a background music library corresponding to the video scene classification result, and randomly select one piece of music audio data to send to the requesting terminal or the assisting terminal.
  • the video scene classification result corresponding to the video feature sequence is obtained, it is necessary to randomly obtain a piece of music from the background music library corresponding to the video scene classification result and send it to the requesting end as the currently pushed audio file. Or assist the end.
  • the video scene classification result is 1 (representing a cheerful scene)
  • the corresponding background music library is Music Library 1 (the cheerful style music stored in it)
  • the audio data of one of the music is randomly selected from the music library 1 and sent To the requesting end or the assisting end, as the waiting music of the requesting or assisting end.
  • the user identity verification device 100 for multi-party video further includes:
  • connection prompt unit is configured to send a prompt message for enabling multi-party video to the requesting end or the assisting end if the interval between the sending time of the audio data to the requesting end or the assisting end and the current system time exceeds the preset second time threshold. end.
  • the applicant at the requesting end or the assisting personnel corresponding to at least one of the assisting ends connected to the server has listened to the audio for a duration equal to the second time threshold (for example, set to 30-120 seconds).
  • start the multi-party video interview as soon as possible. You can send the prompt message to start the multi-video interview to the requester and the assisting end of the connected server, prompting to start the video conference of the multi-party video interview in the absence of an assisting end to reduce waiting time .
  • the user identity verification device 100 for multi-party video further includes:
  • the micro-expression recognition unit is used to obtain the target video data of the preset duration corresponding to the requesting end if the current video data acquisition instruction is detected, and preprocess the target video data through the streamer method to obtain the target video data.
  • the server clicks the "Emotion Detection" button on the user interaction interface as shown in Figure 4 it can obtain and request The end corresponds to the target video data of the preset duration.
  • the applicant’s micro-expression is identified by the optical flow method to determine whether there is fraud.
  • optical flow expresses the change of the image, contains the information of the target's movement, and can be used to determine the target's movement.
  • the three elements of optical flow one is the motion velocity field, which is a necessary condition for the formation of optical flow; the second is the part with optical characteristics such as gray-scale pixels, which can carry motion information; the third is the imaging projection from the scene to the The image plane can thus be observed.
  • optical flow is based on points. Specifically, let (u, v) be the optical flow of the image point (x, y), then (x, y, u, v) is called the optical flow point.
  • the collection of all optical flow points is called the optical flow field.
  • a corresponding image motion field, or image velocity field is formed on the image plane.
  • the optical flow field corresponds to the sports field.
  • the image can be dynamically analyzed. If there is no moving target in the image, the optical flow vector changes continuously throughout the image area. When there is a moving object in the image (when the user has a micro-expression, the face will move, which is equivalent to a moving object), there is relative movement between the target and the background. The velocity vector formed by the moving object must be different from the background velocity vector, so that the position of the moving object can be calculated.
  • Preprocessing is performed by the optical flow method to obtain a target picture set corresponding to the target video data. The optical flow method is used for preprocessing, and a target picture set composed of pictures with micro-expressions in the target video data is obtained.
  • the target picture set can be pushed to the corresponding receiving end (this receiving end can be another cloud server for micro-expression detection, or it can be a sub-module for micro-expression detection set in the server) Perform micro-expression analysis to determine whether the applicant is likely to be fraudulent in the process of video communication with the server.
  • the device realizes real-time verification of the authenticity of the participants’ identities during multi-party video, ensuring that the participants themselves participate in the multi-party video conference, and can also randomly play the background music during the waiting period according to the video scene classification results during the video waiting period, which improves Data security of the video.
  • the above-mentioned multi-party video user identity verification device can be implemented in the form of a computer program, and the computer program can be run on a computer device as shown in FIG. 11.
  • FIG. 11 is a schematic block diagram of a computer device according to an embodiment of the present application.
  • the computer device 500 is a server, and the server may be an independent server or a server cluster composed of multiple servers.
  • the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
  • the non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032.
  • the computer program 5032 When executed, it can make the processor 502 execute the method of user authentication for multi-party video.
  • the processor 502 is used to provide computing and control capabilities, and support the operation of the entire computer device 500.
  • the internal memory 504 provides an environment for the running of the computer program 5032 in the non-volatile storage medium 503.
  • the processor 502 can make the processor 502 execute the method of user authentication for multi-party video.
  • the network interface 505 is used for network communication, such as providing data information transmission.
  • the structure shown in FIG. 11 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied.
  • the specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.
  • the processor 502 is configured to run a computer program 5032 stored in a memory, so as to implement the user identity verification method for multi-party video disclosed in the embodiment of the present application.
  • the embodiment of the computer device shown in FIG. 11 does not constitute a limitation on the specific configuration of the computer device.
  • the computer device may include more or less components than those shown in the figure. Or some parts are combined, or different parts are arranged.
  • the computer device may only include a memory and a processor. In such an embodiment, the structures and functions of the memory and the processor are consistent with the embodiment shown in FIG. 11, and will not be repeated here.
  • the processor 502 may be a central processing unit (Central Processing Unit, CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), and special purpose processors.
  • Integrated circuit Application Specific Integrated Circuit, ASIC
  • ready-made programmable gate array Field-Programmable Gate Array, FPGA
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor.
  • a computer-readable storage medium may be non-volatile or may be a volatile computer-readable storage medium.
  • the computer-readable storage medium stores a computer program, where the computer program, when executed by a processor, implements the user identity verification method for multi-party videos disclosed in the embodiments of the present application.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium.
  • the technical solution of this application is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. It includes several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), magnetic disk or optical disk and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请公开了多方视频的用户身份验证方法、装置、计算机设备及存储介质。该方法包括与视频连接同意请求对应的请求端建立连接；根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息；若有至少一个与请求端对应的协助端未与服务器建立连接，获取请求端在当前时刻对应的视频信息，根据视频信息中各信息对应的取值组成视频特征序列，将其输入至卷积神经网络得到对应的视频场景分类结果；获取与其对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端。该方法实现了进行多方视频时实时验证参与方的身份真实性，还能在视频等待期间根据视频场景分类结果对应随机播放等待期间的音乐，提高视频的数据安全性。

Description

多方视频的用户身份验证方法、装置及计算机设备
本申请要求于2019年10月12日提交中国专利局、申请号为201910968909.9，发明名称为“多方视频的用户身份验证方法、装置及计算机设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像识别技术领域，尤其涉及一种多方视频的用户身份验证方法、装置、计算机设备及存储介质。
背景技术
目前，在进行多方视频的过程中，一般是多人同时参与视频，在一方发起视频请求邀请多方参与者进行视频时，受邀的参与者在操作终端接收视频请求时，可以是本人接收视频请求，也可以是他人代替本人来接收视频请求，发明人意识到这就导致一方发起视频请求时，无法确保受邀的各参与者是本人参与视频。即在多方视频时是无法验证各参与方的身份真实性，也就无法确保是参与者本人来参与多方视频会议，导致视频的安全性低下。
发明内容
本申请实施例提供了一种多方视频的用户身份验证方法、装置、计算机设备及存储介质，旨在解决现有技术中进行多方视频时是无法验证各参与方的身份真实性，也就无法确保是参与者本人来参与多方视频会议，导致视频的安全性低下的问题。
第一方面，本申请实施例提供了一种多方视频的用户身份验证方法，该方法包括若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接；根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息；若检测到已将定位信息获取指令发送至请求端，接收请求端根据服务器所发送的定位信息获取指令对应推送的当前定位信息；判断在预设的第一时间阈值内是否有至少一个与请求端对应的协助端未与服务器建立连接；若有至少一个与请求端对应的协助端未与服务器建立连接，获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，将所述视频特征序列输入至预先训练的卷积神经网络，得到与所述视频特征序列对应的视频场景分类结果；获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端。
第二方面，本申请实施例提供了一种多方视频的用户身份验证装置，其包括：
连接建立单元，用于若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接；
身份识别单元，用于根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息；
定位单元，用于若检测到已将定位信息获取指令发送至请求端，接收请求端根据服务器所发送的定位信息获取指令对应推送的当前定位信息；
连接判断单元，用于判断在预设的第一时间阈值内是否有至少一个与请求端对应的协助端未与服务器建立连接；
视频场景获取单元，用于若有至少一个与请求端对应的协助端未与服务器建立连接，获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，将所述视频特征序列输入至预先训练的卷积神经网络，得到与所述视频特征序列对应的视频场景分类结果；其中，所述视频信息包括当前时刻对应的时间参数、请求端的当前定位信息、视频场景的背景色信息；以及
音频数据发送单元，用于获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端。
第三方面，本申请实施例又提供了一种计算机设备，其包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现一种用户身份验证方法，该方法包括若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接；根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息；若检测到已将定位信息获取指令发送至请求端，接收请求端根据服务器所发送的定位信息获取指令对应推送的当前定位信息；判断在预设的第一时间阈值内是否有至少一个与请求端对应的协助端未与服务器建立连接；若有至少一个与请求端对应的协助端未与服务器建立连接，获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，将所述视频特征序列输入至预先训练的卷积神经网络，得到与所述视频特征序列对应的视频场景分类结果；获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端。。
第四方面，本申请实施例还提供了一种计算机可读存储介质，其中所述计算机可读存储介质存储有计算机程序，所述计算机程序当被处理器执行时使所述处理器执行一种多方视频的用户身份验证方法，该方法包括若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接；根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息；若检测到已将定位信息获取指令发送至请求端，接收请求端根据服务器所发送的定位信息获取指令对应推送的当前定位信息；判断在预设的第一时间阈值内是否有至少一个与请求端对应的协助端未与服务器建立连接；若有至少一个与请求端对应的协助端未与服务器建立连接，获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，将所述视频特征序列输入至预先训练的卷积神经网络，得到与所述视频特征序列对应的视频场景分类结果；获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端。
本申请实施例提供了一种多方视频的用户身份验证方法、装置、计算机设备及存储介质。实现了进行多方视频时实时验证参与方的身份真实性，确保是参与者本人来参与多方视频会议，还能在视频等待期间根据视频场景分类结果对应随机播放等待期间的背景音乐，提高了视频的数据安全性。
附图说明
为了更清楚地说明本申请实施例技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的多方视频的用户身份验证方法的应用场景示意图；
图2为本申请实施例提供的多方视频的用户身份验证方法的流程示意图；
图3为本申请实施例提供的多方视频的用户身份验证方法的另一流程示意图；
图4为本申请实施例提供的多方视频的用户身份验证方法中服务器的用户交互界面的显示区域分布示意图；
图5为本申请实施例提供的多方视频的用户身份验证方法的子流程示意图；
图6为本申请实施例提供的多方视频的用户身份验证方法的另一子流程示意图；
图7为本申请实施例提供的多方视频的用户身份验证装置的示意性框图；
图8为本申请实施例提供的多方视频的用户身份验证装置的另一示意性框图；
图9为本申请实施例提供的多方视频的用户身份验证装置的子单元示意性框图；
图10为本申请实施例提供的多方视频的用户身份验证装置的另一子单元示意性框图；
图11为本申请实施例提供的计算机设备的示意性框图。
具体实施方式
请参阅图1和图2，图1为本申请实施例提供的多方视频的用户身份验证方法的应用场景示意图；图2为本申请实施例提供的多方视频的用户身份验证方法的流程示意图，该多方视频的用户身份验证方法应用于服务器中，该方法通过安装于服务器中的应用软件进行执行。
如图2所示，该方法包括步骤S110～S160。
S110、若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接。
在本实施例中，为了更清楚的理解本申请的技术方案，下面对具体应用场景及所涉及的终端进行介绍。本申请是站在服务器的角度描述技术方案。
一是服务器，对应审核人操作的终端(如台式电脑)，用于在多方视频的场景下接收请求端和/或协助端的视频数据，对请求端和/或协助端对应的用户进行身份验证，并根据视频数据在多方视频的等待期间自动生成背景配乐。
二是请求端，对应的是请求人(也可以理解为申请人)操作的终端(如智能手机或平安电脑)，用于将请求人的申请信息发送至服务器，并可将请求人的实时视频等数据发送至服务器。
三是协助端，对应的是请求人所设置的协助人员操作的终端(如智能手机或平安电脑)，用于将协助人员的身份信息发送至服务器，并可将协助人员的实时视频数据发送至服务器。
当服务器检测到操作人员所录入的当前录入编号时，则进入等待请求端接入的等待状态。具体的，当服务器检测到操作人员所录入的当前录入编号后，服务器将视频连接请求发送至与当前录入编号对应的请求端，请求端根据视频连接请求对应发送视频连接同意请求至服务器。若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接，此时服务器与所述请求端进行视频通讯。
例如，将本申请的具体使用场景设置为信贷业务的多人视频面审场景，申请人请求信贷业务提供方进行视频面审之前，可以预先设置1-2个增信人(如申请人的配偶或好友，这些增信人使用协助端参与多人视频面审)以辅助参与视频面审。当申请人完成了申请信息(包括贷款申请信息、借款人信息、抵押信息及其他贷款信息)的编辑后，生成订单及与订单对应的唯一的订单编号，该订单编号会存储在服务器中。
当服务器对应的审核人此时需与申请人建立连接时，表示审核人已做好多方视频面审的准备工作，此时可以输入当前时刻可进行面审的订单编号。一旦审核人在服务器中完成订单编号的输入，则申请人对应的请求端的用户交互界面中更新一个“参与面审”的按钮。当申请人在请求端上点击订单所对应的“参与面审”按钮时，则向服务器发出视频连接请求。若服务器未在预设的答复时间阈值(如20-30s内任意一个时间值)内与请求端建立连接，则服务器自动向请求端发送“客服不在线”的提示信息。若服务器在所述答复时间阈值与请求端建立连接，则所述服务器与所述请求端进行视频通讯。
在一实施例中，如图3所示，步骤S110之后还包括：
S111、将所述请求端对应的视频数据在预先设置的申请人显示区域进行显示；
S112、根据所述视频连接请求对应的申请信息获取协助端信息；
S113、将协助视频连接请求发送至与所述协助端信息对应的协助端。
在本实施例中，当请求端与服务器建立连接后，请求端对应的申请人视频数据在如图4所示的用户交互界面的左上角区域(即申请人显示区域)进行显示，服务器对应的审核人视频数据在如图4所示用户交互界面的右下角(即审核人显示区域)进行显示。如图4所示用户交互界面的右上角和左下角则为增信人显示区域，也就是协助端的用户参与该多方面审时，是显示在用户交互界面的右上角和左下角则为增信人显示区域(例如用户交互界面的右上角为增信人1显示区域，左下角则为增信人2显示区域)。如图4所示用户交互界面中还设置有申请信息显示区域，用于显示申请信息。
当请求端与服务器成功建立连接后，此时需及时的通知协助端对应的增信人及时上线以参与多方视频。此时服务器为了精准的通知协助端，需要先根据所述视频连接请求对应的请 求端上传的申请信息获取协助端信息(主要是获取协助端的电话号码或用户账号等信息)，之后服务器根据协助端信息将协助视频连接请求发送至对应的协助端。这样即可实现当请求端对应的申请人和服务器对应的审核人均在线时，及时的通知协助端上线。
S120、根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息。
在本实施例中，请求端对应的申请人视频数据在如图4所示用户交互界面的左上角进行显示时，服务器对应的审核人选中左上角的申请人视频数据的头像右下角按钮后，选中申请人并显示在如图4所示用户交互界面的中央处的申请人放大显示区域。点击中央处的申请人放大显示区域下方的“人脸识别”按钮，服务器将申请人的头像与公安部留存的照片(即人脸数据库中已存储的特征模板)进行匹配验证，并在右侧的申请人及增信人身份验证结果显示区域显示相似度系数(一般是100％)，实现申请人身份验证。
在一实施例中，如图5所示，步骤S120包括：
S121、若检测到与请求端对应的人脸识别同意请求，获取与所述人脸识别同意请求对应时刻的当前图像；
S122、将所述当前图像对应的特征向量与人脸数据库中已存储的特征模板进行比对，以判断人脸数据库中已存储的特征模板中是否存在与所述当前图像对应的图片特征向量相同的特征模板；
S123、若人脸数据库中已存储的特征模板中存在与所述当前图像对应的图片特征向量相同的特征模板，获取对应的用户身份识别信息；
S124、若人脸数据库中已存储的特征模板中不存在与所述当前图像对应的图片特征向量相同的特征模板，进行增加当前用户身份识别信息的提示。
在本实施例中，当服务器与请求端建立连接后，服务器向请求端发出人脸识别请求，当请求端对应的申请人同意人脸识别时，则请求端向服务器发送人脸识别同意请求。当服务器检测到与请求端对应的人脸识别同意请求，获取与所述人脸识别同意请求对应时刻的当前图像(也即请求端采集当前图像后上传至服务器进行人脸识别)。之后将所述当前图像对应的特征向量与与人脸数据库中已存储的特征模板进行比对，若人脸数据库中已存储的特征模板中存在与所述当前图像对应的图片特征向量相同的特征模板，获取对应的用户身份识别信息。
在一实施例中，如图5所示，步骤S122之前还包括：
S1221、对所述当前图像进行灰度校正及噪声过滤，得到预处理后图片；
S1222、通过卷积神经网络模型获取与所述预处理后图片对应的图片特征向量。
在本实施例中，对于人脸的图像预处理是基于人脸检测结果，对图像进行处理并最终服务于特征提取的过程。服务器获取的原始图像由于受到各种条件的限制和随机干扰，往往不能直接使用，必须在图像处理的早期阶段对它进行灰度校正、噪声过滤等图像预处理。对于人脸图像而言，其预处理过程主要包括人脸图像的光线补偿、灰度变换、直方图均衡化、归一化、几何校正、滤波以及锐化等。
在获取图片的特征向量时，先获取与预处理后图片对应的像素矩阵，然后将预处理后图片对应的像素矩阵作为卷积神经网络模型中输入层的输入，得到多个特征图，之后将特征图输入池化层，得到每一特征图对应的最大值所对应一维行向量，最后将每一特征图对应的最大值所对应一维行向量输入至全连接层，得到与预处理后图片对应的图片特征向量。
由于人脸数据库中已存储的特征模板中存储了已采集的海量的人脸图片对应的特征向量，也即每一个人的人脸均对应唯一的特征向量，有了这些海量的特征模板为数据基础后，可以用来确定预处理后图片对应的一个或多个人，从而实现人脸识别。
最后，所得到的用户身份识别信息可以是用户的身份证号，由于每一公民的身份证号是唯一的，可以作为其唯一识别码。当完成了对申请人的用户身份识别之后与申请信息中对应的用户身份信息一致时，即可确保申请人是本人参与多方视频。
S130、若检测到已将定位信息获取指令发送至请求端，接收请求端根据服务器所发送的定位信息获取指令对应推送的当前定位信息。
在本实施例中，服务器需要获取请求端的定位信息时，先由服务器触发一个定位信息获取指令，然后由服务器将所述定位信息获取指令发送至请求端，请求端在获取了当前定位信息后将当前定位信息发送至服务器，最后服务器接收由请求端发送的当前定位信息，即实现了审核人员可实时监测申请人的位置信息，也即可以再次核实申请人提供的申请信息中所包括的地址信息是否有误。例如，请求端所发送的当前定位信息在如图4所示的用户交互界面中的定位信息显示区域进行显示。
S140、判断在预设的第一时间阈值内是否有至少一个与请求端对应的协助端未与服务器建立连接。
在本实施例中，当服务器与请求端成功建立连接后，此时服务器发送协助端上线请求至与请求端对应的一个或多个协助端。当协助端均及时在所述第一时间阈值(如设置第一时间阈值为5-10s)响应协助端上线请求并上线时，所有的协助端均及时上线参与多方视频，此时请求端和服务器在等待协助端上线的过程较短，无需做等待处理。具体的，所述第一时间阈值内无请求端对应的协助端未与服务器建立连接，将协助端对应的视频数据在如图4预先设置的增信人显示区域进行显示。当协助端与服务器连接成功时，也可在如图4设置的定位信息显示区域显示协助端对应的当前定位信息，例如此时申请人、增信人1和增信人2均在线时，此时定位信息显示区域以电子地图中点位点的方式分别显示申请人、增信人1和增信人2的地理位置定位点，从而实现人员地理分布图的显示效果，直观的显示各人员当前地理位置。
S150、若有至少一个与请求端对应的协助端未与服务器建立连接，获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，将所述视频特征序列输入至预先训练的卷积神经网络，得到与所述视频特征序列对应的视频场景分类结果；其中，所述视频信息包括当前时刻对应的时间参数、请求端的当前定位信息、视频场景的背景色信息。
在本实施例中，由于多方视频面审时，需要申请人和协助人员参与，此时若服务器检测到在请求端连接成功后，仍存在至少一个协助端未与服务器连接时，可在等待连接的区间，由服务器自动根据请求端在当前时刻对应的视频信息获取视频场景分类结果，由视频场景分类结果确定一个等待音乐进行播放，作为请求端和服务器中等待协助端连接过程中的背景音乐。
在一实施例中，如图6所示，步骤S150包括：
S151、获取所述视频信息中当前时刻对应的时间参数，将所述时间参数除以24进行归一化，得到第一取值；
S152、获取所述视频信息中所述当前定位信息，根据所述当前定位信息对应获取时区编号，将所述时区编号除以24进行归一化，得到第二取值；
S153、获取所述视频信息中所述视频场景的背景色信息，根据所述背景色信息对应获取RGB参数值，将所述RGB参数值除以256，得到第三取值序列；
S154、将所述第一取值、第二取值、第三取值序列进行串接，得到视频特征序列。
在本实施例中，例如当前时刻为12点整，则第一取值为12/24＝0、5；所述当前定位信息对应的经纬度处于东8区，则所述第二取值为8/24＝1/3。由于背景色信息一般包括R、G、B三个参数值，故将所述所述RGB参数值除以256，得到由三个值组成的第三取值序列；
例如背景色信息的RGB参数值为(128,128,128,)，则第三取值序列为128/256 128/256 128/256，即0、5 0、5 0、5。上述举例的第一取值、第二取值、第三取值序列进行串接后，得到视频特征序列为[1/2 1/3 1/2 1/2 1/2]。
在一实施例中，步骤S150之前还包括：
将训练集中每一视频特征序列作为待训练卷积神经网络的输入，将对应的视频场景分类结果作为待训练卷积神经网络的输出，对所述待训练卷积神经网络进行训练，得到用于分类视频场景的卷积神经网络；
训练集中与每一视频特征序列对应的视频场景分类结果是预先标注的，例如视频场景分类结果可以标注的值为1-10，其中1表示欢快的场景，2表示严肃的场景等。
S160、获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端。
在本实施例中，当得到与所述视频特征序列对应的视频场景分类结果之后，需要在该视频场景分类结果对应的背景音乐库中随机获取一首音乐作为当前推送的音频文件发送至请求端或协助端。例如视频场景分类结果为1时(表示欢快的场景)，其对应的背景音乐库为音乐库1(其中保存的欢快风格的音乐)，从音乐库1中随机选择其中一首音乐的音频数据发送至请求端或协助端，以作为请求端或协助端的等待音乐。
在一实施例中，步骤S160之后还包括：
若所述音频数据发送至请求端或协助端的发送时间与当前系统时间之间的间隔超出预先设置的第二时间阈值，将开启多方视频的提示信息发送至请求端和协助端。
在本实施例中，在请求端的申请人或至少一个已连接服务器的协助端对应的协助人员在收听了与第二时间阈值(如设置为30-120秒)等时长的音频后，此时为了尽快开启多方视频面审，可以将开启多视频方面审的提示信息发送至已连接服务器的请求端和协助端，提示在缺少一个协助端的情况下开启多方视频面审的视频会议，以减少等待时间。
在一实施例中，步骤S160之后还包括：
若检测到当前视频数据的获取指令，获取与请求端相对应预设时长的目标视频数据，通过流光法对所述目标视频数据进行预处理，得到与所述目标视频数据对应的目标图片集合。
在本实施例中，当请求端、协助端均与服务器建立连接并开启多方视频时，此时服务器在如图4所示的用户交互界面上点击“表情检测”按钮时，即可获取与请求端相对应预设时长的目标视频数据。此时通过光流法对申请人的微表情进行识别，以判断是否存在欺诈。
光流法的原理是当人的眼睛观察运动物体时，物体的景象在人眼的视网膜上形成一系列连续变化的图像，这一系列连续变化的信息不断“流过”视网膜(即图像平面)，好像是一种光的“流”，故称之为光流。光流表达图像的变化，包含目标运动的信息，可用来确定目标的运动。光流三个要素：一是运动速度场,这是形成光流的必要条件；二是带光学特征的部分例如有灰度的象素点，它可以携带运动信息；三是成像投影从场景到图像平面，因而能被观察到。
定义光流以点为基础，具体来说，设(u,v)为图像点(x,y)的光流，则把(x,y,u,v)称为光流点。所有光流点的集合称为光流场。当带光学特性的物体在三维空间运动时，在图像平面上就形成了相应的图像运动场，或称为图像速度场。在理想情况下，光流场对应于运动场。
给图像中的每个像素点赋予一个速度矢量，这样就形成了一个运动矢量场。根据各个像素点的速度矢量特征，可以对图像进行动态分析。如果图像中没有运动目标，则光流矢量在整个图像区域是连续变化的。当图像中有运动物体时(当用户有微表情时，脸部会有运动，相当于运动物体)，目标和背景存在着相对运动。运动物体所形成的速度矢量必然和背景的速度矢量有所不同，如此便可以计算出运动物体的位置。通过光流法进行预处理，获取与所述目标视频数据对应的目标图片集合。通过光流法进行预处理，获取了由目标视频数据中存在微表情的图片组成的目标图片集合。
此时，可以将所述目标图片集合推送至对应的接收端(这一接收端可以是另一用于微表情检测的云服务器，也可以是服务器中设置的用于微表情检测的子模块)进行微表情分析，判断申请人在于服务器进行视频通讯的过程中是否存在欺诈的可能。
该方法实现了进行多方视频时实时验证参与方的身份真实性，确保是参与者本人来参与多方视频会议，还能在视频等待期间根据视频场景分类结果对应随机播放等待期间的背景音乐，提高了视频的数据安全性。
本申请实施例还提供一种多方视频的用户身份验证装置，该多方视频的用户身份验证装 置用于执行前述多方视频的用户身份验证方法的任一实施例。具体地，请参阅图7，图7是本申请实施例提供的多方视频的用户身份验证装置的示意性框图。该多方视频的用户身份验证装置100可以配置于服务器中。
如图7所示，多方视频的用户身份验证装置100包括连接建立单元110、身份识别单元120、定位单元130、连接判断单元140、视频场景获取单元150、音频数据发送单元160。
连接建立单元110，用于若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接。
在本实施例中，当服务器检测到操作人员所录入的当前录入编号时，则进入等待请求端接入的等待状态。具体的，当服务器检测到操作人员所录入的当前录入编号后，服务器将视频连接请求发送至与当前录入编号对应的请求端，请求端根据视频连接请求对应发送视频连接同意请求至服务器。若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接，此时服务器与所述请求端进行视频通讯。
例如，将本申请的具体使用场景设置为信贷业务的多人视频面审场景，申请人请求信贷业务提供方进行视频面审之前，可以预先设置1-2个增信人(如申请人的配偶或好友，这些增信人使用协助端参与多人视频面审)以辅助参与视频面审。当申请人完成了申请信息(包括贷款申请信息、借款人信息、抵押信息及其他贷款信息)的编辑后，生成订单及与订单对应的唯一的订单编号，该订单编号会存储在服务器中。
当服务器对应的审核人此时需与申请人建立连接时，表示审核人已做好多方视频面审的准备工作，此时可以输入当前时刻可进行面审的订单编号。一旦审核人在服务器中完成订单编号的输入，则申请人对应的请求端的用户交互界面中更新一个“参与面审”的按钮。当申请人在请求端上点击订单所对应的“参与面审”按钮时，则向服务器发出视频连接请求。若服务器未在预设的答复时间阈值(如20-30s内任意一个时间值)内与请求端建立连接，则服务器自动向请求端发送“客服不在线”的提示信息。若服务器在所述答复时间阈值与请求端建立连接，则所述服务器与所述请求端进行视频通讯。
在一实施例中，如图8所示，多方视频的用户身份验证装置100还包括：
申请人视频显示单元111，用于将所述请求端对应的视频数据在预先设置的申请人显示区域进行显示；
协助端信息获取单元112，用于根据所述视频连接请求对应的申请信息获取协助端信息；
协助端连接发送单元113，用于将协助视频连接请求发送至与所述协助端信息对应的协助端。
在本实施例中，当请求端与服务器建立连接后，请求端对应的申请人视频数据在如图4所示的用户交互界面的左上角区域(即申请人显示区域)进行显示，服务器对应的审核人视频数据在如图4所示用户交互界面的右下角(即审核人显示区域)进行显示。如图4所示用户交互界面的右上角和左下角则为增信人显示区域，也就是协助端的用户参与该多方面审时，是显示在用户交互界面的右上角和左下角则为增信人显示区域(例如用户交互界面的右上角为增信人1显示区域，左下角则为增信人2显示区域)。如图4所示用户交互界面中还设置有申请信息显示区域，用于显示申请信息。
当请求端与服务器成功建立连接后，此时需及时的通知协助端对应的增信人及时上线以参与多方视频。此时服务器为了精准的通知协助端，需要先根据所述视频连接请求对应的请求端上传的申请信息获取协助端信息(主要是获取协助端的电话号码或用户账号等信息)，之后服务器根据协助端信息将协助视频连接请求发送至对应的协助端。这样即可实现当请求端对应的申请人和服务器对应的审核人均在线时，及时的通知协助端上线。
身份识别单元120，用于根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息。
在本实施例中，请求端对应的申请人视频数据在如图4所示用户交互界面的左上角进行显示时，服务器对应的审核人选中左上角的申请人视频数据的头像右下角按钮后，选中申请 人并显示在如图4所示用户交互界面的中央处的申请人放大显示区域。点击中央处的申请人放大显示区域下方的“人脸识别”按钮，服务器将申请人的头像与公安部留存的照片(即人脸数据库中已存储的特征模板)进行匹配验证，并在右侧的申请人及增信人身份验证结果显示区域显示相似度系数(一般是100％)，实现申请人身份验证。
在一实施例中，如图9所示，身份识别单元120包括：
当前图像获取单元121，用于若检测到与请求端对应的人脸识别同意请求，获取与所述人脸识别同意请求对应时刻的当前图像；
比对单元122，用于将所述当前图像对应的特征向量与人脸数据库中已存储的特征模板进行比对，以判断人脸数据库中已存储的特征模板中是否存在与所述当前图像对应的图片特征向量相同的特征模板；
第一处理单元123，用于若人脸数据库中已存储的特征模板中存在与所述当前图像对应的图片特征向量相同的特征模板，获取对应的用户身份识别信息；
第二处理单元124，用于若人脸数据库中已存储的特征模板中不存在与所述当前图像对应的图片特征向量相同的特征模板，进行增加当前用户身份识别信息的提示。
在本实施例中，当服务器与请求端建立连接后，服务器向请求端发出人脸识别请求，当请求端对应的申请人同意人脸识别时，则请求端向服务器发送人脸识别同意请求。当服务器检测到与请求端对应的人脸识别同意请求，获取与所述人脸识别同意请求对应时刻的当前图像(也即请求端采集当前图像后上传至服务器进行人脸识别)。之后将所述当前图像对应的特征向量与与人脸数据库中已存储的特征模板进行比对，若人脸数据库中已存储的特征模板中存在与所述当前图像对应的图片特征向量相同的特征模板，获取对应的用户身份识别信息。
在一实施例中，如图9所示，身份识别单元120还包括：
预处理单元1221，用于对所述当前图像进行灰度校正及噪声过滤，得到预处理后图片；
特征向量获取单元1222，用于通过卷积神经网络模型获取与所述预处理后图片对应的图片特征向量。
在本实施例中，对于人脸的图像预处理是基于人脸检测结果，对图像进行处理并最终服务于特征提取的过程。服务器获取的原始图像由于受到各种条件的限制和随机干扰，往往不能直接使用，必须在图像处理的早期阶段对它进行灰度校正、噪声过滤等图像预处理。对于人脸图像而言，其预处理过程主要包括人脸图像的光线补偿、灰度变换、直方图均衡化、归一化、几何校正、滤波以及锐化等。
在获取图片的特征向量时，先获取与预处理后图片对应的像素矩阵，然后将预处理后图片对应的像素矩阵作为卷积神经网络模型中输入层的输入，得到多个特征图，之后将特征图输入池化层，得到每一特征图对应的最大值所对应一维行向量，最后将每一特征图对应的最大值所对应一维行向量输入至全连接层，得到与预处理后图片对应的图片特征向量。
由于人脸数据库中已存储的特征模板中存储了已采集的海量的人脸图片对应的特征向量，也即每一个人的人脸均对应唯一的特征向量，有了这些海量的特征模板为数据基础后，可以用来确定预处理后图片对应的一个或多个人，从而实现人脸识别。
最后，所得到的用户身份识别信息可以是用户的身份证号，由于每一公民的身份证号是唯一的，可以作为其唯一识别码。当完成了对申请人的用户身份识别之后与申请信息中对应的用户身份信息一致时，即可确保申请人是本人参与多方视频。
定位单元130，用于若检测到已将定位信息获取指令发送至请求端，接收请求端根据服务器所发送的定位信息获取指令对应推送的当前定位信息。
在本实施例中，服务器需要获取请求端的定位信息时，先由服务器触发一个定位信息获取指令，然后由服务器将所述定位信息获取指令发送至请求端，请求端在获取了当前定位信息后将当前定位信息发送至服务器，最后服务器接收由请求端发送的当前定位信息，即实现了审核人员可实时监测申请人的位置信息，也即可以再次核实申请人提供的申请信息中所包括的地址信息是否有误。例如，请求端所发送的当前定位信息在如图4所示的用户交互界面 中的定位信息显示区域进行显示。
连接判断单元140，用于判断在预设的第一时间阈值内是否有至少一个与请求端对应的协助端未与服务器建立连接。
在本实施例中，当服务器与请求端成功建立连接后，此时服务器发送协助端上线请求至与请求端对应的一个或多个协助端。当协助端均及时在所述第一时间阈值(如设置第一时间阈值为5-10s)响应协助端上线请求并上线时，所有的协助端均及时上线参与多方视频，此时请求端和服务器在等待协助端上线的过程较短，无需做等待处理。具体的，所述第一时间阈值内无请求端对应的协助端未与服务器建立连接，将协助端对应的视频数据在如图4预先设置的增信人显示区域进行显示。当协助端与服务器连接成功时，也可在如图4设置的定位信息显示区域显示协助端对应的当前定位信息，例如此时申请人、增信人1和增信人2均在线时，此时定位信息显示区域以电子地图中点位点的方式分别显示申请人、增信人1和增信人2的地理位置定位点，从而实现人员地理分布图的显示效果，直观的显示各人员当前地理位置。
视频场景获取单元150，用于若有至少一个与请求端对应的协助端未与服务器建立连接，获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，将所述视频特征序列输入至预先训练的卷积神经网络，得到与所述视频特征序列对应的视频场景分类结果；其中，所述视频信息包括当前时刻对应的时间参数、请求端的当前定位信息、视频场景的背景色信息。
在本实施例中，由于多方视频面审时，需要申请人和协助人员参与，此时若服务器检测到在请求端连接成功后，仍存在至少一个协助端未与服务器连接时，可在等待连接的区间，由服务器自动根据请求端在当前时刻对应的视频信息获取视频场景分类结果，由视频场景分类结果确定一个等待音乐进行播放，作为请求端和服务器中等待协助端连接过程中的背景音乐。
在一实施例中，如图10所示，视频场景获取单元150包括：
第一取值计算单元151，用于获取所述视频信息中当前时刻对应的时间参数，将所述时间参数除以24进行归一化，得到第一取值；
第二取值计算单元152，用于获取所述视频信息中所述当前定位信息，根据所述当前定位信息对应获取时区编号，将所述时区编号除以24进行归一化，得到第二取值；
第三取值序列获取单元153，用于获取所述视频信息中所述视频场景的背景色信息，根据所述背景色信息对应获取RGB参数值，将所述RGB参数值除以256，得到第三取值序列；
取值串接单元154，用于将所述第一取值、第二取值、第三取值序列进行串接，得到视频特征序列。
在本实施例中，例如当前时刻为12点整，则第一取值为12/24＝0、5；所述当前定位信息对应的经纬度处于东8区，则所述第二取值为8/24＝1/3。由于背景色信息一般包括R、G、B三个参数值，故将所述所述RGB参数值除以256，得到由三个值组成的第三取值序列；
例如背景色信息的RGB参数值为(128,128,128,)，则第三取值序列为128/256 128/256 128/256，即0、5 0、5 0、5。上述举例的第一取值、第二取值、第三取值序列进行串接后，得到视频特征序列为[1/2 1/3 1/2 1/2 1/2]。
在一实施例中，多方视频的用户身份验证装置100还包括：
模型训练单元，用于将训练集中每一视频特征序列作为待训练卷积神经网络的输入，将对应的视频场景分类结果作为待训练卷积神经网络的输出，对所述待训练卷积神经网络进行训练，得到用于分类视频场景的卷积神经网络；
训练集中与每一视频特征序列对应的视频场景分类结果是预先标注的，例如视频场景分类结果可以标注的值为1-10，其中1表示欢快的场景，2表示严肃的场景等。
视频场景获取单元160，用于获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端。
在本实施例中，当得到与所述视频特征序列对应的视频场景分类结果之后，需要在该视频场景分类结果对应的背景音乐库中随机获取一首音乐作为当前推送的音频文件发送至请求端或协助端。例如视频场景分类结果为1时(表示欢快的场景)，其对应的背景音乐库为音乐库1(其中保存的欢快风格的音乐)，从音乐库1中随机选择其中一首音乐的音频数据发送至请求端或协助端，以作为请求端或协助端的等待音乐。
在一实施例中，多方视频的用户身份验证装置100还包括：
连接提示单元，用于若所述音频数据发送至请求端或协助端的发送时间与当前系统时间之间的间隔超出预先设置的第二时间阈值，将开启多方视频的提示信息发送至请求端和协助端。
在本实施例中，在请求端的申请人或至少一个已连接服务器的协助端对应的协助人员在收听了与第二时间阈值(如设置为30-120秒)等时长的音频后，此时为了尽快开启多方视频面审，可以将开启多视频方面审的提示信息发送至已连接服务器的请求端和协助端，提示在缺少一个协助端的情况下开启多方视频面审的视频会议，以减少等待时间。
在一实施例中，多方视频的用户身份验证装置100还包括：
微表情识别单元，用于若检测到当前视频数据的获取指令，获取与请求端相对应预设时长的目标视频数据，通过流光法对所述目标视频数据进行预处理，得到与所述目标视频数据对应的目标图片集合。
在本实施例中，当请求端、协助端均与服务器建立连接并开启多方视频时，此时服务器在如图4所示的用户交互界面上点击“表情检测”按钮时，即可获取与请求端相对应预设时长的目标视频数据。此时通过光流法对申请人的微表情进行识别，以判断是否存在欺诈。
光流法的原理是当人的眼睛观察运动物体时，物体的景象在人眼的视网膜上形成一系列连续变化的图像，这一系列连续变化的信息不断“流过”视网膜(即图像平面)，好像是一种光的“流”，故称之为光流。光流表达图像的变化，包含目标运动的信息，可用来确定目标的运动。光流三个要素：一是运动速度场,这是形成光流的必要条件；二是带光学特征的部分例如有灰度的象素点，它可以携带运动信息；三是成像投影从场景到图像平面，因而能被观察到。
定义光流以点为基础，具体来说，设(u,v)为图像点(x,y)的光流，则把(x,y,u,v)称为光流点。所有光流点的集合称为光流场。当带光学特性的物体在三维空间运动时，在图像平面上就形成了相应的图像运动场，或称为图像速度场。在理想情况下，光流场对应于运动场。
给图像中的每个像素点赋予一个速度矢量，这样就形成了一个运动矢量场。根据各个像素点的速度矢量特征，可以对图像进行动态分析。如果图像中没有运动目标，则光流矢量在整个图像区域是连续变化的。当图像中有运动物体时(当用户有微表情时，脸部会有运动，相当于运动物体)，目标和背景存在着相对运动。运动物体所形成的速度矢量必然和背景的速度矢量有所不同，如此便可以计算出运动物体的位置。通过光流法进行预处理，获取与所述目标视频数据对应的目标图片集合。通过光流法进行预处理，获取了由目标视频数据中存在微表情的图片组成的目标图片集合。
此时，可以将所述目标图片集合推送至对应的接收端(这一接收端可以是另一用于微表情检测的云服务器，也可以是服务器中设置的用于微表情检测的子模块)进行微表情分析，判断申请人在于服务器进行视频通讯的过程中是否存在欺诈的可能。
该装置实现了进行多方视频时实时验证参与方的身份真实性，确保是参与者本人来参与多方视频会议，还能在视频等待期间根据视频场景分类结果对应随机播放等待期间的背景音乐，提高了视频的数据安全性。
上述多方视频的用户身份验证装置可以实现为计算机程序的形式，该计算机程序可以在如图11所示的计算机设备上运行。
请参阅图11，图11是本申请实施例提供的计算机设备的示意性框图。该计算机设备500 是服务器，服务器可以是独立的服务器，也可以是多个服务器组成的服务器集群。
参阅图11，该计算机设备500包括通过系统总线501连接的处理器502、存储器和网络接口505，其中，存储器可以包括非易失性存储介质503和内存储器504。
该非易失性存储介质503可存储操作系统5031和计算机程序5032。该计算机程序5032被执行时，可使得处理器502执行多方视频的用户身份验证方法。
该处理器502用于提供计算和控制能力，支撑整个计算机设备500的运行。
该内存储器504为非易失性存储介质503中的计算机程序5032的运行提供环境，该计算机程序5032被处理器502执行时，可使得处理器502执行多方视频的用户身份验证方法。
该网络接口505用于进行网络通信，如提供数据信息的传输等。本领域技术人员可以理解，图11中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备500的限定，具体的计算机设备500可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
其中，所述处理器502用于运行存储在存储器中的计算机程序5032，以实现本申请实施例公开的多方视频的用户身份验证方法。
本领域技术人员可以理解，图11中示出的计算机设备的实施例并不构成对计算机设备具体构成的限定，在其他实施例中，计算机设备可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。例如，在一些实施例中，计算机设备可以仅包括存储器及处理器，在这样的实施例中，存储器及处理器的结构及功能与图11所示实施例一致，在此不再赘述。
应当理解，在本申请实施例中，处理器502可以是中央处理单元(Central ProcessingUnit，CPU)，该处理器502还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable GateArray，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。其中，通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
在本申请的另一实施例中提供计算机可读存储介质。该计算机可读存储介质可以为非易失性或可以为易失性的计算机可读存储介质。该计算机可读存储介质存储有计算机程序，其中计算机程序被处理器执行时实现本申请实施例公开的多方视频的用户身份验证方法。
在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以是两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分，或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，因此，本申请的保护范围应以权利要求的保护范围为准。

Claims (20)

  1. 一种多方视频的用户身份验证方法，其中，包括：
    若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接；
    根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息；
    若检测到已将定位信息获取指令发送至请求端，接收请求端根据服务器所发送的定位信息获取指令对应推送的当前定位信息；
    判断在预设的第一时间阈值内是否有至少一个与请求端对应的协助端未与服务器建立连接；
    若有至少一个与请求端对应的协助端未与服务器建立连接，获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，将所述视频特征序列输入至预先训练的卷积神经网络，得到与所述视频特征序列对应的视频场景分类结果；其中，所述视频信息包括当前时刻对应的时间参数、请求端的当前定位信息、视频场景的背景色信息；以及
    获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端。
  2. 根据权利要求1所述的多方视频的用户身份验证方法，所述若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接之后，还包括：
    将所述请求端对应的视频数据在预先设置的申请人显示区域进行显示；
    根据所述视频连接请求对应的申请信息获取协助端信息；
    将协助视频连接请求发送至与所述协助端信息对应的协助端。
  3. 根据权利要求2所述的多方视频的用户身份验证方法，其中，所述获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端之后，还包括：
    若所述音频数据发送至请求端或协助端的发送时间与当前系统时间之间的间隔超出预先设置的第二时间阈值，将开启多方视频的提示信息发送至请求端和协助端。
  4. 根据权利要求3所述的多方视频的用户身份验证方法，其中，所述获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端之后，还包括：
    若检测到当前视频数据的获取指令，获取与请求端相对应预设时长的目标视频数据，通过流光法对所述目标视频数据进行预处理，得到与所述目标视频数据对应的目标图片集合。
  5. 根据权利要求1-4任一项所述的多方视频的用户身份验证方法，其中，所述根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息，包括：
    若检测到与请求端对应的人脸识别同意请求，获取与所述人脸识别同意请求对应时刻的当前图像；
    将所述当前图像对应的特征向量与人脸数据库中已存储的特征模板进行比对，以判断人脸数据库中已存储的特征模板中是否存在与所述当前图像对应的图片特征向量相同的特征模板；
    若人脸数据库中已存储的特征模板中存在与所述当前图像对应的图片特征向量相同的特征模板，获取对应的用户身份识别信息；
    若人脸数据库中已存储的特征模板中不存在与所述当前图像对应的图片特征向量相同的特征模板，进行增加当前用户身份识别信息的提示。
  6. 根据权利要求5所述的多方视频的用户身份验证方法，其中，所述将所述当前图像对应的特征向量与人脸数据库中已存储的特征模板进行比对之前，还包括：
    对所述当前图像进行灰度校正及噪声过滤，得到预处理后图片；
    通过卷积神经网络模型获取与所述预处理后图片对应的图片特征向量。
  7. 根据权利要求1-4任一项所述的多方视频的用户身份验证方法，其中，所述获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，包括：
    获取所述视频信息中当前时刻对应的时间参数，将所述时间参数除以24进行归一化，得到第一取值；
    获取所述视频信息中所述当前定位信息，根据所述当前定位信息对应获取时区编号，将所述时区编号除以24进行归一化，得到第二取值；
    获取所述视频信息中所述视频场景的背景色信息，根据所述背景色信息对应获取RGB参数值，将所述RGB参数值除以256，得到第三取值序列；
    将所述第一取值、第二取值、第三取值序列进行串接，得到视频特征序列。
  8. 一种多方视频的用户身份验证装置，其中，包括：
    连接建立单元，用于若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接；
    身份识别单元，用于根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息；
    定位单元，用于若检测到已将定位信息获取指令发送至请求端，接收请求端根据服务器所发送的定位信息获取指令对应推送的当前定位信息；
    连接判断单元，用于判断在预设的第一时间阈值内是否有至少一个与请求端对应的协助端未与服务器建立连接；
    视频场景获取单元，用于若有至少一个与请求端对应的协助端未与服务器建立连接，获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，将所述视频特征序列输入至预先训练的卷积神经网络，得到与所述视频特征序列对应的视频场景分类结果；其中，所述视频信息包括当前时刻对应的时间参数、请求端的当前定位信息、视频场景的背景色信息；以及
    音频数据发送单元，用于获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端。
  9. 一种计算机设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，其中，所述处理器执行所述计算机程序时实现一种多方视频的用户身份验证方法，包括：
    若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接；
    根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息；
    若检测到已将定位信息获取指令发送至请求端，接收请求端根据服务器所发送的定位信息获取指令对应推送的当前定位信息；
    判断在预设的第一时间阈值内是否有至少一个与请求端对应的协助端未与服务器建立连接；
    若有至少一个与请求端对应的协助端未与服务器建立连接，获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，将所述视频特征序列输入至预先训练的卷积神经网络，得到与所述视频特征序列对应的视频场景分类结果；其中，所述视频信息包括当前时刻对应的时间参数、请求端的当前定位信息、视频场景的背景色信息；以及
    获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端。
  10. 根据权利要求9所述的一种计算机设备，所述若检测到请求端发送的与当前录入编 号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接之后，还包括：
    将所述请求端对应的视频数据在预先设置的申请人显示区域进行显示；
    根据所述视频连接请求对应的申请信息获取协助端信息；
    将协助视频连接请求发送至与所述协助端信息对应的协助端。
  11. 根据权利要求10所述的计算机设备，其中，所述获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端之后，还包括：
    若所述音频数据发送至请求端或协助端的发送时间与当前系统时间之间的间隔超出预先设置的第二时间阈值，将开启多方视频的提示信息发送至请求端和协助端。
  12. 根据权利要求11所述的计算机设备，其中，所述获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端之后，还包括：
    若检测到当前视频数据的获取指令，获取与请求端相对应预设时长的目标视频数据，通过流光法对所述目标视频数据进行预处理，得到与所述目标视频数据对应的目标图片集合。
  13. 根据权利要求9-12任一项所述的计算机设备，其中，所述根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息，包括：
    若检测到与请求端对应的人脸识别同意请求，获取与所述人脸识别同意请求对应时刻的当前图像；
    将所述当前图像对应的特征向量与人脸数据库中已存储的特征模板进行比对，以判断人脸数据库中已存储的特征模板中是否存在与所述当前图像对应的图片特征向量相同的特征模板；
    若人脸数据库中已存储的特征模板中存在与所述当前图像对应的图片特征向量相同的特征模板，获取对应的用户身份识别信息；
    若人脸数据库中已存储的特征模板中不存在与所述当前图像对应的图片特征向量相同的特征模板，进行增加当前用户身份识别信息的提示。
  14. 根据权利要求13所述的计算机设备，其中，所述将所述当前图像对应的特征向量与人脸数据库中已存储的特征模板进行比对之前，还包括：
    对所述当前图像进行灰度校正及噪声过滤，得到预处理后图片；
    通过卷积神经网络模型获取与所述预处理后图片对应的图片特征向量。
  15. 根据权利要求9-12任一项所述的计算机设备，其中，所述获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，包括：
    获取所述视频信息中当前时刻对应的时间参数，将所述时间参数除以24进行归一化，得到第一取值；
    获取所述视频信息中所述当前定位信息，根据所述当前定位信息对应获取时区编号，将所述时区编号除以24进行归一化，得到第二取值；
    获取所述视频信息中所述视频场景的背景色信息，根据所述背景色信息对应获取RGB参数值，将所述RGB参数值除以256，得到第三取值序列；
    将所述第一取值、第二取值、第三取值序列进行串接，得到视频特征序列。
  16. 一种计算机可读存储介质，其中，所述计算机可读存储介质存储有计算机程序，所述计算机程序当被处理器执行时使所述处理器执行一种多方视频的用户身份验证方法，其中，包括：
    若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接；
    根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息；
    若检测到已将定位信息获取指令发送至请求端，接收请求端根据服务器所发送的定位信息获取指令对应推送的当前定位信息；
    判断在预设的第一时间阈值内是否有至少一个与请求端对应的协助端未与服务器建立连接；
    若有至少一个与请求端对应的协助端未与服务器建立连接，获取请求端在当前时刻对应的视频信息，根据所述视频信息中各信息对应的取值组成视频特征序列，将所述视频特征序列输入至预先训练的卷积神经网络，得到与所述视频特征序列对应的视频场景分类结果；其中，所述视频信息包括当前时刻对应的时间参数、请求端的当前定位信息、视频场景的背景色信息；以及
    获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端。
  17. 根据权利要求16所述的计算机可读存储介质，所述若检测到请求端发送的与当前录入编号对应的视频连接同意请求，与所述视频连接同意请求对应的请求端建立连接之后，还包括：
    将所述请求端对应的视频数据在预先设置的申请人显示区域进行显示；
    根据所述视频连接请求对应的申请信息获取协助端信息；
    将协助视频连接请求发送至与所述协助端信息对应的协助端。
  18. 根据权利要求17所述的计算机可读存储介质，其中，所述获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端之后，还包括：
    若所述音频数据发送至请求端或协助端的发送时间与当前系统时间之间的间隔超出预先设置的第二时间阈值，将开启多方视频的提示信息发送至请求端和协助端。
  19. 根据权利要求18所述的计算机可读存储介质，其中，所述获取与所述视频场景分类结果对应的背景音乐库，并随机选择其中一首音乐的音频数据发送至请求端或协助端之后，还包括：
    若检测到当前视频数据的获取指令，获取与请求端相对应预设时长的目标视频数据，通过流光法对所述目标视频数据进行预处理，得到与所述目标视频数据对应的目标图片集合。
  20. 根据权利要求16-19任一项所述的计算机可读存储介质，其中，所述根据请求端发送的当前图像进行人脸识别，得到对应的用户身份识别信息，包括：
    若检测到与请求端对应的人脸识别同意请求，获取与所述人脸识别同意请求对应时刻的当前图像；
    将所述当前图像对应的特征向量与人脸数据库中已存储的特征模板进行比对，以判断人脸数据库中已存储的特征模板中是否存在与所述当前图像对应的图片特征向量相同的特征模板；
    若人脸数据库中已存储的特征模板中存在与所述当前图像对应的图片特征向量相同的特征模板，获取对应的用户身份识别信息；
    若人脸数据库中已存储的特征模板中不存在与所述当前图像对应的图片特征向量相同的特征模板，进行增加当前用户身份识别信息的提示。
PCT/CN2020/087025 2019-10-12 2020-04-26 多方视频的用户身份验证方法、装置及计算机设备 WO2021068485A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910968909.9A CN111126124A (zh) 2019-10-12 2019-10-12 多方视频的用户身份验证方法、装置及计算机设备
CN201910968909.9 2019-10-12

Publications (1)

Publication Number Publication Date
WO2021068485A1 true WO2021068485A1 (zh) 2021-04-15

Family

ID=70495300

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/087025 WO2021068485A1 (zh) 2019-10-12 2020-04-26 多方视频的用户身份验证方法、装置及计算机设备

Country Status (2)

Country Link
CN (1) CN111126124A (zh)
WO (1) WO2021068485A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113034110B (zh) * 2021-03-30 2023-12-22 泰康保险集团股份有限公司 基于视频审核的业务处理方法、系统、介质与电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107979603A (zh) * 2017-12-05 2018-05-01 江苏飞视文化发展有限公司 一种会议系统中成员的跟踪控制方法
CN108717663A (zh) * 2018-05-18 2018-10-30 深圳壹账通智能科技有限公司 基于微表情的面签欺诈判断方法、装置、设备及介质
CN109558192A (zh) * 2018-10-31 2019-04-02 深圳壹账通智能科技有限公司 一种视频面签的实现方法、装置、存储介质及计算机设备
CN109587554A (zh) * 2018-10-29 2019-04-05 百度在线网络技术（北京）有限公司 视频数据的处理方法、装置及可读存储介质
US20190213420A1 (en) * 2018-01-09 2019-07-11 Qualcomm Incorporated Adaptive object detection and recognition

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102143203A (zh) * 2010-11-25 2011-08-03 北京播思软件技术有限公司 一种实现移动终端同步分享多媒体文件系统及方法
CN102207966B (zh) * 2011-06-01 2013-07-10 华南理工大学 基于对象标签的视频内容快速检索方法
WO2015025189A1 (en) * 2013-08-18 2015-02-26 Yogesh Chunilal Rathod Enable user to establish request data specific connections with other users for communication, participation and collaboration
CN103686349B (zh) * 2013-12-27 2017-04-12 深圳市九洲电器有限公司 一种多屏互动的方法、装置及机顶盒
CN105681920B (zh) * 2015-12-30 2017-03-15 深圳市鹰硕音频科技有限公司 一种具有语音识别功能的网络教学方法及系统
CN108427874A (zh) * 2018-03-12 2018-08-21 平安科技（深圳）有限公司 身份认证方法、服务器及计算机可读存储介质
CN109961530B (zh) * 2019-01-16 2024-03-08 深圳市武智科技有限公司 基于图像识别的考勤方法、装置、计算机设备及存储介质
CN109787975A (zh) * 2019-01-17 2019-05-21 深圳壹账通智能科技有限公司 身份认证方法、装置、计算机设备及存储介质
CN109787976B (zh) * 2019-01-17 2021-11-16 深圳壹账通智能科技有限公司 信息更新方法、装置、计算机设备及存储介质
CN110147471A (zh) * 2019-04-04 2019-08-20 平安科技（深圳）有限公司 基于视频的轨迹跟踪方法、装置、计算机设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107979603A (zh) * 2017-12-05 2018-05-01 江苏飞视文化发展有限公司 一种会议系统中成员的跟踪控制方法
US20190213420A1 (en) * 2018-01-09 2019-07-11 Qualcomm Incorporated Adaptive object detection and recognition
CN108717663A (zh) * 2018-05-18 2018-10-30 深圳壹账通智能科技有限公司 基于微表情的面签欺诈判断方法、装置、设备及介质
CN109587554A (zh) * 2018-10-29 2019-04-05 百度在线网络技术（北京）有限公司 视频数据的处理方法、装置及可读存储介质
CN109558192A (zh) * 2018-10-31 2019-04-02 深圳壹账通智能科技有限公司 一种视频面签的实现方法、装置、存储介质及计算机设备

Also Published As

Publication number Publication date
CN111126124A (zh) 2020-05-08

Similar Documents

Publication Publication Date Title
KR101731404B1 (ko) 음성 및/또는 안면 인식 기반 서비스 제공
US8548208B2 (en) Multifactor multimedia biometric authentication
US20160104253A1 (en) Device, System, and Method of Enhancing User Privacy and Security Within a Location-Based Virtual Social Networking Context
CN108108012B (zh) 信息交互方法和装置
JP7473556B2 (ja) 承諾確認
WO2018201965A1 (zh) 设备指纹生成方法、装置及计算设备
US10499097B2 (en) Methods, systems, and media for detecting abusive stereoscopic videos by generating fingerprints for multiple portions of a video frame
WO2021042513A1 (zh) 视频聊天中添加表情的方法、装置、计算机设备及存储介质
US11568615B2 (en) Collaborative on-demand experiences
CN111768478B (zh) 一种图像合成方法、装置、存储介质和电子设备
CN109005104A (zh) 一种即时通信方法、装置、服务器及存储介质
CN110175849A (zh) 收款方法、装置、设备、服务器及系统
KR20230063772A (ko) 메타버스 개인 맞춤형 콘텐츠 생성 및 인증 방법 및 그를 위한 장치 및 시스템
WO2021068485A1 (zh) 多方视频的用户身份验证方法、装置及计算机设备
CN113792662B (zh) 图像检测方法、装置、电子设备以及存储介质
JP2019159647A (ja) プログラム、ネットワークシステム及び画像判定方法
CN108647613B (zh) 一种应用于考场的考生查验方法
US20200387693A1 (en) Systems and methods for facial recognition-based participant identification and management in multi-participant activity
US10212592B2 (en) Systems and methods for keyword—and location-based user authentication
CN110602405A (zh) 拍摄方法和装置
WO2022222957A1 (zh) 一种目标识别的方法和系统
CN113596597A (zh) 一种游戏视频加速方法、装置、计算机设备和存储介质
JP2023520737A (ja) オンラインネットワークにおいてアソシエーションを管理するためのシステムおよび方法
CN110100445B (zh) 信息处理系统、信息处理装置和计算机可读介质
CN113194323B (zh) 信息交互方法、多媒体信息互动方法以及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20874458

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 19.08.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20874458

Country of ref document: EP

Kind code of ref document: A1