CN115294630A - Conference system and conference management method - Google Patents

Conference system and conference management method Download PDF

Info

Publication number
CN115294630A
CN115294630A CN202210911710.4A CN202210911710A CN115294630A CN 115294630 A CN115294630 A CN 115294630A CN 202210911710 A CN202210911710 A CN 202210911710A CN 115294630 A CN115294630 A CN 115294630A
Authority
CN
China
Prior art keywords
meeting
image
images
frame
conference
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210911710.4A
Other languages
Chinese (zh)
Inventor
朱鹰
张兵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Huigong Yun Information Technology Co ltd
Original Assignee
Suzhou Huigong Yun Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Huigong Yun Information Technology Co ltd filed Critical Suzhou Huigong Yun Information Technology Co ltd
Priority to CN202210911710.4A priority Critical patent/CN115294630A/en
Publication of CN115294630A publication Critical patent/CN115294630A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management
    • G06Q10/109—Time management, e.g. calendars, reminders, meetings or time accounting
    • G06Q10/1091—Recording time for administrative or management purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/70—Multimodal biometrics, e.g. combining information from different biometric modalities
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C1/00—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people
    • G07C1/10—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people together with the recording, indicating or registering of other data, e.g. of signs of identity

Abstract

The present disclosure provides a conference system and a conference management method, the conference system including: the system comprises an image acquisition unit, a control unit, a reference database, an identity identification unit and a calculation unit, wherein the image acquisition unit acquires at least one group of images containing meeting personnel, and the at least one group of images comprise multi-frame images at a certain moment before or after the meeting starts; the identity recognition unit extracts the feature vectors of the meeting personnel and each frame of face image in the multi-frame images, compares the feature vectors with a reference database to obtain a comparison result, and obtains the identity recognition result of the meeting personnel according to the comparison result; and the computing unit obtains a participant evaluation result according to the identity recognition result. The adoption of multi-frame images and more than two groups of images improves the accuracy of automatic check-in and monitoring of the conference.

Description

Conference system and conference management method
Technical Field
The present disclosure relates to the field of office automation technologies, and in particular, to a conference system and a conference management method.
Background
With the development of network technology, video communication modes such as video conferences, video teaching, video phones and the like are increasingly popularized. The existing online conference system has an automatic sign-in technology based on face recognition, and adopts a face recognition module to collect and screen facial photos of participants, and attendance check-in is carried out after verification, specifically referring to CN212624175U.
The automatic conference sign-in technology based on face recognition is simple, when facing conferences of various different occasions, the conference sign-in accuracy is low, whether head portraits are used for conference entrance can not be recognized, whether conference participants can not be known continuously, and the conference management level is low.
Disclosure of Invention
In view of the above, an object of the present disclosure is to provide a conference system and a conference management method, which can solve at least one technical problem that the accuracy of automatic conference sign-in is low and that the in-conference state of the participants cannot be monitored or quantitatively monitored.
In view of the above object, the present disclosure provides a conference system including: an image acquisition unit, a control unit, a reference database, an identity recognition unit and a calculation unit,
the image acquisition unit acquires at least one group of images containing the meeting personnel, wherein the at least one group of images comprises multi-frame images before the meeting starts or at a certain moment after the meeting starts;
the control unit controls the image acquisition unit to acquire images according to preset image acquisition parameters;
the reference database comprises identity information of the participants and a facial image reference characteristic vector;
the identity recognition unit extracts the feature vector of each frame of face image in the multi-frame image of each meeting person, compares the feature vector with a reference database and obtains the identity recognition result of each meeting person;
and the computing unit obtains a participant evaluation result according to the identity recognition result.
Optionally, the facial image reference feature vector includes a plurality of facial image reference feature vectors at different angles.
Optionally, the reference database further includes a body shape image reference feature vector.
Optionally, the body shape image reference feature vector includes a plurality of body shape image reference feature vectors at different angles.
Optionally, the identity recognition unit further extracts the feature vectors of the meeting people and each frame of body image in the multiple frames of images, and compares the feature vectors with a reference database to obtain the identity recognition result of the meeting people.
Optionally, the meeting evaluation result includes whether to check in.
Optionally, the image acquisition parameters include shooting time of each group of images, or a time interval of shooting each group of images, or conference start time, conference end time, or conference duration.
Optionally, the conference evaluation result includes a full-distance conference rate, where the full-distance conference rate = (total conference time-cumulative time of the absence of the staff)/total conference time.
The present disclosure also provides a conference management method, including the following steps:
s10, acquiring at least one group of images containing meeting personnel according to the image acquisition parameters, wherein the at least one group of images comprise multi-frame images at a certain moment before or after the meeting is started;
s20, extracting the feature vector of each frame of face image in the multiple frames of images of the meeting people;
s30, comparing the feature vector obtained in the step S20 with reference data to obtain a comparison result of each frame of face image of the meeting person;
s40, confirming the identity of the meeting person according to the comparison result of the face image;
and S50, obtaining a participant evaluation result.
Optionally, the step S40 further includes:
s41, judging whether the comparison result of each frame of face image of the same meeting person is the same, and if so, confirming the identity of the meeting person according to the comparison result;
s42, if the comparison results of each frame of face images of the same meeting person are different, the identity recognition of the meeting person is unsuccessful, and the feature vectors of each frame of body shape image in the multi-frame images of the meeting person of which the identity recognition is unsuccessful are extracted;
s43, comparing the characteristic vector of the body shape image obtained in the step S42 with reference data to obtain a comparison result of each frame of body shape image of the meeting personnel.
Optionally, the step S44 further includes:
s45, judging whether the comparison result of each frame of body shape image of the meeting people is the same or not, and if so, confirming the identity of the meeting people according to the comparison result;
s46, if the comparison results of each frame of body shape image of the arriving person are different, the identity recognition of the arriving person is unsuccessful, and then the image data of the arriving person is screened out for manual recognition.
Optionally, the meeting evaluation result includes whether to check in, or the overall meeting rate, or the attendance rate.
As described above, compared with the prior art, the technical scheme provided by the present disclosure achieves the following beneficial effects:
1. the identity of the meeting personnel is identified by adopting at least one group of multi-frame images, and because the image shooting time interval of the multi-frame images is short and the similarity among the multi-frame images is very high, the influence on subsequent image identification caused by the change of shooting background, shooting light intensity, shooting angle and the like caused by the continuous movement of the personnel during the meeting is effectively avoided, and the accuracy of automatic sign-in and monitoring of the meeting is improved; more than two groups of multi-frame images with certain time intervals are adopted for recognition, so that face recognition failure caused by face shielding, head lowering, head twisting and the like at a certain shooting moment is avoided, and the accuracy of personnel identity recognition is improved;
2. after the face image identification fails, the body shape image is further adopted for identity identification, so that the accuracy of personnel identity identification is effectively improved, and the accuracy and efficiency of automatic conference sign-in and conference monitoring are improved;
3. the face image characteristic vectors and the body shape image characteristic vectors at different angles are used as reference data to be compared, so that image recognition failure caused by image shooting angles is effectively avoided, and the accuracy and efficiency of automatic conference sign-in and conference monitoring are improved;
4. after the face image or the body shape image is not successfully identified, the image data which is not successfully identified is screened out, manual identification is further carried out, the automatic conference sign-in efficiency is improved, and meanwhile the accuracy of conference sign-in and conference monitoring is guaranteed.
5. When the face image and the body shape image are identified, data with high occurrence frequency of the IDs of the participants are screened out for next identification, and the efficiency of automatic sign-in and monitoring is further improved.
6. The continuous meeting state of the meeting personnel is evaluated in a quantitative mode through the whole-course meeting rate, and the meeting management level and the meeting efficiency are improved.
Drawings
In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings needed to be used in the description of the embodiments or related technologies are briefly introduced below, and it is obvious that the drawings in the following description are only embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic diagram of an embodiment of a conferencing system of the present disclosure;
FIG. 2 is a schematic diagram of another embodiment of a conferencing system of the present disclosure;
FIG. 3 is a schematic diagram of yet another embodiment of a conferencing system of the present disclosure;
FIG. 4 is a schematic diagram of a reference database of the conferencing system of the present disclosure;
fig. 5 is a schematic diagram of an identification unit of the conferencing system of the present disclosure;
FIG. 6 is a schematic diagram illustrating an embodiment of a conference management method according to the present disclosure;
FIG. 7 is a schematic diagram of another embodiment of a meeting management method of the present disclosure;
fig. 8 is a schematic diagram of a conference management method according to still another embodiment of the disclosure.
Detailed Description
For the purpose of promoting a better understanding of the objects, aspects and advantages of the present disclosure, reference is made to the following detailed description taken in conjunction with the accompanying drawings.
It is to be noted that technical terms or scientific terms used in the embodiments of the present disclosure should have a general meaning as understood by one having ordinary skill in the art to which the present disclosure belongs, unless otherwise defined. The use of "first," "second," and similar terms in the embodiments of the disclosure is not intended to indicate any order, quantity, or importance, but rather to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.
As shown in fig. 1, an embodiment of a conference system according to the present disclosure includes: the device comprises an image acquisition unit, a control unit, a reference database, an identity recognition unit and a calculation unit.
The image acquisition unit is used for acquiring at least one group of images containing the meeting personnel, and the at least one group of images comprises multi-frame images before the meeting starts or at a certain moment after the meeting starts.
It will be appreciated that the at least one set of images may be one, two or more sets during an automatic check-in, or monitoring whether a meeting continues. The at least one group of images can be acquired within a predetermined time period before the conference starts or after the conference starts, for example, within 10 minutes before the conference starts or after the conference starts, and the at least one group of images is used as a check-in link and is set according to actual conditions. The certain time may be 2 seconds or less, preferably 1 second or less. The acquisition interval of each set of images is 20s to 120s, preferably 60s.
The at least one group of images is continuous multi-frame images at a certain moment, or multi-frame images with a preset time interval, such as multi-frame images with an interval of 10ms to 50 ms. The similarity between the multi-frame images is high, the changes of shooting background, shooting light intensity, shooting angle and the like caused by continuous movement of personnel during a conference can be effectively avoided by adopting the multi-frame images, the success rate of image acquisition is improved, the influence on subsequent image identification caused by low image acquisition quality is reduced, and then the accuracy of automatic attendance and state monitoring of the conference is improved.
The image is a face image and can be a face image at any angle.
Optionally, the multi-frame image further includes a body shape image, which may be a body shape image at any angle, and the body shape image may be a whole body shape image, a half body shape image, or a head portrait.
The image acquisition unit is preferably a camera and can shoot multi-frame images or videos containing the participants during the conference;
optionally, the image obtaining unit is an image or video data receiving device, and is capable of receiving or extracting required image data from the image or video data shot by the camera.
Optionally, the image capturing unit captures an image while recording the image capturing time.
Optionally, the image obtaining unit stores the obtained image or video in a conference memory, and labels the conference ID.
And the control unit is used for setting the image acquisition parameters of the image acquisition unit and controlling the image acquisition unit to acquire images according to the set image acquisition parameters.
Optionally, the setting mode of the image acquisition parameters may be implemented through a human-computer interaction interface of the control unit.
Optionally, the image acquisition parameters include: the number of groups of images, the acquisition time and time interval of each group of images, the number of frames of each group of images, the acquisition mode (continuous acquisition or interval acquisition) of multiple frames of images, the time interval of each frame of images, and the like. Besides the image acquisition parameters, conference information such as conference ID, participants, conference start time, conference end time, and conference duration can be set through the control unit, so as to obtain conference plan and conference history data, and image comparison parameters such as a first threshold, a second threshold, a frequency threshold, and an identification interval of each group of images. The identification interval is used for monitoring the conference and calculating the accumulated time of the absence of the participants.
Optionally, the control unit does not include a memory and directly receives or processes the video stream data.
Optionally, the control unit comprises a memory for storing the image or video data obtained by the image acquisition unit, as shown in fig. 2.
The mode for controlling the image acquisition unit is preferably an automatic control mode, and the image acquisition device is automatically controlled by a timer in the control unit to realize the acquisition of images or videos according to preset shooting parameters. Specifically, in one embodiment, the timer may trigger the camera to start capturing at the conference start time, capture a set of images at each predetermined point in time, and stop capturing at the end of the conference.
Optionally, the mode of controlling the image acquisition unit is implemented by a manual trigger mode, specifically, after the shooting parameters are set through a human-computer interaction interface of the control unit, a shooting button is manually clicked to control the camera device to start shooting.
A reference database storing reference information of the participant, as shown in fig. 4; the reference information specifically comprises feature vectors of face images of a plurality of different angles of the participants recorded in advance and corresponding participantsPerson ID, name, gender, age, etc. For example, the reference database stores reference information of S participants, corresponding to S records, each record including P n A field. For participant S1, field P1 is the ID of the participant S1, field P 2 ～P i Is the facial image feature vector P of the participant under i-1 different angles 2 Is the feature vector, P, of the face image of the front face of the participant S1 3 The feature vectors of the face image on the left side of the participant S1 are 8230823082308230and so on.
The identity recognition unit is used for recognizing the identity of an actual meeting person (called the meeting person for short) and recognizing a face, and as shown in fig. 5, the identity recognition unit includes a feature extraction module and a comparison module, the feature extraction module extracts a feature vector of each frame of face in at least one group of multi-frame face images, optionally, the extraction method is to call a face recognition algorithm to process the face images to obtain the feature vector of each face. Specifically, assuming that a certain group of images obtained by the image obtaining unit includes N frames of images, each frame of image includes M meeting people, N × M face images are obtained altogether, feature extraction is performed on the N × M face images in sequence, and finally feature vectors of the N × M faces are obtained.
The comparison module compares the feature vectors of each frame of face obtained by the feature extraction module with the reference features of the participants stored in the reference database one by one in a similarity manner, and obtains a comparison result according to a preset first threshold.
Optionally, the identity recognition unit includes a memory for storing the comparison result.
Specifically, the feature vector of the first frame face image in a certain group of images of the obtained first meeting person is compared with the first record (P) stored in the reference database 2 ～P i ) Comparing the face feature vectors of different angles one by one, wherein the comparison method can be difference or quotient, comparing the comparison result with a preset first threshold, if the comparison result is greater than or equal to the first threshold, the identity of the face is identified by the comparison, and the ID of the P1 field of the record of the current comparison is extracted as the comparison resultAnd is marked as D1. If the comparison result is less than the first threshold, the comparison result is not accepted, and the comparison result is continued to be recorded with the second record (P) 2 ～P i ) And comparing the face feature vectors of different angles one by one until the face feature vectors are compared with the S records, and obtaining comparison results.
After the comparison of the first frame of face image of the first arriving person is completed, the feature vector of the second frame of face image of the first arriving person in the group of images is compared with S records (P) stored in the reference database 2 ～P i ) Comparing the face feature vectors of different angles one by one, comparing the comparison result with a preset first threshold, if the comparison result is greater than or equal to the first threshold, determining that the identity of the face is identified by the comparison, extracting the ID of the P1 field of the record which is currently compared, and recording the ID as a comparison result as D2. If the comparison result is less than the first threshold, the comparison result is not accepted, and the comparison result is continued to be compared with the second record (P) 2 ～P i ) And comparing the face feature vectors of different angles one by one until the face feature vectors are compared with the S records, and obtaining comparison results.
And analogizing in sequence, comparing the characteristic vectors of the N frames of images in the group of images of the first meeting person with the reference data in sequence, and obtaining D in all the images N And (6) comparing the results.
Similarly, the N frames of face images of the second meeting person in a certain group of images are identified until the Mth meeting person, and finally M meeting persons and each meeting person D are obtained N And (5) comparing the results.
It is to be understood that, if the image acquisition unit acquires two or more sets of images, after the recognition of the first set of images is completed, the recognition of the next set of images is continued until the recognition of the images of all the sets is completed.
And the computing unit is used for obtaining a conference evaluation result or a conference statistical result according to the comparison result. The result of the evaluation of the meeting comprises whether the person signs in or not, if the person who meets the meeting is the same, the D N If the comparison results are the same and are the same ID, the identity identification of the meeting personnel is successful, and the meeting personnel is automatically determined to have signed in.
The participant who has not checked in can be known by confirming the participant who has checked in. If the identified participant does not belong to the current meeting, it is ignored.
In another embodiment, the reference database further includes a plurality of different angle omnidirectional body image feature vectors of the participant, which are obtained in advance, and the specific storage manner is the same as that of the face image feature vector, such as the field P i+1 ～P j The system is used for respectively storing the body shape image feature vectors of a plurality of different angles.
The identity recognition unit further performs body shape image recognition. When the same person is in a meeting D N If the comparison result is not unique, the identity of the meeting person is not successfully identified. And screening out a plurality of personnel IDs with the highest frequency of occurrence of the personnel IDs in the comparison result or higher than a preset threshold value. For the person whose identity is not successfully recognized after face recognition, the feature extraction module extracts the feature vector of each frame of image in the group of images and the multi-frame figure image of the person who meets, optionally, the extraction method is to call a figure recognition algorithm to process the figure image to obtain the feature vector of the figure image. Specifically, assuming that m persons are unsuccessfully identified after face recognition is performed on a certain group of images, feature vector extraction is sequentially performed on N frame body images of the m persons in the group of images to obtain feature vectors of m × N body images in total.
The comparison module compares the feature vectors of the N frames of body shape images of the m people with the screened reference features of the IDs of the multiple people one by one in a similarity manner, and obtains a comparison result according to a preset second threshold value.
Specifically, assuming that s personal ID's are screened out when the face of a certain meeting person is recognized, the feature vector of the first frame figure image in the N frame figure images of the meeting person and P of the first record in the s records stored in the reference database are compared with each other i+1 ～P j Comparing the body shape characteristic vectors of different angles one by one, comparing the comparison result with a preset second threshold, if the comparison result is greater than or equal to the second threshold, determining that the identity of the body shape is identified by the comparison, and extracting the current record of comparisonThe ID of the P1 field is denoted as d1 as the alignment result. If the comparison result is less than the second threshold, the comparison result is not accepted, and the comparison result is continued to be compared with the P of the second record i+1 ～P j And comparing the face feature vectors of different angles one by one until the face feature vectors are compared with the s records, and obtaining comparison results.
And analogizing in sequence, comparing the feature vectors of the N frames of images of the meeting people with the s pieces of reference data in sequence to obtain d N And (6) comparing the results.
Similarly, the N frame body-shape images of the second meeting personnel in a certain group of images are identified until the m meeting personnel, and finally m meeting personnel and each meeting personnel d are obtained N And (5) comparing the results.
It is to be understood that, if the image acquisition unit acquires two or more sets of images, after the first set of images is recognized, the recognition of the next set of images is continued until the image recognition of all sets is completed.
The computing unit calculates d according to each person in the meeting N And obtaining the participant evaluation result by the comparison result. If d is N If the comparison results are the same and are the same ID, the identity identification of the meeting personnel is successful, and the meeting personnel is automatically determined to have signed in.
In another embodiment, as shown in FIG. 3, when d N When the comparison result is not unique, the identity identification of the meeting personnel is failed, the data of the failed identification are extracted, and the relevant meeting ID is taken for manual comparison.
In another embodiment, the conference evaluation result further comprises a whole-course conference rate, so that conference monitoring during a conference can be realized, and the conference condition of the participants can be quantitatively evaluated. The full-course participation rate = (total duration of the conference-cumulative duration of the absence of the staff)/total duration of the conference.
And setting the identification interval of each group of images through the control unit, and calculating the accumulated time length of the person in the absence. If the person is identified within a certain identification interval, the person is considered to be continuously present within the identification interval. If the person is not identified in the identification interval through face image identification, body shape image identification and manual identification, the person is considered to be absent in the identification interval, and the absent duration of the identification is equal to the identification interval. And identifying the same person in the whole meeting period at a set identification interval, and obtaining the duration of the absence of the meeting after accumulation.
Optionally, the identification interval is 1 to 10 minutes, preferably 3 minutes, and may be specifically selected according to actual situations such as meeting duration.
Specifically, assuming that the identification interval is set to 3 minutes, if the person is identified at least one time within the identification interval, it is considered that the person is in a meeting for the 3-minute identification interval; if the person is not identified through face image identification and body shape image identification within the identification interval of 3 minutes, a prompt for manual identification is given to the data system, manual identification is carried out, and if the participant is not identified after manual identification, the time length of the participant not meeting in the identification is considered to be 3 minutes. After the identification interval of the 3 minutes is finished, the identification of the next 3-minute time period is carried out until the whole period of the conference. And accumulating the whole meeting period to obtain the accumulated time length of the personnel in the meeting.
And marking according to the result of the manual identification, and marking the whole course of participation, late arrival, early departure, non-whole course of participation, non-participation and the like.
Optionally, in the beginning or ending time period of the conference, the identification interval is shortened so as to identify whether to participate in the whole course, automatically judge whether to arrive late or leave early, and modify the manual labeling part.
The calculating unit is respectively connected with the image acquiring unit, the control unit and the identity identifying unit, the shooting time of each group of images is acquired from the image acquiring unit, whether the person is identified or not is recorded at different time points by combining the identification result of each group of images acquired from the identity identifying unit, and the duration of the person which is not identified is obtained by accumulating calculation. The calculating unit obtains the conference starting time and the conference ending time from the control unit to obtain the total conference time. And finally, obtaining the whole-course participation rate according to a calculation formula.
In another embodiment, the meeting assessment result includes an attendance rate = actual meeting people/number of meeting participants.
Another embodiment of the present disclosure further provides a conference management method, as shown in fig. 6, including the following steps:
s10, acquiring at least one group of images containing the meeting personnel according to the image acquisition parameters, wherein the at least one group of images comprise multi-frame images at a certain moment before the meeting starts or after the meeting starts;
optionally, before step S10, setting image acquisition parameters includes: shooting time of each group of images, the number of the shooting groups, shooting intervals of each group of images, identification intervals of each group of images, the number of acquisition frames of each group of images, acquisition mode (continuous acquisition or interval acquisition) of each group of images, interval time of each frame of images and other shooting parameters. Besides setting image acquisition parameters, conference information such as conference ID, participants, conference start time, conference end time and conference duration can be set to obtain conference plan and conference historical data; and image comparison parameters such as a first threshold, a second threshold, a frequency threshold, an identification interval of each group of images, and the like.
The image acquisition may start at some time before and after the start of the conference, for example 0-10 minutes before the start of the conference, as a check-in link, and end until the end of the conference. And storing the acquired picture in a conference image memory, and corresponding to the ID of the conference. Optionally, a sign-in link may not be set, and the camera is automatically triggered to start shooting the video at the beginning of the conference, and the video recording is ended until the conference is finished.
The image and the manner of acquiring the image, the shooting time, and the storage are the same as those in the other embodiments.
S20, extracting the feature vectors of each frame of face image in the multi-frame images of the meeting people;
s30, comparing the feature vector of the face image obtained in the step S20 with reference data to obtain a comparison result of each frame of face image of the meeting personnel; the comparison result is ID of the participant, and is obtained according to a first threshold value, namely the comparison result of the feature vector is compared with a preset first threshold value, if the comparison result is greater than or equal to the first threshold value, the identity of the frame of the face image is identified by the comparison, and the ID of the participant corresponding to the current comparison reference data is extracted as the comparison result; the reference data comprises face image feature vectors of a plurality of different angles;
optionally, step S30 further comprises the step of,
s31, screening out IDs with the occurrence frequency higher than a preset threshold value or the highest occurrence frequency from the IDs of the participants; an ID having a high frequency of appearance represents a high possibility that it is the owner of the image;
s40, confirming the identity of the meeting person according to the comparison result of the face image;
and S50, obtaining a participant evaluation result.
As shown in fig. 7, in another embodiment of the present disclosure, optionally, the step S40 further includes:
s41, judging whether the comparison results of each frame of face images of the same meeting person are the same or not, if so, determining the identity of the meeting person according to the comparison result, and taking the person ID corresponding to the comparison result as the identity ID of the meeting person;
s42, if the comparison result of each frame of face image of the same meeting person is different and the comparison result is not unique, the identity recognition of the meeting person is unsuccessful, and the feature vector of each frame of body image in the multi-frame image of the meeting person of which the identity recognition is unsuccessful is extracted;
s43, comparing the characteristic vector of the body shape image obtained in the step S42 with reference data to obtain a comparison result of each frame of body shape image of the meeting personnel; the comparison result is ID of the participant, and is obtained according to a second threshold value, namely the comparison result of the feature vector is compared with a preset second threshold value, if the comparison result is greater than or equal to the second threshold value, the identity of the frame body shape image is identified by the comparison, and the ID of the participant corresponding to the currently compared reference data is extracted as the comparison result; the reference data comprises body shape image feature vectors of a plurality of different angles;
and S44, confirming the identity of the meeting person according to the comparison result of the body shape image.
Optionally, in step S43, the reference data is the reference data corresponding to the ID screened in step S31; the reference data of the ID of the person screened in the face image recognition is used as the comparison data of the next step, so that all the reference data do not need to be compared, and the efficiency of identity recognition is improved;
as shown in fig. 8, in another embodiment of the present disclosure, optionally, the step S44 further includes:
s45, judging whether the comparison result of each frame of body shape image of the meeting person is the same, if so, indicating that the meeting person is uniquely matched with a person, confirming the identity of the meeting person according to the comparison result, namely, taking the person ID corresponding to the comparison result as an identity identification result;
s46, if the comparison result of each frame of body shape image of the meeting personnel is different and the comparison result is not unique, the identification of the meeting personnel is not successful, and the image data of the meeting personnel which is not successfully identified is screened out for manual identification.
Optionally, after the face recognition in step S42 is unsuccessful, the body shape image recognition is not performed, and manual recognition is directly performed.
The method for comparing the face image feature vector, the body shape image feature vector and the reference data in the embodiment is the same as that in other embodiments.
Optionally, the meeting evaluation result includes whether to check in, or the overall meeting rate, or the attendance rate.
If the ID of the meeting personnel is identified in a certain time period, such as a check-in link, the check-in is automatically confirmed. According to the person who signs in and the person who should attend the meeting which is set in advance, the person who does not sign in can be known.
Optionally, the conference participation evaluation result further includes a whole conference participation rate, which is used for realizing conference monitoring during a conference and quantitatively evaluating the conference situations of the conference participants. The conference evaluation result comprises a whole conference rate = (total conference time-cumulative time of the absence of the staff)/total conference time. The absence cumulative time period of the person is calculated by defining the identification interval for each set of images. If the person is identified within a certain identification interval, the person is considered to be continuously present within the identification interval. If the person is not identified in the identification interval through face image identification, body shape image identification and manual identification, the person is considered to be absent in the identification interval, and the absent period of the identification is equal to the identification interval. And identifying the same person in the whole meeting period at a set identification interval, and obtaining the duration of the absence of the meeting after accumulation.
Optionally, the identification interval is 1 to 10 minutes, preferably 3 minutes, and may be specifically selected according to actual situations such as meeting duration.
Specifically, assuming that the identification interval is set to 3 minutes, if the person is identified at least one time within the identification interval, it is considered that the person is in a meeting for the 3-minute identification interval; if the person is not identified through face image identification and body shape image identification within the identification interval of 3 minutes, a prompt for manual identification is given to the data system, manual identification is carried out, and if the participant is not identified after manual identification, the time length of the participant not meeting in the identification is considered to be 3 minutes. After the identification interval of the 3 minutes is finished, the identification of the next 3-minute time period is carried out until the whole period of the conference. And accumulating the whole meeting period to obtain the accumulated time length of the personnel in the meeting.
And marking according to the result of the manual identification, and marking the whole course of participation, late arrival, early departure, non-whole course of participation, non-participation and the like.
Optionally, in the beginning or ending time period of the conference, the identification interval is shortened so as to identify whether to participate in the whole course, automatically judge whether to arrive late or leave early, and modify the manual labeling part.
By calculating the whole-course meeting rate, whether meeting personnel participate in the meeting in the whole course can be automatically and quantitatively evaluated, the meeting management efficiency and level are improved, and the meeting opening quality is improved.
The attendance = actual attended/attended.
Optionally, the face image is obtained directly through a face positioning frame during shooting. Or segmenting the shot body shape image to obtain a face image and a body shape image.
In some embodiments, for the setting of the first threshold, the second threshold, and the predetermined threshold, the face and the body may be set according to different called algorithms and actual needs, respectively, so as to help the processor to filter the final result of the result output by the algorithm.
In the above embodiment, the feature vector of the reference data is obtained by: the method comprises the steps of inputting face images or body images of a plurality of different angles of front, left side, right side, back, overlooking, looking up and the like of participants in advance, calling a recognition algorithm to carry out face modeling and body modeling, and obtaining face image reference feature vectors and body image reference feature vectors of the participants.
The modeling and calculation of the feature vector may be performed by identifying specific points in the image and calculating the scale.
Optionally, the face feature vector includes: face length/width, face aspect ratio, face skin tone, width or height of local details on the face, such as mouth, nose, ears, etc. Optionally, the face recognition algorithm includes: a recognition algorithm based on human face feature points, a recognition algorithm based on the whole human face image, a recognition algorithm based on a template, a recognition algorithm based on a neural network and the like.
Optionally, the body shape image may be a whole body shape, a half body shape or a head portrait. The body shape feature vector comprises: head length/width, head aspect ratio, shoulder width, head width to shoulder aspect ratio, vertical distance of crown to shoulder, vertical distance of chin to shoulder, distance of shoulder to ground, etc.
Optionally, the method for obtaining the body shape image feature vector may obtain the body shape feature vector meeting expectations by inputting the sampled picture into a deep learning network model and by large data driven model self-learning; a gray value profile acquisition method may also be used.
Compared with the prior art, the technical scheme provided by the disclosure has the following beneficial effects:
1. the identity recognition of the meeting personnel is carried out by adopting at least one group of multi-frame images, and because the image shooting time interval of the multi-frame images is short and the similarity among the multi-frame images is very high, the influence on the subsequent image recognition caused by the change of shooting background, shooting light intensity, shooting angle and the like caused by the continuous movement of personnel during the meeting is effectively avoided, and the accuracy of automatic sign-in and monitoring of the meeting is further improved; more than two groups of multi-frame images with certain time intervals are adopted for recognition, so that face recognition failure caused by face shielding, head lowering, head twisting and the like at a certain shooting moment is avoided, and the accuracy of personnel identity recognition is improved;
2. after the face image identification fails, the body shape image is further adopted for identity identification, so that the accuracy of personnel identity identification is effectively improved, and the accuracy and efficiency of automatic conference sign-in and conference monitoring are improved;
3. the face image characteristic vectors and the body shape image characteristic vectors at different angles are used as reference data to be compared, so that image recognition failure caused by image shooting angles is effectively avoided, and the accuracy and efficiency of automatic conference sign-in and conference monitoring are improved;
4. after the face image or the body shape image is not successfully identified, image data which are not successfully identified are screened out, manual identification is further carried out, automatic conference sign-in efficiency is improved, and meanwhile conference sign-in and conference monitoring accuracy is guaranteed.
5. When the face image and the body shape image are identified, data with high occurrence frequency of the IDs of the participants are screened out for next identification, and the efficiency of automatic sign-in and monitoring is further improved.
6. The continuous meeting state of the meeting personnel is evaluated in a quantitative mode through the whole-course meeting rate, and the meeting management level and the meeting efficiency are improved.
While the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description.
The disclosed embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalents, improvements, and the like that may be made within the spirit and principles of the embodiments of the disclosure are intended to be included within the scope of the disclosure.

Claims (10)

1. A conferencing system, comprising: an image acquisition unit, a control unit, a reference database, an identity recognition unit and a calculation unit,
the image acquisition unit acquires at least one group of images containing the meeting personnel, wherein the at least one group of images comprises multi-frame images before the meeting starts or at a certain moment after the meeting starts;
the control unit controls the image acquisition unit to acquire images according to preset image acquisition parameters;
the reference database comprises identity information of the participants and a facial image reference characteristic vector;
the identity recognition unit extracts the feature vectors of each frame of face image in the multi-frame images of the meeting personnel, compares the feature vectors with a reference database to obtain a face image comparison result, and obtains the identity recognition result of the meeting personnel according to the comparison result;
and the computing unit obtains a participant evaluation result according to the identity recognition result.
2. The conferencing system of claim 1, wherein the facial image reference feature vector comprises a plurality of facial image reference feature vectors at different angles.
3. The conferencing system of claim 1, wherein the results of the face image comparison are filtered, wherein the frequency of occurrence of the results is greater than a predetermined threshold or the frequency of occurrence of the results is the highest.
4. The conferencing system of claim 1, wherein the reference database further comprises a figure-of-body reference feature vector.
5. The conferencing system of claim 4, wherein the body image reference feature vector comprises a plurality of body image reference feature vectors at different angles.
6. The conference system according to any one of claims 3 to 5, wherein the identification unit further extracts feature vectors of each frame of body image in the multi-frame images of the meeting persons, compares the feature vectors with a reference database to obtain a body image comparison result, and obtains the identification result of the meeting persons.
7. The conferencing system of claim 1, wherein the participant evaluation result comprises a check-in, or a global participation rate, or an attendance rate.
8. A conference management method, comprising the steps of:
s10, acquiring at least one group of images containing the meeting personnel according to the image acquisition parameters, wherein the at least one group of images comprise multi-frame images at a certain moment before the meeting starts or after the meeting starts;
s20, extracting the feature vectors of each frame of face image in the multi-frame images of the meeting people;
s30, comparing the feature vector obtained in the step S20 with reference data to obtain a comparison result of each frame of face image of the meeting person;
s40, confirming the identity of the meeting person according to the comparison result of the face image;
and S50, obtaining the evaluation result of the participant.
9. The conference management method according to claim 8, wherein said step S40 further comprises:
s41, judging whether the comparison result of each frame of face image of the same meeting person is the same, and if so, confirming the identity of the meeting person according to the comparison result;
s42, if the comparison results of each frame of face images of the same meeting person are different, the identity recognition of the meeting person is unsuccessful, and the feature vectors of each frame of body shape image in the multi-frame images of the meeting person of which the identity recognition is unsuccessful are extracted;
s43, comparing the characteristic vector of the body shape image obtained in the step S42 with reference data to obtain a comparison result of each frame of body shape image of the meeting person;
and S44, confirming the identity of the meeting person according to the comparison result of the body shape image.
10. The conference management method according to claim 9, wherein said step S44 further comprises:
s45, judging whether the comparison result of each frame of body shape image of the meeting person is the same, and if so, confirming the identity of the meeting person according to the comparison result;
s46, if the comparison result of each frame of body shape image of the arriving person is different, the identity recognition of the arriving person is not successful, and image data of the arriving person which is not successfully recognized is screened out for manual recognition.
CN202210911710.4A 2022-07-28 2022-07-28 Conference system and conference management method Pending CN115294630A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210911710.4A CN115294630A (en) 2022-07-28 2022-07-28 Conference system and conference management method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210911710.4A CN115294630A (en) 2022-07-28 2022-07-28 Conference system and conference management method

Publications (1)

Publication Number Publication Date
CN115294630A true CN115294630A (en) 2022-11-04

Family

ID=83826309

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210911710.4A Pending CN115294630A (en) 2022-07-28 2022-07-28 Conference system and conference management method

Country Status (1)

Country Link
CN (1) CN115294630A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116052260A (en) * 2023-03-24 2023-05-02 江西省气象服务中心(江西省专业气象台、江西省气象宣传与科普中心) Method and system for roll call of weather consultation video conference

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116052260A (en) * 2023-03-24 2023-05-02 江西省气象服务中心(江西省专业气象台、江西省气象宣传与科普中心) Method and system for roll call of weather consultation video conference

Similar Documents

Publication Publication Date Title
KR102177235B1 (en) An attendance check system using deep learning based face recognition
CN112801608A (en) Remote video conference intelligent management system based on big data and cloud computing and cloud conference management platform
CN108961509A (en) A kind of cell recognition of face entrance guard security system and its method
CN110414381A (en) Tracing type face identification system
CN109299690B (en) Method capable of improving video real-time face recognition precision
CN113411542A (en) Intelligent working condition monitoring equipment
CN115294630A (en) Conference system and conference management method
CN109754478A (en) A kind of face intelligent Checking on Work Attendance method of low user's fitness
CN106710020A (en) Intelligent attendance checking method and system
CN109840962A (en) Recognition of face movable attendance checking system and method
CN112487928A (en) Classroom learning condition real-time monitoring method and system based on feature model
CN110599129A (en) Campus attendance checking method, device, identification terminal and system based on image tracking
CN109410387A (en) A kind of recognition of face prison safeguard management method and system
CN114898443A (en) Face data acquisition method and device
CN111191506A (en) Personnel flow statistical method and device, computer equipment and storage medium
CN109345427B (en) Classroom video frequency point arrival method combining face recognition technology and pedestrian recognition technology
CN112911195B (en) Online live television teleconference intelligent management system based on cloud computing and artificial intelligence
CN210933642U (en) Body-building guidance system
KR20070077264A (en) A method of face recognizing
CN110895685A (en) Smile service quality evaluation system and evaluation method based on deep learning
CN116884088A (en) Monitoring scene sleep sentry recognition method
CN112950801B (en) Remote office attendance recording method and device
CN113537073A (en) Method and system for accurately processing special events in business hall
CN111310602A (en) System and method for analyzing attention of exhibit based on emotion recognition
CN112052706B (en) Electronic device and face recognition method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination