CN111325082A - Personnel concentration degree analysis method and device - Google Patents

Personnel concentration degree analysis method and device Download PDF

Info

Publication number
CN111325082A
CN111325082A CN201910577403.5A CN201910577403A CN111325082A CN 111325082 A CN111325082 A CN 111325082A CN 201910577403 A CN201910577403 A CN 201910577403A CN 111325082 A CN111325082 A CN 111325082A
Authority
CN
China
Prior art keywords
image
analyzed
person
processed
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910577403.5A
Other languages
Chinese (zh)
Other versions
CN111325082B (en
Inventor
裘沙怡
郑伟健
罗志强
赖言慧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision System Technology Co Ltd
Original Assignee
Hangzhou Hikvision System Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision System Technology Co Ltd filed Critical Hangzhou Hikvision System Technology Co Ltd
Priority to CN201910577403.5A priority Critical patent/CN111325082B/en
Publication of CN111325082A publication Critical patent/CN111325082A/en
Application granted granted Critical
Publication of CN111325082B publication Critical patent/CN111325082B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Image Analysis (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The embodiment of the invention provides a method for analyzing concentration of a person, which comprises the following steps: firstly, acquiring at least one frame of to-be-processed image containing an image of a person to be analyzed, performing behavior analysis on the person to be analyzed in the frame of to-be-processed image aiming at each frame of to-be-processed image to obtain an analysis result corresponding to the frame of to-be-processed image, then determining behavior data of the person to be analyzed based on the analysis result corresponding to each frame of to-be-processed image, and obtaining the concentration degree of the person to be analyzed according to a pre-established mapping relation between the behavior data and the concentration degree and the behavior data of the person to be analyzed. Therefore, the concentration degree of the to-be-analyzed person in the to-be-processed image can be determined by analyzing the acquired to-be-processed image, the to-be-analyzed person does not need to actively report various behaviors of the to-be-analyzed person, and the concentration degree of the to-be-analyzed person can be automatically analyzed without depending on manual work.

Description

Personnel concentration degree analysis method and device
Technical Field
The invention relates to the technical field of behavior analysis, in particular to a method and a device for analyzing concentration degree of personnel.
Background
In some scenarios, it is often necessary to know the concentration of the person. For example, in a classroom, the concentration of students is generally required to be known; in a corporate meeting, it is often necessary to know the concentration of employees, and so on.
In the existing scheme, most of the personnel actively report the times of various behaviors per se, and the concentration degree of the personnel is determined according to the information reported by the personnel. For example, in the classroom, the student can carry classroom feedback terminal, is provided with the button that different actions correspond on this terminal, for example when the student carried out the action of holding up one's hands, can press the button that the action of holding up one's hands that sets up on this terminal corresponds, and like this, the teacher can add up the number of times of the various actions of student to confirm that the student is absorbed in the degree in the study of classroom.
Therefore, a scheme which does not depend on manual work and automatically analyzes the concentration degree of the staff is needed urgently.
Disclosure of Invention
The embodiment of the invention aims to provide a method for analyzing the concentration degree of a person, so that the concentration degree of the person can be automatically analyzed without depending on manpower. The specific technical scheme is as follows:
the embodiment of the invention provides a method for analyzing concentration of a person, which comprises the following steps:
acquiring at least one frame of to-be-processed image containing an image of a person to be analyzed;
aiming at each frame of image to be processed, performing behavior analysis on the person to be analyzed in the frame of image to be processed to obtain an analysis result corresponding to the frame of image to be processed;
determining behavior data of the person to be analyzed based on the analysis result corresponding to each frame of image to be processed;
and obtaining the concentration degree of the person to be analyzed according to the pre-established mapping relation between the behavior data and the concentration degree and the behavior data of the person to be analyzed.
Optionally, after the acquiring at least one frame of to-be-processed image including an image of a person to be analyzed, the method further includes:
determining the identity information of the personnel to be analyzed in each frame of image to be processed based on each frame of image to be processed;
and establishing a corresponding relation between the identity information of the person to be analyzed and the concentration degree of the person to be analyzed.
Optionally, the acquiring at least one frame of to-be-processed image including an image of a person to be analyzed includes:
acquiring at least one frame of panoramic image acquired aiming at a scene where a person to be analyzed is located;
intercepting an area containing the human body of the person to be analyzed from each frame of panoramic image to serve as an image to be processed corresponding to the frame of panoramic image;
before determining the identity information of the person to be analyzed in each frame of image to be processed based on each frame of image to be processed, the method further comprises:
intercepting each face area from the panoramic image corresponding to the image to be processed as a candidate face image;
determining a first coordinate value of each candidate face image in the panoramic image corresponding to the image to be processed, and identifying identity information corresponding to the candidate face image;
the determining the identity information of the person to be analyzed in each frame of image to be processed based on each frame of image to be processed comprises:
determining a second coordinate value of the image to be processed in the corresponding panoramic image;
determining a first coordinate value matched with the second coordinate value as a target first coordinate value in the determined first coordinate values;
and determining the identity information corresponding to the target first coordinate value as the identity information of the person to be analyzed.
Optionally, the obtaining the concentration degree of the person to be analyzed according to the pre-established mapping relationship between the behavior data and the concentration degree and the behavior data of the person to be analyzed includes:
inputting the behavior data of the person to be analyzed into a behavior analysis model obtained by pre-training, and performing regression analysis processing to obtain the concentration degree of the person to be analyzed;
the behavior analysis model is obtained by training based on a preset training set, and the preset training set comprises sample behavior data and concentration degrees corresponding to the sample behavior data.
Optionally, the behavior analysis model is a ridge regression model:
Y＝θTX
wherein θ is a mapping coefficient of the ridge regression model, T represents transposing a matrix, X is behavior data of the person to be analyzed, and Y is concentration of the person to be analyzed;
training the preset training set by adopting the following loss function to obtain a mapping coefficient theta of the ridge regression model:
Figure BDA0002112448370000031
wherein J (θ ') is a loss value, X' is the sample behavior data, Y 'is the concentration corresponding to the sample behavior data, T represents transposing the matrix, α is a constant coefficient, | θ' | magnetism2Ridge regression for θ'; and theta ' is a variable coefficient of the loss function, and the corresponding variable coefficient theta ' when the J (theta ') is a minimum value is used as a mapping coefficient theta of the ridge regression model.
Optionally, the person to be analyzed is a student to be analyzed; the method for analyzing the behavior of the person to be analyzed in each frame of image to be processed to obtain the analysis result corresponding to the frame of image to be processed includes:
aiming at each frame of image to be processed, analyzing whether the student to be analyzed in the frame of image to be processed has any one of the following behaviors: standing, listening, speaking, reading, writing, lifting hands, lying down a desk and playing a mobile phone to obtain an analysis result corresponding to the frame of image to be processed;
the determining the behavior data of the person to be analyzed based on the analysis result corresponding to each frame of image to be processed includes:
and counting the times of various behaviors of the student to be analyzed in each frame of image to be processed as behavior data of the student to be analyzed.
The embodiment of the invention also provides a device for analyzing the concentration degree of the personnel, which comprises:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring at least one frame of image to be processed containing an image of a person to be analyzed;
the analysis module is used for performing behavior analysis on the person to be analyzed in each frame of image to be processed to obtain an analysis result corresponding to the frame of image to be processed;
the determining module is used for determining the behavior data of the person to be analyzed based on the analysis result corresponding to each frame of image to be processed;
and the mapping module is used for obtaining the concentration degree of the person to be analyzed according to the pre-established mapping relation between the behavior data and the concentration degree and the behavior data of the person to be analyzed.
Optionally, the apparatus further comprises:
the identity determining module is used for determining the identity information of the person to be analyzed in each frame of image to be processed based on each frame of image to be processed; and establishing a corresponding relation between the identity information of the person to be analyzed and the concentration degree of the person to be analyzed.
Optionally, the obtaining module is specifically configured to obtain at least one frame of panoramic image collected for a scene where a person to be analyzed is located; intercepting an area containing the human body of the person to be analyzed from each frame of panoramic image to serve as an image to be processed corresponding to the frame of panoramic image;
the device further comprises:
the identity information identification module is used for intercepting each face area from the panoramic image corresponding to the image to be processed as a candidate face image; determining a first coordinate value of each candidate face image in the panoramic image corresponding to the image to be processed, and identifying identity information corresponding to the candidate face image;
the identity determination module is specifically configured to determine a second coordinate value of the to-be-processed image in the corresponding panoramic image; determining a first coordinate value matched with the second coordinate value as a target first coordinate value in the determined first coordinate values; and determining the identity information corresponding to the target first coordinate value as the identity information of the person to be analyzed.
Optionally, the mapping module is specifically configured to:
inputting the behavior data of the person to be analyzed into a behavior analysis model obtained by pre-training, and performing regression analysis processing to obtain the concentration degree of the person to be analyzed;
the behavior analysis model is obtained by training based on a preset training set, and the preset training set comprises sample behavior data and concentration degrees corresponding to the sample behavior data.
Optionally, the behavior analysis model is a ridge regression model:
Y＝θTX
wherein θ is a mapping coefficient of the ridge regression model, T represents transposing a matrix, X is behavior data of the person to be analyzed, and Y is concentration of the person to be analyzed;
the mapping module is specifically configured to:
training the preset training set by adopting the following loss function to obtain a mapping coefficient theta of the ridge regression model:
Figure BDA0002112448370000051
wherein J (θ ') is a loss value, X' is the sample behavior data, Y 'is the concentration corresponding to the sample behavior data, T represents transposing the matrix, α is a constant coefficient, | θ' | magnetism2Ridge regression for θ'; theta’And for the variable coefficient of the loss function, taking the corresponding variable coefficient theta 'when the J (theta') is a minimum value as the mapping coefficient theta of the ridge regression model.
Optionally, the person to be analyzed is a student to be analyzed;
the analysis module is specifically configured to analyze, for each frame of to-be-processed image, whether a student to be analyzed in the frame of to-be-processed image has any one of the following behaviors: standing, listening, speaking, reading, writing, lifting hands, lying down a desk and playing a mobile phone to obtain an analysis result corresponding to the frame of image to be processed;
the determining module is specifically configured to count the times of various behaviors of the student to be analyzed in each frame of image to be processed, and use the counted times as behavior data of the student to be analyzed.
The embodiment of the invention also provides electronic equipment which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
a memory for storing a computer program;
and a processor for implementing any of the aforementioned methods of analyzing concentration of a person when executing the program stored in the memory.
The embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements any of the above-mentioned methods for analyzing concentration of a person.
Embodiments of the present invention also provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform any of the above described methods of human concentration analysis.
The method and the device for analyzing the concentration degree of the personnel, provided by the embodiment of the invention, comprise the steps of firstly, acquiring at least one frame of image to be processed containing an image of the personnel to be analyzed, carrying out behavior analysis on the personnel to be analyzed in the frame of image to be processed aiming at each frame of image to be processed to obtain an analysis result corresponding to the frame of image to be processed, then, determining behavior data of the personnel to be analyzed based on the analysis result corresponding to each frame of image to be processed, and obtaining the concentration degree of the personnel to be analyzed according to a pre-established mapping relation between the behavior data and the concentration degree and the behavior data of the personnel to be analyzed. Therefore, the concentration degree of the to-be-analyzed person in the to-be-processed image can be determined by analyzing the acquired to-be-processed image, the to-be-analyzed person does not need to actively report various behaviors of the to-be-analyzed person, and the concentration degree of the to-be-analyzed person can be automatically analyzed without depending on manual work. Of course, not all of the above advantages need be achieved in the practice of any one product or method of the present invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a first schematic flow chart of a method for analyzing concentration of a person according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a method for analyzing concentration of a person according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a staff concentration degree analysis apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In general, in some scenarios, it is often necessary to know the concentration of the person. For example, in a classroom, the concentration of students is generally required to be known; in a corporate meeting, it is often necessary to know the concentration of employees, and so on.
In the related schemes, most of the people actively report the times of various behaviors per se, and the concentration degree of the people is determined according to the information reported by the people. For example, in the classroom, the student can carry classroom feedback terminal, for example electronic equipment such as intelligent wrist-watch or cell-phone, is provided with the button that different actions correspond on this terminal, for example when the student has carried out the action of holding one's hands, can press the button that the action of holding one's hands that sets up on this terminal corresponds, and like this, the teacher can add up the number of times of the various actions of student, carries out the number of times of various actions according to the student in the classroom, confirms the study of student in the classroom and concentrates on the degree.
However, in the above scheme, the to-be-analyzed person needs to report the behavior of the to-be-analyzed person in real time, which may affect the concentration of the to-be-analyzed person in work or study, and in addition, the information reported by the to-be-analyzed person cannot be monitored, and there may be a situation that data does not match the reality, which may affect the analysis result of the concentration of the to-be-analyzed person.
Therefore, in order to solve the above technical problems, embodiments of the present invention provide a method for analyzing attention of a person, which can be applied to various electronic devices, such as a computer, a terminal server, a video recorder, and the like, without limitation.
The method of analyzing concentration of a person provided in the embodiment of the present invention is generally described below.
In one implementation, the method for analyzing the concentration of the person includes:
acquiring at least one frame of to-be-processed image containing an image of a person to be analyzed;
aiming at each frame of image to be processed, performing behavior analysis on the person to be analyzed in the frame of image to be processed to obtain an analysis result corresponding to the frame of image to be processed;
determining behavior data of a person to be analyzed based on an analysis result corresponding to each frame of image to be processed;
and obtaining the concentration degree of the person to be analyzed according to the pre-established mapping relation between the behavior data and the concentration degree and the behavior data of the person to be analyzed.
As can be seen from the above, the method for analyzing the concentration of the person provided in the embodiment of the present invention can determine the concentration of the person to be analyzed in the image to be processed by analyzing the obtained image to be processed, and the person to be analyzed does not need to actively report various behaviors of the person to be analyzed, so that the person to be analyzed can automatically analyze the concentration of the person to be analyzed without depending on manual work.
The method for analyzing concentration of a person provided by the embodiment of the present invention will be described in detail by specific examples.
As shown in fig. 1, a first flowchart of a method for analyzing concentration of a person according to an embodiment of the present invention includes the following steps:
s101: at least one frame of image to be processed containing an image of a person to be analyzed is acquired.
For example, the image to be processed may be a human body image acquired for a person to be analyzed, or may also be a panoramic image acquired for a current scene, and the image to be processed may include a human body image of the person to be analyzed, or may include human body images of a plurality of persons to be analyzed, which is not limited specifically.
In some scenes, the scene where the person to be analyzed is located may be monitored by the image acquisition device, so as to obtain at least one frame of image of the person to be analyzed as the image to be processed, for example, the person to be analyzed may be a student in a classroom, and then a classroom where the student to be analyzed is located may be monitored, so as to obtain a human body image of the student; or, the person to be analyzed may also be an employee in a conference, and then, the conference room where the employee to be analyzed is located may be monitored, so as to obtain a human body image of the employee, and so on.
Specifically, at least one frame of panoramic image for the scene where the person to be analyzed is located can be acquired through image acquisition equipment, and then the area of the human body of the person to be analyzed is intercepted from each frame of panoramic image and is used as the image to be processed corresponding to the frame of panoramic image; or, tracking the person to be analyzed through the image acquisition equipment, and directly taking the human body image of the person to be analyzed obtained through tracking as the image to be processed.
The image capturing device may be a network camera, a surveillance camera, or a dome camera, for example, the image capturing device may be a dual-lens camera including a fixed-point lens and a moving-point lens; the fixed-point lens is used for shooting the panorama of the scene where the to-be-analyzed person is located, and the moving-point lens can sequentially scan each to-be-analyzed person in the shot scene according to preset parameters. The image capturing device and the electronic device (execution main body) may be the same device or different devices, and are not limited specifically.
In one implementation, after at least one frame of to-be-processed image including an image of a person to be analyzed is acquired, identity information of the person to be analyzed in the frame of to-be-processed image may be further determined based on each frame of to-be-processed image.
Specifically, for each frame of image to be processed, the image to be processed may be identified, the position of the person to be analyzed is determined according to the coordinates of the person to be analyzed in the image to be processed and the corresponding relationship between the coordinates of the person to be analyzed in the image to be processed and the position in the current scene, and further, the identity information of the person to be analyzed is determined according to the pre-established corresponding relationship between the position and the identity information. For example, if the person to be analyzed is a student, then a fixed seat may be arranged for all students in a classroom, after the image to be processed is acquired, the seat of the student to be analyzed may be determined by the identification of the image to be processed, and further, the identity information of the student may be determined according to the correspondence between the student and the seat.
Or, the image to be processed may include a clearer face region, so that the face feature information of the person to be analyzed in the image to be processed may be extracted, and then, according to the extracted face feature information, matching is performed on the pre-stored sample face feature information of each person, and if matching is successful, the identity information of the person corresponding to the sample face feature information is used as the identity information of the person to be analyzed.
For example, continuing the above example, the face feature information of all students may be collected in advance to serve as sample face feature information, after the image to be processed is obtained, the face feature information of the student is extracted from the image to be processed, then the face feature information of the person to be analyzed is matched with the sample face information of each student, and if the matching is successful, the identity information of the student corresponding to the sample face feature information is used as the identity information of the person to be analyzed.
Further, the face image of the person to be analyzed can be acquired, so that only the feature extraction can be performed on the face image, and the identity information of the person to be analyzed can be determined, or the face image of the person to be analyzed can be directly matched with the sample face image in the pre-established sample library of the face image, and the feature extraction of the human body in the image to be processed is not needed, so that the calculated amount is reduced. The human face image of the person to be analyzed can be directly intercepted from the image to be processed; or, the face image may be a face image acquired by another image acquisition device for the person to be analyzed, which is not limited specifically.
In one implementation, if the image to be processed is captured from a panoramic image of a scene where the person to be analyzed is located, before determining the identity information of the person to be analyzed in the frame of image to be processed based on each frame of image to be processed, a face area of each person may be captured from the panoramic image corresponding to the image to be processed as a candidate face image, then, the identity information corresponding to each candidate face image is identified, and a candidate face image corresponding to the person to be analyzed in the image to be processed is determined from all candidate face images, so that the identity information of the candidate face image may be used as the identity information of the person to be analyzed.
Then, based on each frame of image to be processed, matching can be performed according to the coordinates of the candidate face image and the image to be processed in the panoramic image, and identity information of the person to be analyzed in the frame of image to be processed is determined. First, a first coordinate value of each candidate face image in the panoramic image and a second coordinate value of the image to be processed in the panoramic image are determined, and then, of the determined first coordinate values, a first coordinate value matched with the second coordinate value is determined as a target first coordinate value.
Specifically, for each first coordinate value, the distance between the second coordinate value and the first coordinate value is calculated, and then the first coordinate value with the smallest distance is used as the target first coordinate value. That is, in the panoramic image, the face image closest to the human body image of the person to be analyzed is taken as the face image of the person to be analyzed.
Or, the first coordinate value of the target may be determined according to a position relationship between the second coordinate value and the target frame corresponding to the first coordinate value, specifically, the second target frame corresponding to the target human body image and the first target frame corresponding to each first coordinate value are determined according to the second coordinate value, then, whether the first target frame corresponding to the first coordinate value is within the determined second target frame is determined, and if so, the first coordinate value is taken as the first coordinate value of the target. That is, in the panoramic image, the face image corresponding to the first target frame in the second target frame corresponding to the human body of the person to be analyzed is taken as the face image of the person to be analyzed.
S102: and aiming at each frame of image to be processed, performing behavior analysis on the person to be analyzed in the frame of image to be processed to obtain an analysis result corresponding to the frame of image to be processed.
After the at least one frame of image to be processed is acquired, behavior analysis can be performed on each person to be analyzed in the image to be processed based on the image to be processed, so that an analysis result is obtained. The analyzing method includes the steps that behavior analysis is conducted on each to-be-analyzed person in an image to be processed, whether a certain behavior exists in the to-be-analyzed person can be analyzed based on the to-be-analyzed image, and for example, if the to-be-analyzed person is a to-be-analyzed student in a classroom, the behaviors can include standing, listening and speaking, reading, writing, lifting hands, lying down on a desk, playing a mobile phone and the like.
For example, continuing the above example, behavior analysis may be performed on the student to be analyzed in each frame of the image to be processed, and the possible behavior of the student to be analyzed in the frame of the image to be processed, and the confidence and the behavior time of each behavior may be obtained.
The behavior with the highest confidence level may be determined as the behavior type of the student to be analyzed in the frame of image to be processed according to the confidence level of each behavior of the student to be analyzed in the frame of image to be processed, for example, after the frame of image to be processed is analyzed, it is obtained that the student to be analyzed may have any one of the following behaviors: standing, listening and speaking, reading, writing, lifting hands, lying on desk and playing mobile phones, and meanwhile, obtaining the confidence coefficient corresponding to each behavior, so that the behavior with the highest confidence coefficient can be used as the behavior of the student to be analyzed in the frame of image to be processed, namely the analysis result of the frame of image to be processed.
The behavior analysis model can be established in advance according to various behaviors of the person to be analyzed, and then the image to be processed is analyzed through the behavior analysis model to obtain an analysis result corresponding to the frame of image to be processed.
For example, continuing the above example, a behavior analysis model may be established for various behaviors of the student to be analyzed, such as: standing models, listening and speaking models, reading models, writing models, hand lifting models, lying desk models, mobile phone playing models, and the like. The process of performing behavior analysis on the person to be analyzed in the frame of image to be processed may include: and matching the frame of image to be processed with various behavior analysis models, and determining an analysis result corresponding to the frame of image to be processed according to the matching result.
S103: and determining behavior data of the person to be analyzed based on the analysis result corresponding to each frame of image to be processed.
In one implementation, multiple frames of images to be processed are acquired, and in this case, the behavior data of the person to be analyzed may be statistical data of various behaviors of the person to be analyzed in each frame of image to be processed.
Specifically, the analysis result corresponding to each frame of the image to be processed may be counted, so as to obtain the times of performing various behaviors in each frame of the image to be processed by the person to be analyzed, and the times are used as the behavior data of the person to be analyzed. The analysis result corresponding to each frame of the image to be processed may include a certain behavior with the highest confidence level in the frame of the image to be processed by the person to be analyzed.
For example, continuing the above example, by counting the analysis result corresponding to each frame of the to-be-processed image, the behavior data of the student to be analyzed can be obtained, that is, the times of standing, listening, speaking, reading, writing, raising hands, lying on the desk, and playing a mobile phone of the student to be analyzed in the time corresponding to each frame of the to-be-processed image can be known.
S104: and obtaining the concentration degree of the person to be analyzed according to the pre-established mapping relation between the behavior data and the concentration degree and the behavior data of the person to be analyzed.
In one implementation, the mapping relationship between the behavior data and the concentration degree may be established according to a preset training set, and the preset training set may include the sample behavior data and the concentration degree corresponding to the sample behavior data.
For example, a plurality of sample images corresponding to an evaluation time period may be obtained first, for example, in units of a class, the concentration of each student in the class needs to be evaluated, and then a plurality of sample images within the time period corresponding to the class may be obtained first; then, for each sample image, manually judging the concentration degree score of the sample person in the sample image, further, for convenience of calculation, normalizing the manually judged concentration degree score, and scaling the concentration degree score into a range of [0,1], wherein a specific normalization formula is as follows:
Figure BDA0002112448370000111
wherein, ViAn accumulated value, V, representing the concentration score of any one sample person in a plurality of sample imagesminA minimum concentration score, V, of the accumulated values of concentration scores representing all sample persons in the plurality of sample imagesmaxMaximum concentration score, Y, of the accumulated values of concentration scores representing all sample persons in the plurality of sample imagesiIndicating a normalized concentration of the sample person among the plurality of sample imagesAnd (5) degree scores.
Meanwhile, for each sample image, the sample personnel in the sample image is subjected to behavior analysis to obtain sample behavior data corresponding to the sample image, and the sample behavior data of the sample personnel in each sample image is accumulated, so that a training set is obtained. And then inputting the sample behavior data and the concentration degree corresponding to the sample behavior data into a preset model, and judging whether the output of the model is converged through a loss function, thereby obtaining a behavior analysis model.
The behavior analysis model may be a ridge regression model:
Y＝θTX
and theta is a mapping coefficient of the ridge regression model, T represents that the matrix is transposed, X is behavior data of the person to be analyzed, and Y is the concentration degree of the person to be analyzed.
Specifically, a preset training set may be trained by using the following loss function to obtain a mapping coefficient θ of the ridge regression model:
Figure BDA0002112448370000121
wherein J (θ ') is a loss value, X' is sample behavior data, Y 'is a concentration degree corresponding to the sample behavior data, T represents transposing the matrix, α is a constant coefficient, | θ' | Y2Is a ridge regression of θ ', θ' being a variable coefficient of the loss function; when J (theta') is minimum, the corresponding variable coefficient theta’The mapping coefficient theta of the ridge regression model is obtained;
in one implementation, the value of the corresponding variable coefficient θ ', i.e., the value of the mapping coefficient θ of the ridge regression model, when J (θ') is the minimum value, can be calculated by the following formula:
θ＝(X‘TX‘+E)-1X‘TY’
wherein E is an identity matrix.
Alternatively, the behavior analysis model may be a neural network model or other linear regression model, which is not limited in detail.
In addition, in another implementation manner, a mapping relationship between the behavior data and the concentration degree may also be directly established, for example, scores corresponding to various behavior types in the behavior data of the person to be analyzed may be preset, and then, the concentration degree of the person to be analyzed may be directly scored according to the behavior data. For example, continuing the above example, if it is set that the student adds one minute to his concentration when standing, listening, speaking, reading, writing, raising hands, respectively, and subtracts one minute to his concentration when the student leans on the desk and plays the cell phone, respectively, then if in the student's behavioral data, including 3 times listening and speaking and 2 times leaning on the desk, the student's concentration score is 1 minute.
Further, if the identity information of the to-be-analyzed person is determined, then, the corresponding relationship between the identity information of the to-be-analyzed person and the concentration degree of the to-be-analyzed person can be established, so that richer analysis results of the concentration degree of the to-be-analyzed person can be output, and the concentration degree of the to-be-analyzed person can be further analyzed based on the identity information of the to-be-analyzed person.
As can be seen from the above, the method for analyzing the concentration of the person provided in the embodiment of the present invention includes, first, obtaining at least one frame of to-be-processed image including an image of the person to be analyzed, performing behavior analysis on the person to be analyzed in the frame of to-be-processed image for each frame of to-be-processed image to obtain an analysis result corresponding to the frame of to-be-processed image, then, determining behavior data of the person to be analyzed based on the analysis result corresponding to each frame of to-be-processed image, and obtaining the concentration of the person to be analyzed according to a mapping relationship between the behavior data and the concentration, which is established in advance, and the behavior data of the person to be analyzed. Therefore, the concentration degree of the to-be-analyzed person in the to-be-processed image can be determined by analyzing the acquired to-be-processed image, the to-be-analyzed person does not need to actively report various behaviors of the to-be-analyzed person, and the concentration degree of the to-be-analyzed person can be automatically analyzed without depending on manual work.
As shown in fig. 2, a second flowchart of the method for analyzing concentration of a person according to the embodiment of the present invention includes the following steps:
s201: and acquiring at least one frame of to-be-processed image containing the to-be-analyzed student image.
For example, the image to be processed may be a human body image acquired for a student to be analyzed, or may also be a panoramic image acquired for a current scene, and the image to be processed may include a human body image of the student to be analyzed, or may include human body images of a plurality of students to be analyzed, which is not limited specifically.
In some scenarios, a classroom in which a student to be analyzed is located may be monitored to obtain images to be processed for the student.
Specifically, a panoramic image of a scene where a student to be analyzed is located can be acquired through image acquisition equipment, and then an area of a human body of the student to be analyzed is intercepted from the panoramic image and is used as an image to be processed; or, the student to be analyzed can be tracked through the video acquisition equipment, and the tracked image is directly used as the image to be processed.
The image acquisition device may be a network camera, a monitoring camera, a dome camera, or the like, and the image acquisition device and the electronic device (execution main body) may be the same device or different devices, and are not limited specifically.
In one implementation, after at least one frame of to-be-processed image including an image of a student to be analyzed is acquired, identity information of the student to be analyzed in the frame of to-be-processed image may be further determined based on each frame of to-be-processed image.
Specifically, for each frame of image to be processed, the image to be processed may be identified, the position of the student to be analyzed is determined according to the coordinates of the student to be analyzed in the image to be processed and the corresponding relationship between the coordinates of the student to be processed and the position in the current scene, and further, the identity information of the student to be analyzed is determined according to the pre-established corresponding relationship between the position and the identity information. For example, if the student to be analyzed is a student, a fixed seat may be arranged for all students in a classroom, after the image to be processed is acquired, the seat of the student to be analyzed may be determined by the identification of the image to be processed, and further, the identity information of the student may be determined according to the correspondence between the student and the seat.
Or the image to be processed may include a clearer face region, so that the face feature information of the student to be analyzed in the image to be processed may be extracted, and then, according to the extracted face feature information, matching is performed on the pre-stored sample face feature information of each student, and if matching is successful, the identity information of the student corresponding to the sample face feature information is used as the identity information of the student to be analyzed.
For example, continuing the above example, the face feature information of all students may be collected in advance to serve as sample face feature information, after the image to be processed is obtained, the face feature information of the student is extracted from the image to be processed, then the face feature information of the student to be analyzed is matched with the sample face information of each student, and if the matching is successful, the identity information of the student corresponding to the sample face feature information is used as the identity information of the student to be analyzed.
Furthermore, the face images of the students to be analyzed can be acquired, so that only the face images can be subjected to feature extraction to determine the identity information of the students to be analyzed, or the face images of the students to be analyzed can be directly matched with the sample face images in a sample library of the face images established in advance without performing feature extraction on the human body in the images to be processed, so that the calculated amount is reduced. The face image of the student to be analyzed can be directly intercepted from the image to be processed; or, the face image collected by the student to be analyzed may be collected by other image collection devices, which is not limited specifically.
In one implementation, if the image to be processed is captured from a panoramic image of a scene where a student to be analyzed is located, before determining the identity information of the student to be analyzed in the frame of image to be processed based on each frame of image to be processed, the face area of each person may be captured from the panoramic image corresponding to the image to be processed as a candidate face image, then the identity information corresponding to each candidate face image is identified, and a candidate face image corresponding to the student to be analyzed in the image to be processed is determined from all candidate face images, so that the identity information of the face candidate image may be used as the identity information of the student to be analyzed.
Then, based on each frame of image to be processed, matching can be performed according to the candidate face image and the coordinates of the image to be processed in the panoramic image, and identity information of students to be analyzed in the frame of image to be processed is determined. First, a first coordinate value of each candidate face image in the panoramic image and a second coordinate value of the image to be processed in the panoramic image are determined, and then, of the determined first coordinate values, a first coordinate value matched with the second coordinate value is determined as a target first coordinate value.
Specifically, for each first coordinate value, the distance between the second coordinate value and the first coordinate value is calculated, and then the first coordinate value with the smallest distance is used as the target first coordinate value. That is, in the panoramic image, the face image closest to the body image of the student to be analyzed is taken as the face image of the student to be analyzed.
Or, the first coordinate value of the target may be determined according to a position relationship between the second coordinate value and the target frame corresponding to the first coordinate value, specifically, the second target frame corresponding to the target human body image and the first target frame corresponding to each first coordinate value are determined according to the second coordinate value, then, whether the first target frame corresponding to the first coordinate value is within the determined second target frame is determined, and if so, the first coordinate value is taken as the first coordinate value of the target. That is, in the panoramic image, the face image corresponding to the first target frame within the second target frame corresponding to the human body of the student to be analyzed is taken as the face image of the student to be analyzed.
S202: aiming at each frame of image to be processed, analyzing whether the student to be analyzed in the frame of image to be processed has any one of the following behaviors: standing, listening, speaking, reading, writing, lifting hands, lying on a desk and playing a mobile phone to obtain an analysis result corresponding to the frame of image to be processed.
For example, continuing the above example, behavior analysis may be performed on the student to be analyzed in each frame of the image to be processed, and the possible behavior of the student to be analyzed in the frame of the image to be processed, and the confidence and the behavior time of each behavior may be obtained.
The behavior with the highest confidence level may be determined as the behavior type of the student to be analyzed in the frame of image to be processed according to the confidence level of each behavior of the student to be analyzed in the frame of image to be processed, for example, after the frame of image to be processed is analyzed, it is obtained that the student to be analyzed may have any one of the following behaviors: standing, listening and speaking, reading, writing, lifting hands, lying on desk and playing mobile phones, and meanwhile, obtaining the confidence coefficient corresponding to each behavior, so that the behavior with the highest confidence coefficient can be used as the behavior of the student to be analyzed in the frame of image to be processed, namely the analysis result of the frame of image to be processed.
The behavior analysis model can be established for various behaviors of the student to be analyzed, such as: standing models, listening and speaking models, reading models, writing models, hand lifting models, lying desk models, mobile phone playing models, and the like. The process of performing behavior analysis on the person to be analyzed in the frame of image to be processed may include: and matching the frame of image to be processed with various behavior analysis models, and determining an analysis result corresponding to the frame of image to be processed according to the matching result.
S203: and counting the times of various behaviors of the students to be analyzed in each frame of images to be processed to serve as behavior data of the students to be analyzed.
Specifically, the analysis result corresponding to each frame of the image to be processed may be counted, so as to obtain the times of performing various behaviors in each frame of the image to be processed by the student to be analyzed, and the times are used as the behavior data of the student to be analyzed. The analysis result corresponding to each frame of the image to be processed may include a certain behavior of the student to be analyzed with the highest confidence level in the frame of the image to be processed.
For example, continuing the above example, by counting the analysis result corresponding to each frame of the to-be-processed image, the behavior data of the student to be analyzed can be obtained, that is, the times of standing, listening, speaking, reading, writing, raising hands, lying on the desk, and playing a mobile phone of the student to be analyzed in the time corresponding to each frame of the to-be-processed image can be known.
S204: and obtaining the concentration degree of the student to be analyzed according to the pre-established behavior data, the ridge regression model of the concentration degree and the behavior data of the student to be analyzed.
In one implementation, the behavior data and the ridge regression model of the concentration degree may be established according to a preset training set, and the preset training set may include the sample behavior data and the concentration degree corresponding to the sample behavior data.
For example, a plurality of sample images corresponding to an evaluation time period may be obtained first, for example, in units of a class, the concentration of each student in the class needs to be evaluated, and then a plurality of sample images within the time period corresponding to the class may be obtained first; then, for each sample image, manually judging the concentration degree score of the sample student in the sample image, further, for convenience of calculation, normalizing the manually judged concentration degree score, and scaling the concentration degree score into a range of [0,1], wherein a specific normalization formula is as follows:
Figure BDA0002112448370000171
wherein, ViCumulative value, V, representing the concentration score of any sample student in multiple sample imagesminMinimum concentration score, V, of the accumulated values of concentration scores representing all sample students in a plurality of sample imagesmaxMaximum concentration score, Y, of the accumulated values representing concentration scores of all sample students in the plurality of sample imagesiRepresenting the normalized concentration score of the sample student in the plurality of sample images.
Meanwhile, for each sample image, the sample students in the sample image are subjected to behavior analysis to obtain sample behavior data corresponding to the sample image, and the sample behavior data of the sample students in each sample image are accumulated, so that a training set is obtained. And then, inputting the sample behavior data and the concentration degree corresponding to the sample behavior data into a preset ridge regression model, and judging whether the output of the model is converged through a loss function, so as to obtain a behavior analysis ridge regression model:
Y＝θTX
and theta is a mapping coefficient of the ridge regression model, T represents that the matrix is transposed, X is behavior data of the person to be analyzed, and Y is the concentration degree of the person to be analyzed.
Specifically, a preset training set may be trained by using the following loss function to obtain a mapping coefficient θ of the ridge regression model:
Figure BDA0002112448370000172
wherein J (θ ') is a loss value, X' is sample behavior data, Y 'is a concentration degree corresponding to the sample behavior data, T represents transposing the matrix, α is a constant coefficient, | θ' | Y2Is a ridge regression of θ ', θ' being a variable coefficient of the loss function; when J (theta ') is the minimum value, the corresponding variable coefficient theta' is the mapping coefficient theta of the ridge regression model;
in one implementation, the variable coefficient θ corresponding to the minimum value of J (θ) can be calculated by the following formula’The value of (a), i.e., the value of the mapping coefficient θ of the ridge regression model, is calculated:
θ＝(X‘TX‘+E)-1X‘TY’
wherein E is an identity matrix.
Further, if the identity information of the student to be analyzed is determined, the corresponding relationship between the identity information of the student to be analyzed and the concentration degree of the student to be analyzed can be established, so that richer analysis results of the concentration degree of the student to be analyzed can be output.
As can be seen from the above, the method for analyzing the concentration of the person provided in the embodiment of the present invention includes, first, obtaining at least one frame of to-be-processed image including an image of the person to be analyzed, performing behavior analysis on the person to be analyzed in the frame of to-be-processed image for each frame of to-be-processed image to obtain an analysis result corresponding to the frame of to-be-processed image, then, determining behavior data of the person to be analyzed based on the analysis result corresponding to each frame of to-be-processed image, and obtaining the concentration of the person to be analyzed according to a mapping relationship between the behavior data and the concentration, which is established in advance, and the behavior data of the person to be analyzed. Therefore, the concentration degree of the to-be-analyzed person in the to-be-processed image can be determined by analyzing the acquired to-be-processed image, the to-be-analyzed person does not need to actively report various behaviors of the to-be-analyzed person, and the concentration degree of the to-be-analyzed person can be automatically analyzed without depending on manual work.
An embodiment of the present invention further provides a device for analyzing concentration of people, as shown in fig. 3, which is a schematic structural diagram of the device for analyzing concentration of people provided in the embodiment of the present invention, and the device includes:
an obtaining module 301, configured to obtain at least one frame of to-be-processed image including an image of a person to be analyzed;
the analysis module 302 is configured to perform behavior analysis on a person to be analyzed in each frame of image to be processed to obtain an analysis result corresponding to the frame of image to be processed;
the determining module 303 is configured to determine behavior data of a person to be analyzed based on an analysis result corresponding to each frame of image to be processed;
the mapping module 304 is configured to obtain the concentration degree of the person to be analyzed according to the pre-established mapping relationship between the behavior data and the concentration degree and the behavior data of the person to be analyzed.
In one implementation, the apparatus further comprises:
an identity determining module 305, configured to determine identity information of a person to be analyzed in each frame of image to be processed based on the image to be processed; and establishing a corresponding relation between the identity information of the person to be analyzed and the concentration degree of the person to be analyzed.
In one implementation, the obtaining module 301 is specifically configured to obtain at least one frame of panoramic image collected for a scene where a person to be analyzed is located; intercepting an area containing a human body of a person to be analyzed from each frame of panoramic image to serve as an image to be processed corresponding to the frame of panoramic image;
the device also includes:
an identity information recognition module (not shown in the figure) for intercepting each face region from the panoramic image corresponding to the image to be processed as a candidate face image; determining a first coordinate value of each candidate face image in a panoramic image corresponding to the image to be processed, and identifying identity information corresponding to the candidate face image;
an identity determining module 305, configured to determine a second coordinate value of the to-be-processed image in the corresponding panoramic image; determining a first coordinate value matched with the second coordinate value as a target first coordinate value in the determined first coordinate values; and determining the identity information corresponding to the target first coordinate value as the identity information of the person to be analyzed.
In one implementation, the mapping module 304 is specifically configured to:
inputting the behavior data of the personnel to be analyzed into a behavior analysis model obtained by pre-training, and performing regression analysis processing to obtain the concentration degree of the personnel to be analyzed;
the behavior analysis model is obtained by training based on a preset training set, and the preset training set comprises sample behavior data and concentration degrees corresponding to the sample behavior data.
In one implementation, the behavior analysis model is a ridge regression model:
Y＝θTX
wherein θ is a mapping coefficient of the ridge regression model, T represents transposing the matrix, X is behavior data of the person to be analyzed, and Y is concentration of the person to be analyzed;
the mapping module 304 is specifically configured to:
training a preset training set by adopting the following loss function to obtain a mapping coefficient theta of the ridge regression model:
Figure BDA0002112448370000201
wherein J (θ ') is a loss value, X' is sample behavior data, Y 'is a concentration degree corresponding to the sample behavior data, T represents transposing the matrix, α is a constant coefficient, | θ' | Y2Ridge regression for θ'; θ ' is a variable coefficient of the loss function, and the corresponding variable coefficient θ ' when J (θ ') is the minimum value is used as the mapping coefficient θ of the ridge regression model.
In one implementation, the person to be analyzed is a student to be analyzed;
the analysis module 302 is specifically configured to, for each frame of to-be-processed image, analyze whether a student to be analyzed in the frame of to-be-processed image has any one of the following behaviors: standing, listening, speaking, reading, writing, lifting hands, lying down a desk and playing a mobile phone to obtain an analysis result corresponding to the frame of image to be processed;
the determining module 303 is specifically configured to count the times of various behaviors of the student to be analyzed in each frame of the image to be processed, and use the times as behavior data of the student to be analyzed.
As can be seen from the above, in the apparatus for analyzing concentration of a person provided in the embodiment of the present invention, at least one frame of to-be-processed image including an image of the person to be analyzed is obtained, behavior analysis is performed on the person to be analyzed in the frame of to-be-processed image for each frame of to-be-processed image, an analysis result corresponding to the frame of to-be-processed image is obtained, then, behavior data of the person to be analyzed is determined based on the analysis result corresponding to each frame of to-be-processed image, and the concentration of the person to be analyzed is obtained according to a mapping relationship between the behavior data and the concentration that is established in advance and the behavior data of the person to be analyzed. Therefore, the concentration degree of the to-be-analyzed person in the to-be-processed image can be determined by analyzing the acquired to-be-processed image, the to-be-analyzed person does not need to actively report various behaviors of the to-be-analyzed person, and the concentration degree of the to-be-analyzed person can be automatically analyzed without depending on manual work.
An embodiment of the present invention further provides an electronic device, as shown in fig. 4, including a processor 401, a communication interface 402, a memory 403, and a communication bus 404, where the processor 401, the communication interface 402, and the memory 403 complete mutual communication through the communication bus 404,
a memory 403 for storing a computer program;
the processor 401, when executing the program stored in the memory 403, implements the following steps:
acquiring at least one frame of to-be-processed image containing an image of a person to be analyzed;
aiming at each frame of image to be processed, performing behavior analysis on the person to be analyzed in the frame of image to be processed to obtain an analysis result corresponding to the frame of image to be processed;
determining behavior data of a person to be analyzed based on an analysis result corresponding to each frame of image to be processed;
and obtaining the concentration degree of the person to be analyzed according to the pre-established mapping relation between the behavior data and the concentration degree and the behavior data of the person to be analyzed.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
As can be seen from the above, the electronic device provided in the embodiment of the present invention can determine the concentration degree of the to-be-analyzed person in the to-be-processed image by analyzing the acquired to-be-processed image, and does not need to actively report various behaviors of the to-be-analyzed person, so that the concentration degree of the to-be-analyzed person can be automatically analyzed without depending on manual work.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, the electronic device embodiment and the storage medium embodiment, since they are substantially similar to the method embodiment, the description is relatively simple, and in relation to the description, reference may be made to some portions of the description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims (14)

1. A method of human concentration analysis, the method comprising:
acquiring at least one frame of to-be-processed image containing an image of a person to be analyzed;
aiming at each frame of image to be processed, performing behavior analysis on the person to be analyzed in the frame of image to be processed to obtain an analysis result corresponding to the frame of image to be processed;
determining behavior data of the person to be analyzed based on the analysis result corresponding to each frame of image to be processed;
and obtaining the concentration degree of the person to be analyzed according to the pre-established mapping relation between the behavior data and the concentration degree and the behavior data of the person to be analyzed.
2. The method of claim 1, wherein after said acquiring at least one frame of a to-be-processed image containing an image of a person to be analyzed, the method further comprises:
determining the identity information of the personnel to be analyzed in each frame of image to be processed based on each frame of image to be processed;
and establishing a corresponding relation between the identity information of the person to be analyzed and the concentration degree of the person to be analyzed.
3. The method of claim 2, wherein said obtaining at least one frame of the image to be processed including the image of the person to be analyzed comprises:
acquiring at least one frame of panoramic image acquired aiming at a scene where a person to be analyzed is located;
intercepting an area containing the human body of the person to be analyzed from each frame of panoramic image to serve as an image to be processed corresponding to the frame of panoramic image;
before determining the identity information of the person to be analyzed in each frame of image to be processed based on each frame of image to be processed, the method further comprises:
intercepting each face area from the panoramic image corresponding to the image to be processed as a candidate face image;
determining a first coordinate value of each candidate face image in the panoramic image corresponding to the image to be processed, and identifying identity information corresponding to the candidate face image;
the determining the identity information of the person to be analyzed in each frame of image to be processed based on each frame of image to be processed comprises:
determining a second coordinate value of the image to be processed in the corresponding panoramic image;
determining a first coordinate value matched with the second coordinate value as a target first coordinate value in the determined first coordinate values;
and determining the identity information corresponding to the target first coordinate value as the identity information of the person to be analyzed.
4. The method according to claim 1, wherein the obtaining the concentration degree of the person to be analyzed according to the pre-established mapping relationship between the behavior data and the concentration degree and the behavior data of the person to be analyzed comprises:
inputting the behavior data of the person to be analyzed into a behavior analysis model obtained by pre-training, and performing regression analysis processing to obtain the concentration degree of the person to be analyzed;
the behavior analysis model is obtained by training based on a preset training set, and the preset training set comprises sample behavior data and concentration degrees corresponding to the sample behavior data.
5. The method of claim 4, wherein the behavior analysis model is a ridge regression model:
Y＝θTX
wherein θ is a mapping coefficient of the ridge regression model, T represents transposing a matrix, X is behavior data of the person to be analyzed, and Y is concentration of the person to be analyzed;
training the preset training set by adopting the following loss function to obtain a mapping coefficient theta of the ridge regression model:
Figure FDA0002112448360000021
wherein J (θ ') is a loss value, X' is the sample behavior data, Y 'is the concentration corresponding to the sample behavior data, T represents transposing the matrix, α is a constant coefficient, | θ' | magnetism2Ridge regression for θ'; and theta ' is a variable coefficient of the loss function, and the corresponding variable coefficient theta ' when the J (theta ') is a minimum value is used as a mapping coefficient theta of the ridge regression model.
6. The method of claim 1, wherein the person to be analyzed is a student to be analyzed; the method for analyzing the behavior of the person to be analyzed in each frame of image to be processed to obtain the analysis result corresponding to the frame of image to be processed includes:
aiming at each frame of image to be processed, analyzing whether the student to be analyzed in the frame of image to be processed has any one of the following behaviors: standing, listening, speaking, reading, writing, lifting hands, lying down a desk and playing a mobile phone to obtain an analysis result corresponding to the frame of image to be processed;
the determining the behavior data of the person to be analyzed based on the analysis result corresponding to each frame of image to be processed includes:
and counting the times of various behaviors of the student to be analyzed in each frame of image to be processed as behavior data of the student to be analyzed.
7. A person concentration degree analysis apparatus, characterized in that the apparatus comprises:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring at least one frame of image to be processed containing an image of a person to be analyzed;
the analysis module is used for performing behavior analysis on the person to be analyzed in each frame of image to be processed to obtain an analysis result corresponding to the frame of image to be processed;
the determining module is used for determining the behavior data of the person to be analyzed based on the analysis result corresponding to each frame of image to be processed;
and the mapping module is used for obtaining the concentration degree of the person to be analyzed according to the pre-established mapping relation between the behavior data and the concentration degree and the behavior data of the person to be analyzed.
8. The apparatus of claim 7, further comprising:
the identity determining module is used for determining the identity information of the person to be analyzed in each frame of image to be processed based on each frame of image to be processed; and establishing a corresponding relation between the identity information of the person to be analyzed and the concentration degree of the person to be analyzed.
9. The apparatus of claim 8,
the acquisition module is specifically used for acquiring at least one panoramic image acquired aiming at a scene where a person to be analyzed is located; intercepting an area containing the human body of the person to be analyzed from each frame of panoramic image to serve as an image to be processed corresponding to the frame of panoramic image;
the device further comprises:
the identity information identification module is used for intercepting each face area from the panoramic image corresponding to the image to be processed as a candidate face image; determining a first coordinate value of each candidate face image in the panoramic image corresponding to the image to be processed, and identifying identity information corresponding to the candidate face image;
the identity determination module is specifically configured to determine a second coordinate value of the to-be-processed image in the corresponding panoramic image; determining a first coordinate value matched with the second coordinate value as a target first coordinate value in the determined first coordinate values; and determining the identity information corresponding to the target first coordinate value as the identity information of the person to be analyzed.
10. The apparatus of claim 7, wherein the mapping module is specifically configured to:
inputting the behavior data of the person to be analyzed into a behavior analysis model obtained by pre-training, and performing regression analysis processing to obtain the concentration degree of the person to be analyzed;
the behavior analysis model is obtained by training based on a preset training set, and the preset training set comprises sample behavior data and concentration degrees corresponding to the sample behavior data.
11. The apparatus of claim 10, wherein the behavior analysis model is a ridge regression model:
Y＝θTX
wherein θ is a mapping coefficient of the ridge regression model, T represents transposing a matrix, X is behavior data of the person to be analyzed, and Y is concentration of the person to be analyzed;
the mapping module is specifically configured to:
training the preset training set by adopting the following loss function to obtain a mapping coefficient theta of the ridge regression model:
Figure FDA0002112448360000041
wherein J (θ ') is a loss value, X' is the sample behavior data, Y 'is the concentration corresponding to the sample behavior data, T represents transposing the matrix, α is a constant coefficient, | θ' | magnetism2Ridge regression for θ'; and theta ' is a variable coefficient of the loss function, and the corresponding variable coefficient theta ' when the J (theta ') is a minimum value is used as a mapping coefficient theta of the ridge regression model.
12. The apparatus of claim 7, wherein the person to be analyzed is a student to be analyzed;
the analysis module is specifically configured to analyze, for each frame of to-be-processed image, whether a student to be analyzed in the frame of to-be-processed image has any one of the following behaviors: standing, listening, speaking, reading, writing, lifting hands, lying down a desk and playing a mobile phone to obtain an analysis result corresponding to the frame of image to be processed;
the determining module is specifically configured to count the times of various behaviors of the student to be analyzed in each frame of image to be processed, and use the counted times as behavior data of the student to be analyzed.
13. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of claims 1-6 when executing a program stored in the memory.
14. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 6.
CN201910577403.5A 2019-06-28 2019-06-28 Personnel concentration analysis method and device Active CN111325082B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910577403.5A CN111325082B (en) 2019-06-28 2019-06-28 Personnel concentration analysis method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910577403.5A CN111325082B (en) 2019-06-28 2019-06-28 Personnel concentration analysis method and device

Publications (2)

Publication Number Publication Date
CN111325082A true CN111325082A (en) 2020-06-23
CN111325082B CN111325082B (en) 2024-02-02

Family

ID=71169114

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910577403.5A Active CN111325082B (en) 2019-06-28 2019-06-28 Personnel concentration analysis method and device

Country Status (1)

Country Link
CN (1) CN111325082B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112528890A (en) * 2020-12-15 2021-03-19 北京易华录信息技术股份有限公司 Attention assessment method and device and electronic equipment
CN112613342A (en) * 2020-11-27 2021-04-06 深圳市捷视飞通科技股份有限公司 Behavior analysis method and apparatus, computer device, and storage medium
CN113657152A (en) * 2021-07-07 2021-11-16 国网江苏省电力有限公司电力科学研究院 Classroom student behavior recognition system construction method
CN113783709A (en) * 2021-08-31 2021-12-10 深圳市易平方网络科技有限公司 Conference system-based participant monitoring and processing method and device and intelligent terminal

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050093697A1 (en) * 2003-11-05 2005-05-05 Sanjay Nichani Method and system for enhanced portal security through stereoscopy
JP2013156718A (en) * 2012-01-27 2013-08-15 National Institute Of Advanced Industrial & Technology Person tracking attribute estimation device, person tracking attribute estimation method and program
CN106250822A (en) * 2016-07-21 2016-12-21 苏州科大讯飞教育科技有限公司 Student's focus based on recognition of face monitoring system and method
CN109740446A (en) * 2018-12-14 2019-05-10 深圳壹账通智能科技有限公司 Classroom students ' behavior analysis method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050093697A1 (en) * 2003-11-05 2005-05-05 Sanjay Nichani Method and system for enhanced portal security through stereoscopy
JP2013156718A (en) * 2012-01-27 2013-08-15 National Institute Of Advanced Industrial & Technology Person tracking attribute estimation device, person tracking attribute estimation method and program
CN106250822A (en) * 2016-07-21 2016-12-21 苏州科大讯飞教育科技有限公司 Student's focus based on recognition of face monitoring system and method
CN109740446A (en) * 2018-12-14 2019-05-10 深圳壹账通智能科技有限公司 Classroom students ' behavior analysis method and device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
DIPANKAR DAS； ET AL: "Supporting Human-Robot Interaction Based on the Level of Visual Focus of Attention" *
徐佳程: "基于SEM的大学生移动学习能力影响模型研究" *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112613342A (en) * 2020-11-27 2021-04-06 深圳市捷视飞通科技股份有限公司 Behavior analysis method and apparatus, computer device, and storage medium
CN112528890A (en) * 2020-12-15 2021-03-19 北京易华录信息技术股份有限公司 Attention assessment method and device and electronic equipment
CN112528890B (en) * 2020-12-15 2024-02-13 北京易华录信息技术股份有限公司 Attention assessment method and device and electronic equipment
CN113657152A (en) * 2021-07-07 2021-11-16 国网江苏省电力有限公司电力科学研究院 Classroom student behavior recognition system construction method
CN113783709A (en) * 2021-08-31 2021-12-10 深圳市易平方网络科技有限公司 Conference system-based participant monitoring and processing method and device and intelligent terminal
CN113783709B (en) * 2021-08-31 2024-03-19 重庆市易平方科技有限公司 Conference participant monitoring and processing method and device based on conference system and intelligent terminal

Also Published As

Publication number Publication date
CN111325082B (en) 2024-02-02

Similar Documents

Publication Publication Date Title
CN109522815B (en) Concentration degree evaluation method and device and electronic equipment
CN111325082B (en) Personnel concentration analysis method and device
CN110659397B (en) Behavior detection method and device, electronic equipment and storage medium
CN110443110B (en) Face recognition method, device, terminal and storage medium based on multipath camera shooting
CN111046819B (en) Behavior recognition processing method and device
CN110969045B (en) Behavior detection method and device, electronic equipment and storage medium
CN110837795A (en) Teaching condition intelligent monitoring method, device and equipment based on classroom monitoring video
CN112418009B (en) Image quality detection method, terminal equipment and storage medium
CN110210301B (en) Method, device, equipment and storage medium for evaluating interviewee based on micro-expression
CN111339801B (en) Personnel attention detection method, device, equipment and system
CN110287862B (en) Anti-candid detection method based on deep learning
CN110941992B (en) Smile expression detection method and device, computer equipment and storage medium
CN111368808A (en) Method, device and system for acquiring answer data and teaching equipment
CN112819665A (en) Classroom state evaluation method and related device and equipment
CN111291627B (en) Face recognition method and device and computer equipment
CN111382655A (en) Hand-lifting behavior identification method and device and electronic equipment
CN113269903A (en) Face recognition class attendance system
CN113657509B (en) Teaching training lifting method, device, terminal and storage medium
CN110443122B (en) Information processing method and related product
CN115937971B (en) Method and device for identifying hand-lifting voting
CN112528797B (en) Question recommending method and device and electronic equipment
CN113259734B (en) Intelligent broadcasting guide method, device, terminal and storage medium for interactive scene
CN114998440A (en) Multi-mode-based evaluation method, device, medium and equipment
CN112149451B (en) Affinity analysis method and device
CN112766150A (en) School classroom student learning behavior tracking analysis method based on big data and artificial intelligence and cloud management platform

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant