CN112001944A - Classroom teaching quality evaluation data acquisition method, computer equipment and medium - Google Patents
Classroom teaching quality evaluation data acquisition method, computer equipment and medium Download PDFInfo
- Publication number
- CN112001944A CN112001944A CN202010656229.6A CN202010656229A CN112001944A CN 112001944 A CN112001944 A CN 112001944A CN 202010656229 A CN202010656229 A CN 202010656229A CN 112001944 A CN112001944 A CN 112001944A
- Authority
- CN
- China
- Prior art keywords
- target
- image frame
- student
- teacher
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 26
- 238000001514 detection method Methods 0.000 claims abstract description 47
- 238000011156 evaluation Methods 0.000 claims abstract description 41
- 238000003860 storage Methods 0.000 claims abstract description 8
- 241001310793 Podium Species 0.000 claims description 25
- 238000009826 distribution Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 238000001303 quality assessment method Methods 0.000 claims description 9
- 230000000630 rising effect Effects 0.000 claims description 8
- 238000002372 labelling Methods 0.000 claims description 6
- PICXIOQBANWBIZ-UHFFFAOYSA-N zinc;1-oxidopyridine-2-thione Chemical class [Zn+2].[O-]N1C=CC=CC1=S.[O-]N1C=CC=CC1=S PICXIOQBANWBIZ-UHFFFAOYSA-N 0.000 claims description 3
- 230000009471 action Effects 0.000 abstract description 19
- 230000006399 behavior Effects 0.000 abstract description 15
- 239000000284 extract Substances 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 7
- 238000013145 classification model Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000013136 deep learning model Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 238000012937 correction Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000007635 classification algorithm Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Educational Administration (AREA)
- Tourism & Hospitality (AREA)
- Economics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Educational Technology (AREA)
- General Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- Marketing (AREA)
- Development Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Game Theory and Decision Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Operations Research (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to a method for acquiring classroom teaching quality evaluation data, computer equipment and a storage medium, wherein the method for acquiring classroom teaching quality evaluation data comprises the following steps: acquiring an image frame sequence of a classroom; detecting and tracking a human target from an image frame sequence to obtain tracking information of the human target; extracting evaluation data for evaluating the classroom teaching quality from the tracking information; the problems that the detection mode of the teacher-student classroom behaviors is single, the detection results of the teacher-student classroom behaviors are inaccurate, and the judgment of teaching quality by the detection results is inaccurate are solved, and the classroom actions of teachers and students and the positions of teachers are tracked to further extract evaluation data for evaluating the classroom teaching quality.
Description
Technical Field
The present application relates to the field of computer vision technology, and in particular, to a method, computer device, and medium for acquiring classroom teaching quality assessment data.
Background
The quality of the teaching directly affects the learning achievement of the students. With the steady advance of the education cause of China, more and more parents pay attention to the teaching quality of teachers and the classroom interaction of children. How to quickly and intuitively evaluate the teaching quality of teachers and grasp the classroom learning enthusiasm of children all the time, which puts higher requirements on teaching scores.
In order to judge the classroom teaching quality, a classroom teaching quality evaluation method is characterized in that a pressure sensor is arranged on a seat of a student, the sitting state or the standing state of the student is judged according to the pressure value change of the pressure sensor, and the state information is bound, uploaded and stored according to the seat number, so that the activity degree of answering questions by the student is evaluated. Meanwhile, the device and the method for evaluating the teaching quality of the ideological and political classes are used for reflecting the class quality of the side face by acquiring the audio information of teachers and students in real time and counting the time of audio. Furthermore, the classroom teaching quality evaluation system based on the machine vision acquires and processes image information of the class listening state of students in a classroom by using a camera and image analysis equipment by utilizing the function of the machine vision, acquires student sign-in information and student class listening attention concentration condition information, establishes a classroom teaching quality evaluation model, and calculates classroom teaching quality average index.
The technical scheme for judging the classroom teaching quality in the patent and the prior art has the problems that the detection mode and the detection object of the teacher-student classroom behavior are single, the teacher-student classroom behavior detection result has great uncertainty, and the teacher-student classroom behavior detection result is difficult to judge the teaching quality and guide the teaching. Meanwhile, the problem of difficulty in recurrence of classroom teaching also exists.
At present, no effective solution is provided aiming at the problems of single detection mode and detection object of teacher and student classroom behavior, inaccurate detection result of teacher and student classroom behavior and inaccurate judgment of teaching quality of the detection result in the related technology.
Disclosure of Invention
The embodiment of the application provides a classroom teaching quality evaluation data acquisition method, computer equipment and a medium, and aims to at least solve the problems that in the related technology, the detection mode and detection object of teacher and student classroom behaviors are single, the detection result of teacher and student classroom behaviors is inaccurate, and the judgment of teaching quality by the detection result is inaccurate.
In a first aspect, an embodiment of the present application provides a method for acquiring classroom teaching quality evaluation data, including: acquiring an image frame sequence of a classroom; detecting and tracking a human target from the image frame sequence to obtain tracking information of the human target; wherein the character goals include a teacher goal and a student goal; and extracting evaluation data for evaluating the classroom teaching quality from the tracking information.
In some of these embodiments, the classroom includes a non-podium area and a podium area; detecting and tracking a human target from the image frame sequence, wherein obtaining tracking information of the human target comprises: segmenting a first image frame sequence corresponding to the non-platform area and a second image frame sequence corresponding to the platform area from the image frame sequences; the human target is detected and tracked from each image frame of the first image frame sequence and each image frame of the second image frame sequence respectively, the human target detected and tracked from each image frame of the first image frame sequence is determined to be a student target, and the human target detected and tracked from each image frame of the second image frame sequence is determined to be a teacher target.
In some of these embodiments, the podium area does not include a video presentation area.
In some of these embodiments, the tracking information includes location information; the step of extracting evaluation data for evaluating the classroom teaching quality from the tracking information comprises the following steps: extracting the tracking information of the teacher target from the tracking information; and generating position distribution cloud picture information of the teacher target according to the position information of the teacher target, wherein the evaluation data comprises the position distribution cloud picture information of the teacher target.
In some embodiments, a human target is detected and tracked from the image frame sequence, and tracking information of the human target is obtained; extracting evaluation data for evaluating the classroom teaching quality from the tracking information comprises the following steps: performing target detection on each image frame of the first image frame sequence, and generating tracking information of the human target, wherein the tracking information includes: a target frame and a tracking identifier for marking the character target; determining the teacher target in the character targets according to the tracking identification, and inputting an image corresponding to a target frame of the teacher target in each image frame of the first image frame sequence into a state classification module to obtain a classification result of whether the teacher target is in a preset state, wherein the preset state comprises: a board writing state and/or a back-to-board writing state; recording a writing event of the teacher target in a case where it is detected from a plurality of consecutive image frames in the first image frame sequence that the teacher target is in the preset state, wherein the evaluation data includes the writing event of the teacher target.
In some embodiments, a human target is detected and tracked from the image frame sequence, and tracking information of the human target is obtained; extracting evaluation data for evaluating the classroom teaching quality from the tracking information comprises the following steps: performing target detection on each image frame of the second image frame sequence, and generating tracking information of the human target, wherein the tracking information includes: a target frame and a tracking identifier for marking the character target; determining the student targets in the character targets according to the tracking identification, and respectively inputting images corresponding to target frames of the student targets in each image frame of the second image frame sequence into a height calculation module to obtain height information of the student targets in each image frame; recording a rising event of the student object in a case where a variation amount of the height information of the student object is detected to be greater than a preset threshold from adjacent image frames in the second image frame sequence, wherein the evaluation data includes the rising event of the student object.
In some of these embodiments, the target boxes of the student targets comprise a first target box for labeling the head and shoulders of the student targets and a second target box for labeling the human body below the shoulders of the student targets; wherein the height calculation module calculates height information of the student object based on the center coordinates of the first object frame.
In some of these embodiments, after recording the student goal's standing event, the method further comprises: and sending the image frame corresponding to the standing event to an object associated with the student object.
In some of these embodiments, the method further comprises: acquiring a left eye image sequence and a right eye image sequence of the classroom through a binocular camera, wherein a left eye image in the left eye image sequence and a corresponding right eye image in the right eye image sequence are shot synchronously; determining the spatial depth information of the corresponding student targets according to the parallax of the same student targets in the left eye image and the right eye image;
respectively inputting images corresponding to the target frame of each student target in each image frame of the image frame sequence into a height calculation module, and obtaining the height information of each student target in each image frame comprises the following steps: and inputting the image corresponding to the target frame of each student target in each image frame of the image frame sequence and the corresponding spatial depth information of the students into the height calculation module to obtain the real height information of each student target in each image frame.
In a second aspect, an embodiment of the present application provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the processor implements the method for acquiring classroom teaching quality evaluation data as described in the first aspect.
In a third aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements the method for obtaining classroom teaching quality assessment data as described in the first aspect above.
Compared with the related technology, the method for acquiring classroom teaching quality evaluation data, the computer equipment and the storage medium provided by the embodiment of the application acquire the image frame sequence of the classroom; then, detecting and tracking a human target from the image frame sequence to obtain tracking information of the human target; and finally, extracting evaluation data for evaluating the classroom teaching quality from the tracking information, solving the problems of single detection mode of teacher and student classroom behaviors, inaccurate detection results of teacher and student classroom behaviors and inaccurate judgment of teaching quality by the detection results in the related technology, and realizing the extraction of the evaluation data for evaluating the classroom teaching quality by tracking classroom actions of teachers and students and positions of teachers.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a flowchart of a method for acquiring classroom teaching quality evaluation data according to an embodiment of the present application;
fig. 2 is a distribution diagram of classroom interior regions for classroom teaching according to an embodiment of the present application;
FIG. 3 is a structural view of a target location information identifier in an embodiment of the present application;
FIG. 4 is a flow diagram of target action recognition according to an embodiment of the present application;
fig. 5 is a structural diagram of an apparatus for acquiring classroom teaching quality evaluation data according to an embodiment of the present application;
fig. 6 is an internal configuration diagram of the computer device of the present embodiment.
Detailed Description
In order to make the purpose, technical solution and advantages of the present application more apparent, the present application will be described and illustrated with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application.
It is obvious that the drawings in the following description are only examples or embodiments of the present application, and that it is also possible for a person skilled in the art to apply the present application to other similar contexts on the basis of these drawings without inventive effort. Moreover, it should be further appreciated that such a development effort might be complex and tedious, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure, and it should be understood that such a development effort might be complex and tedious.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The use of the terms "including," "comprising," "having," and any variations thereof herein, is intended to cover a non-exclusive inclusion; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include additional steps or elements not listed, or may include additional steps or elements inherent to such process, method, article, or apparatus. Reference to "connected," "coupled," and the like in this application is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as referred to herein means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. Reference herein to the terms "first," "second," "third," and the like, are merely to distinguish similar objects and do not denote a particular ordering for the objects.
The various techniques described in this application may be used in various target detection, target classification, and behavior recognition systems and apparatuses.
The embodiment provides a method for acquiring classroom teaching quality evaluation data. Fig. 1 is a flowchart of a method for acquiring classroom teaching quality evaluation data according to an embodiment of the present application, and as shown in fig. 1, the flowchart includes the following steps:
step S101, acquiring an image frame sequence of a classroom.
In this embodiment, after acquiring a sequence of image frames in a classroom in which teaching is performed by a binocular camera, the sequence of image frames is preprocessed, specifically, the binocular camera acquires a video image (that is, a sequence of image frames) of the classroom in which teaching is performed, where the video image includes a left eye original image and a right eye original image, and then performs stereo correction, stereo matching, and the like on the video image to obtain a disparity map (including a left disparity map and a right disparity map), the disparity map is converted to obtain a depth map, and the left disparity map, the right disparity map, and the depth map are used to subsequently calculate the position of a character target; in this embodiment, a semiglobal block matching (SGBM) algorithm for calibrating the gnomon camera and performing binocular stereo matching is used in the preprocessing. In the present embodiment, the acquired image frame sequence of the classroom includes a first image frame sequence corresponding to the lecture area and a second image frame sequence corresponding to the non-lecture area.
Step S102, detecting and tracking a human target from an image frame sequence to obtain tracking information of the human target; wherein the character goals include a teacher goal and a student goal.
In the present embodiment, the image frame sequence for human target detection and tracking is selected from the image frame sequences of the left eye image. Meanwhile, the detection and tracking of the human target is performed by a deep learning algorithm, preferably a yolo3 deep learning model, which detects tracking position information of the human target in the sequence of the output classroom image frames, wherein the human target tracking information includes position information indicating the position of the head, shoulders and body below the shoulders of the human target played in the teacher, student and PPT in the classroom. Meanwhile, in the present embodiment, a target tracking algorithm is used to perform target tracking on a human target in an image frame sequence, so as to obtain tracking information of the human target. In this embodiment, the second image frame sequence is selected as the image frame sequence for completing the student target tracking, and the first image frame sequence is selected as the image frame sequence for completing the teacher target tracking.
And step S103, extracting evaluation data for evaluating the classroom teaching quality from the tracking information.
In the embodiment, evaluation data of a teacher target is acquired at least from two dimensions of a position cloud picture and a writing event so as to acquire evaluation data of a student target at least from a standing event; the three-dimensional position of the student target is reversely deduced through the pixel of the student target by adopting the inverse operation of camera calibration, so that the position coordinate of the student target in a world coordinate system is obtained, and one of evaluation data for evaluating the classroom teaching quality, namely the standing event of the student target, is obtained through the height difference of the position coordinate; classifying the action states of the teacher targets by using the deep learning model, outputting the action states of the teacher targets, and acquiring another evaluation data for evaluating the classroom teaching quality, namely a writing event of the teacher targets; and judging whether the teacher is in the set teaching area or not through the tracking information of the teacher target, and acquiring another kind of evaluation data for evaluating the teaching quality of the classroom, namely the position distribution cloud picture information of the teacher target.
Through the steps from S101 to S103, acquiring an image frame sequence of a classroom; detecting and tracking a person target from the image frame sequence to obtain tracking information of the person target; extracting evaluation data for evaluating the classroom teaching quality from the tracking information; the problems that in the related technology, the detection of the action behaviors of teachers and students depends on the accuracy of sensors, the action states of the teachers and students cannot be correctly shaped, and the behaviors of the students and the teachers are single in detection method, inaccurate in behavior detection and single in detected teacher or student targets are solved; the method and the device realize the purpose of carrying out target detection and tracking processing on the figure targets in the image frames to obtain tracking information by acquiring the real-time image frame sequence of the classroom teaching, and extract evaluation data representing the teaching quality from the tracking information through a preset judgment rule to further evaluate the teaching quality of each classroom.
In some of these embodiments, the classroom includes a non-podium area and a podium area; the method for detecting and tracking the human target from the image frame sequence to obtain the tracking information of the human target comprises the following steps:
step 11, a first image frame sequence corresponding to a non-platform area and a second image frame sequence corresponding to a platform area are separated from the image frame sequence.
And 12, respectively detecting and tracking the human target from each image frame of the first image frame sequence and each image frame of the second image frame sequence, determining that the human target detected and tracked from each image frame of the first image frame sequence is a student target, and determining that the human target detected and tracked from each image frame of the second image frame sequence is a teacher target.
Through the above steps 11 to 12, the first image frame sequence corresponding to the non-podium region and the second image frame sequence corresponding to the podium region are divided from the image frame sequences, the person target is detected and tracked from each image frame of the first image frame sequence and each image frame of the second image frame sequence, the person target detected and tracked from each image frame of the first image frame sequence is determined as the student target, and the person target detected and tracked from each image frame of the second image frame sequence is determined as the teacher target, so that the student target located in the non-podium region and the teacher target located in the podium region are detected from the image frames.
In some of these embodiments, the first sequence of image frames corresponding to the podium area is segmented from the sequence of image frames to define the podium area as excluding the video presentation area.
It should be noted that, in this embodiment, the podium area is configured with a video display area and a podium detection area, the video display area is used for filtering out a person target in a video display (PPT) during person target tracking, the filtering out is to shield the person target (person in video) in the area and not track the person target in the area, and the podium detection area is used for tracking only the person target in the podium area (only tracking a teacher target), and the target outside the area is not tracked, so that the time consumption of tracking is reduced, and the overall time consumption is reduced.
In some embodiments, the tracking information comprises position information, and extracting evaluation data for evaluating classroom teaching quality from the tracking information comprises the following steps:
and step 21, extracting the tracking information of the teacher target from the tracking information.
In this embodiment, the target tracking object only considers the teacher target, and the trajectory of the teacher target can be obtained from the tracking information of the teacher target, thereby determining the position of the teacher target. For example: the teacher target is located in an image frame of the second image frame sequence. When the position information of the teacher object indicates that the teacher object is not located in the image frames of the second image frame sequence, it indicates that the teacher object is in the classroom.
And step 22, generating position distribution cloud picture information of the teacher target according to the position information of the teacher target, wherein the evaluation data comprises the position distribution cloud picture information of the teacher target.
The location distribution cloud map information in this embodiment may represent specific locations where teachers have more targets in a podium and a blackboard area (teaching area) in a certain classroom teaching. In this embodiment, the location distribution cloud map information is characterized as follows: 1. the cloud pictures of the positions of the teacher target are uniformly distributed, so that the teacher repeatedly goes back and forth in front of the blackboard, and the teacher activity is described; 2. if the position cloud picture indicates that the teacher is always more at the platform, the time for the teacher to attend the class is longer, and the time for writing on a blackboard (writing on a blackboard) is shorter; 3. if the location cloud shows that the teacher is more out of the vicinity of the blackboard, the teacher often goes to the classroom to assist more students.
Through the steps 21 to 22, the method extracts the tracking information of the teacher target from the tracking information and generates the position distribution cloud picture information of the teacher target according to the position information of the teacher target, so that the motion trail of the teacher target is obtained from the image frames, the teaching process of the teacher target is judged through the trail of the teacher target, and the teaching quality is evaluated.
In some embodiments, a human target is detected and tracked from an image frame sequence, and tracking information of the human target is obtained; the method for extracting the evaluation data for evaluating the classroom teaching quality from the tracking information comprises the following steps:
step 31, performing target detection on each image frame of the first image frame sequence, and generating tracking information of a human target, where the tracking information includes: and the target frame is used for marking the character target and the tracking identification.
The tracking of the human target in this embodiment is a target frame of the tracked human target. In this embodiment, the target frame includes a head-shoulder frame and a body frame, fig. 3 is a view of a target position information identification structure in the embodiment of the present application, the position is represented by a picture pixel coordinate, and the two points form a rectangle by an upper left corner position and a lower right corner position. As shown in fig. 3, the first pixel point at the upper left corner of the picture is taken as the origin of coordinates, the coordinates are (0,0,0,0), R1 and T1 are the coordinates of the upper left corner of the human body and the upper left corner of the head and shoulder, respectively, and R2 and Y2 are the coordinates of the lower right corner of the human body and the lower right corner of the head and shoulder, respectively. The positions of the targets are head-shoulder frames and body frames, such as the frames shown by K1 and K2 in FIG. 3. In the present embodiment, the image frame sequence in which the teacher's target tracking is completed constitutes a first image frame sequence, and thus the tracking of the teacher's target is completed in the first image frame sequence.
In this embodiment, after the human target is detected, tracking information is generated, where the tracking information includes target frames and tracking identifiers corresponding to each target frame, and the tracking identifiers are used to maintain the human target information during subsequent determination, and if the same target frame is always in an image frame, the tracking identifiers are unchanged. The follow-up maintenance target information is obtained by identifying the tracking identifier, and the tracking identifier is unchanged and represents a target frame needing to be acquired and/or tracked.
Step 32, determining a teacher target in the character target according to the tracking identifier, and inputting an image corresponding to a target frame of the teacher target in each image frame of the first image frame sequence into a state classification module to obtain a classification result of whether the teacher target is in a preset state, wherein the preset state includes: a board writing state and/or a board-back writing state.
In this embodiment, the tracking information (the target frame and the tracking identifier) of the teacher target is input into the state classification algorithm to classify the teacher target state, and the classification output result outputs two states, one is that the teacher writes a blackboard writing, and the other is that the teacher backs on the blackboard writing (lectures). The character target state classification model generated by the character target state classification algorithm is based on a deep learning model, the character target state classification model is a character target state classification model directly trained through picture materials written by 10 ten thousand teachers, the character target state classification model outputs two action states of writing on a blackboard and back on the blackboard, and the character target state classification model mainly judges the writing state on the blackboard.
In this embodiment, the result of the classification of whether the teacher target is in the preset state is obtained by judging whether the state of the teacher target output by the person target state classification model is always writing on a blackboard or writing back on a blackboard in the consecutive multi-frame image frames of the first image frame sequence. Wherein, judge that the state of writing on the board is implemented through presetting teacher's writing on the board rule, the teacher writes on the board and writes the rule and includes: configurable items of continuous duration (preset time period), sensitivity, etc.; the teacher writes on the board continuously, and the teacher can accurately identify the board writing operation instead of one-time board writing operation, but the board writing operation can be identified only by continuous multi-frame image frames; for example, the continuous time is set to 5s, and it is recognized that the states of teachers for the continuous 5s are all writing states, the state of the verification teacher target is generated as writing.
And step 33, recording a writing board writing event of the teacher target under the condition that the teacher target is detected to be in a preset state from a plurality of continuous image frames in the first image frame sequence, wherein the evaluation data comprises the writing board writing event of the teacher target.
Through the above steps 31 to 33, detection of a teacher's target and extraction of evaluation data of a board writing event from the image frame sequence are realized.
In some embodiments, a human target is detected and tracked from an image frame sequence, and tracking information of the human target is obtained; the method for extracting the evaluation data for evaluating the classroom teaching quality from the tracking information comprises the following steps:
step 41, performing target detection on each image frame of the second image frame sequence, and generating tracking information of the human target, where the tracking information includes: and the target frame is used for marking the character target and the tracking identification.
The tracking of the human target in this embodiment is a target frame of the tracked human target. In this embodiment, the target frame includes a head-shoulder frame and a body frame, fig. 3 is a view of a target position information identification structure in the embodiment of the present application, the position is represented by a picture pixel coordinate, and the two points form a rectangle by an upper left corner position and a lower right corner position. As shown in fig. 3, the first pixel point at the upper left corner of the picture is taken as the origin of coordinates, the coordinates are (0,0,0,0), R1 and T1 are the coordinates of the upper left corner of the human body and the upper left corner of the head and shoulder, respectively, and R2 and Y2 are the coordinates of the lower right corner of the human body and the lower right corner of the head and shoulder, respectively. The positions of the targets are head-shoulder frames and body frames, such as the frames shown by K1 and K2 in FIG. 3. In this embodiment, the image frame sequence in which the tracking of the student object is completed constitutes a second image frame sequence, and thus the tracking of the student object is completed in the second image frame sequence.
Step 42, determining student targets in the character targets according to the tracking identifiers, and respectively inputting images corresponding to target frames of the student targets in each image frame of the second image frame sequence to a height calculation module to obtain height information of the student targets in each image frame;
and 43, recording the rising events of the student targets under the condition that the variation of the height information of the student targets is detected to be larger than a preset threshold value from the adjacent image frames in the second image frame sequence, wherein the evaluation data comprises the rising events of the student targets.
The variation of the height information in this embodiment refers to the height difference between the height information of the same student target in the adjacent image frames, and the height difference is determined to be equal to the preset threshold value after being obtained; in the present embodiment, the preset threshold is set to 50cm, and specifically, the position coordinate of the student target in the world coordinate system in the image frame at time t is set to (X)1，Y1，Z1) At time t +1 is (X)2，Y2，Z2) Wherein when Z is2-Z1When the distance is 50cm or more, it means that the student is in the target state from sitting to standing, and in this embodiment, only the student is judged from sitting to standing, and when the student is sitting from standing, it is not judged, that is, when Z is2-Z1A state less than 50cm, an invalid action event as a student target.
In this embodiment, the determination of the student target standing event is performed based on a standing rule, where the standing rule includes a continuous duration and a sensitivity configuration item; when the student targets are judged to generate the standing events, the height of the student targets at the time T is set to be H1, the height of the student targets at the time T +1 is set to be H2, and the like, and the height at the time T +6 is set to be H6. And if the differences of H2-H1, H3-H1, H4-H1, H5-H1 and H6-H1 are all larger than or equal to 50cm, the student object is considered to have a standing event.
Through the steps 41 to 43, the identification of the standing up action of the student target is realized, and the evaluation data for evaluating the classroom teaching quality is obtained.
In some of these embodiments, the goal boxes of the student goal comprise a first goal box for labeling the head and shoulders of the student goal and a second goal box for labeling the body of the student goal below the shoulders; wherein the height calculation module calculates height information of the student object based on the center coordinates of the first object frame.
In this embodiment, the first target frame and the second target frame are used to form target position information of the student target, so that the second image frame sequence corresponding to the non-podium area can be segmented from the image frame sequences, and the student target is detected from each image frame of the second image frame sequence.
In some of these embodiments, after step 43, the following steps are further included: the image frame corresponding to the rise event is sent to an object associated with the student object.
After the student object forms the standing event in this embodiment, the image frame currently forming the standing event is automatically saved, and then the image frame is sent to the object associated with the student object, for example: parents of students; the image frames are sent, and apps corresponding to online class teaching can be selected for real-time pushing, and objects related to the student targets can know the class state of the student targets in real time through the received image frames.
In some embodiments, a human target is detected and tracked from an image frame sequence, and tracking information of the human target is obtained; the method for extracting the evaluation data for evaluating the classroom teaching quality from the tracking information comprises the following steps:
and step 51, acquiring a left eye image sequence and a right eye image sequence of the classroom through a binocular camera, wherein corresponding right eye images in the left eye image sequence and the right eye image sequence are shot synchronously.
Step 52, performing target detection on the left eye image in the left eye image sequence and the right eye image in the right eye image sequence to generate tracking information of the human target, wherein the tracking information includes: and the target frame is used for marking the character target and the tracking identification.
In this embodiment, the target map for target detection is at least a left target map in the left target map sequence and a right target map in the right target map sequence. The tracking of the human target in this embodiment is a target frame of the tracked human target. In this embodiment, the target frame includes a head-shoulder frame and a body frame, fig. 3 is a structural view of a target position information identifier in the embodiment of the present application, the position is represented by a picture pixel coordinate, and the position is represented by an upper left corner position and a lower right corner position, and two points form a rectangle. As shown in fig. 3, the first pixel point at the upper left corner of the picture is taken as a coordinate origin, the coordinates are (0,0,0,0), R1 and T1 are the coordinates of the upper left corner of the human body and the upper left corner of the head and shoulder, respectively, and R2 and Y2 are the coordinates of the lower right corner of the human body and the lower right corner of the head and shoulder, respectively. The positions of the targets are head-shoulder frames and body frames, such as the frames shown by K1 and K2 in FIG. 3.
And step 53, determining the student targets in the character targets according to the tracking identification.
And step 54, determining the spatial depth information of the corresponding student targets according to the parallax of the same student targets in the left eye image and the right eye image.
In this embodiment, video images (i.e., a sequence of image frames) in a classroom in which teaching is performed are acquired by using a binocular camera, the video images include left eye original images and right eye original images, then stereo correction, stereo matching and the like are performed on the video images to obtain disparity maps (including a left disparity map and a right disparity map), and the disparity maps are converted to obtain depth maps.
And step 55, inputting the image corresponding to the target frame of each student target in each image frame of the image frame sequence and the corresponding spatial depth information of the students into the height calculation module to obtain the real height information of each student target in each image frame.
The evaluation data acquisition implementation process of the present embodiment is further described as follows:
fig. 2 is a distribution diagram of the internal areas of a classroom for implementing classroom teaching according to an embodiment of the present application, which mainly includes: blackboard area 2_2, PPT area 2_3, area 2_4 between podium and blackboard, podium 2_5, class seat area 1_6, binocular camera 2_ 7. The method comprises the steps of acquiring video images (namely image frame sequences) for classroom teaching through a binocular camera, and detecting and tracking character targets in the image frames of the video images so as to obtain student targets, teacher targets writing on a blackboard and position information of the teacher targets; in order to reduce false alarm and reduce time consumption as much as possible, a PPT shielding region (video display region) and a platform region (region between the platform region and a blackboard, namely a teaching region) need to be added, so that the teacher target is subjected to state classification independently, state classification of students and teachers is avoided, time consumption can be reduced to a great extent, and the PPT shielding region is added mainly to filter characters in PPT or characters in teacher videos. Fig. 3 is a structural view of target location information identification in an embodiment of the present application, and fig. 4 is a flowchart of target action recognition according to an embodiment of the present application.
Referring to fig. 2 to 4, the status of students and teachers in the present application is identified as follows:
step 1: and (5) preprocessing a video image. The virtual frame shown in fig. 4 is obtained by preprocessing an original image acquired by a binocular camera, performing stereo correction, stereo matching and the like on left and right eye images acquired by the binocular camera to obtain left and right disparity maps, and converting the disparity maps to obtain depth maps; the preprocessing mainly uses a Zhangyingyou camera calibration method and an SGBM algorithm to obtain a disparity map, wherein a left disparity map, a right disparity map and a depth map are used for subsequently calculating three-dimensional information (tracking information) of a human target.
Step 2: target detection, namely performing target detection on a left eye image by using a deep learning method, wherein a yolo3 deep learning model is mainly used, and outputting character target position information in a classroom image by the target detection, wherein the character target position information refers to the head, shoulders and human body positions of a teacher, students and video characters played in PPT (Power Point) in a classroom; the positions are represented by picture pixel coordinates, and are represented by the upper left corner position and the lower right corner position, the two points form a rectangle, as shown in the target position information identifier shown in fig. 3, and the positions of the targets are actually the rectangular frames of the head, the shoulders and the human body, as shown in K1 and K2 in fig. 3.
And step 3: the target frames generated in the step 2 are sent to a tracking module, independent target marks are generated for each target frame (a head shoulder frame and a human body frame), a follow-up module maintains target information by using the target marks, and if the same target is always in the picture, the target marks are kept unchanged; in target tracking, a PPT shielding region 2_9 and a podium detection region 2_8 shown in fig. 2 need to be configured, the PPT shielding region 2_9 is used for filtering out a person target in the PPT during tracking of the target, the podium detection region is used for only tracking a target of the podium region (only tracking the target of a teacher), and the target outside the region is not tracked, so that the time consumption for tracking is reduced, and the overall time consumption is reduced.
And 4, step 4: and (3) inputting the tracking information (the target frame of the teacher and the target identification corresponding to the target frame) output in the step (3) into a target action classification module, wherein the target action module mainly classifies the target action state by utilizing deep learning, the target of the teacher is mainly classified, the classification output result directly outputs the states of writing on a board and writing on a back board of the teacher, and the output state of the classification of the target state is used for judging by a subsequent rule judgment module.
And 5: and inputting the left and right disparity maps, the depth map and the tracking result into a target height calculation module, wherein the module mainly uses the inverse operation of camera calibration and reversely deduces the three-dimensional coordinates (X, Y and Z) of the target in a world coordinate system through the pixels of the target.
Step 6: and (3) uniformly inputting the outputs of the step (1), the step (2), the step (3), the step (4) and the step (5) into a rule judgment module, wherein blackboard writing of a teacher and student standing rules are configured in advance in the rule judgment module, the world coordinates of students are calculated in the step (5), and the height difference between continuous frames can be calculated through the front continuous frame and the rear continuous frame in the rule judgment module. For example: the world coordinate of the head and shoulder center point of the student A in the video frame at the current time t is W1(X1，Y1，Z1) And the world coordinate of the head-shoulder central point of A at the time t +1 is W2(X2，Y2，Z2). The height difference is W2-W1, and when the height difference is larger than 50cm, the student is determined to stand up from a sitting state; at the same time, if continuous multiframes are provided, the teacher's purposeAnd if the state of label classification output is always the writing board writing state or the back-to-back writing state, the action of the teacher is confirmed to be writing board writing or back-to-back writing board writing.
And 7: and restoring the position information of each frame into a position cloud picture by counting the tracking position information of each frame of teacher. The generation of the position cloud picture can indicate that teachers are more in specific positions of the platform and the blackboard area in the class teaching. The positions are characterized as follows: 1. the cloud pictures of the teacher are uniformly distributed, so that the teacher repeatedly goes back and forth in front of the blackboard, and the teacher can move; 2. if the positions of the cloud pictures represent that more teachers are always positioned at the lecture stations, the fact that the teachers have long time to attend the lecture stations is explained, and the time for handwriting the blackboard is short; 3. if more cloud chart teachers are not in the blackboard accessories, the fact that the teachers often go to the classroom to assist the students is more.
And 8: uploading the position information of the teacher in each frame, the student standing up action and the board writing action of the teacher. The standing times and writing times of students in a class period are collected, a cloud picture of the positions of the teachers is drawn, and the teaching quality of the teachers can be evaluated correctly and fairly through the information.
And step 9: the student stands up the picture that the action can automatic save current action, through the head of a family app constantly push to the head of a family in, the head of a family can know the condition that child is at school very first time. Meanwhile, the binocular camera can be used for recording and broadcasting classroom teaching of excellent teachers, and recorded and broadcast videos are further popularized to remote mountainous areas, so that children in the mountainous areas can obtain better teaching resources.
It should be noted that the steps illustrated in the above-described flow diagrams or in the flow diagrams of the figures may be performed in a computer system such as a set of computer-executable instructions and that, while the logic order is illustrated in the flow diagrams, in some cases, the steps illustrated or described may be performed in an order different than here.
The embodiment also provides an obtaining device of classroom teaching quality evaluation data, which is used for implementing the above embodiments and preferred embodiments, and the description of the obtaining device is omitted. As used below, the terms "module," "unit," "subunit," and the like may implement a combination of software and/or hardware for a predetermined function. Although the acquisition means described in the following embodiments are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 5 is a block diagram of an acquisition apparatus of classroom teaching quality evaluation data according to an embodiment of the present application, as shown in fig. 5, the acquisition apparatus includes:
the obtaining module 51 is configured to obtain a sequence of image frames of a classroom.
The object detection module 52 is coupled with the acquisition module 51 and is used for detecting and tracking the human object from the image frame sequence to obtain tracking information of the human object; wherein the character goals include a teacher goal and a student goal.
And the processing module 53 is coupled with the target detection module 52 and is used for extracting evaluation data for evaluating the classroom teaching quality from the tracking information.
In some embodiments, the classroom includes a lecture area and a teaching area, and the object detection module 52 is configured to segment a first image area corresponding to the lecture area and a second image area corresponding to the teaching area from each image frame of the image frame sequence; and respectively detecting and tracking the person target from the first image area and the second image area, determining the person target detected and tracked from the first image area as a student target, and determining the person target detected and tracked from the second image area as a teacher target.
In some embodiments, the tracking information includes location information, and the processing module 53 is configured to extract tracking information of the teacher target from the tracking information; and generating position distribution cloud picture information of the teacher target according to the position information of the teacher target, wherein the evaluation data comprises the position distribution cloud picture information of the teacher target.
In some embodiments, the apparatus is further configured to perform target detection on each image frame of the sequence of image frames, and generate tracking information of the human target, where the tracking information includes: a target frame and a tracking identifier for marking the figure target; determining a teacher target in the character targets according to the tracking identification, inputting an image corresponding to a target frame of the teacher target in each image frame of the image frame sequence into a state classification module, and obtaining a classification result of whether the teacher target is in a preset state, wherein the preset state comprises the following steps: a board writing state and/or a back-to-board writing state; in the case where it is detected from a plurality of consecutive image frames in the image frame sequence that the teacher target is in a preset state, a writing board event of the teacher target is recorded, wherein the evaluation data includes the writing board event of the teacher target.
In some embodiments, the apparatus is further configured to perform target detection on each image frame of the sequence of image frames, and generate tracking information of the human target, where the tracking information includes: a target frame and a tracking identifier for marking the figure target; determining student targets in the character targets according to the tracking identification, and respectively inputting images corresponding to target frames of the student targets in each image frame of the image frame sequence into a height calculation module to obtain height information of the student targets in each image frame; in the event that it is detected from adjacent image frames in the image frame sequence that the amount of change in the height information of the student object is greater than a preset threshold value, an event of rising of the student object is recorded, wherein the evaluation data includes the event of rising of the student object.
In some of these embodiments, the apparatus is further configured to send the image frame corresponding to the rise event to an object associated with the student object.
In some embodiments, the apparatus is further configured to acquire, by the binocular camera, a first image frame sequence and a second image frame sequence of the classroom, wherein corresponding two image frames of the first image frame sequence and the second image frame sequence are taken synchronously; performing target detection on each image frame in the two image frame sequences to generate tracking information of a human target, wherein the tracking information comprises: the target frame and the tracking mark are used for marking the character target; determining student targets in the character targets according to the tracking identification; and determining the spatial depth information of the corresponding student target according to the parallax of the same student target in the two image frames.
The modules may be functional modules or program modules, and may be implemented by software or hardware. For a module implemented by hardware, the modules may be located in the same processor; or the modules can be respectively positioned in different processors in any combination.
In addition, the method for acquiring the evaluation quantity in the embodiment of the present application described in conjunction with fig. 1 may be implemented by a computer device. Fig. 6 is a hardware structure diagram of a computer device according to an embodiment of the present application.
The computer device may comprise a processor 61 and a memory 62 in which computer program instructions are stored.
Specifically, the processor 61 may include a Central Processing Unit (CPU), or A Specific Integrated Circuit (ASIC), or may be configured to implement one or more Integrated circuits of the embodiments of the present Application.
The memory 62 may be used to store or cache various data files that need to be processed and/or used for communication, as well as possible computer program instructions executed by the processor 61.
The processor 61 reads and executes the computer program instructions stored in the memory 62 to realize the method for acquiring classroom teaching quality evaluation data in any one of the above-described embodiments.
In some of these embodiments, the computer device may also include a communication interface 63 and a bus 60. As shown in fig. 6, the processor 61, the memory 62, and the communication interface 63 are connected via a bus 60 to complete mutual communication.
The communication interface 63 is used for implementing communication between modules, devices, units and/or apparatuses in the embodiments of the present application. The communication interface 63 may also enable communication with other components such as: the data communication is carried out among external equipment, image/data acquisition equipment, a database, external storage, an image/data processing workstation and the like.
The computer device may execute the method for acquiring classroom teaching quality evaluation data in the embodiment of the present application based on the acquired image frame sequence of the classroom, thereby implementing the method for acquiring classroom teaching quality evaluation data described in conjunction with fig. 1.
In addition, in combination with the method for acquiring classroom teaching quality evaluation data in the above embodiments, embodiments of the present application may provide a computer-readable storage medium to implement. The computer readable storage medium having stored thereon computer program instructions; the computer program instructions, when executed by the processor, implement any one of the above-described embodiments of the method for obtaining classroom teaching quality assessment data.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (11)
1. A method for acquiring classroom teaching quality evaluation data is characterized by comprising the following steps:
acquiring an image frame sequence of a classroom;
detecting and tracking a human target from the image frame sequence to obtain tracking information of the human target; wherein the character goals include a teacher goal and a student goal;
and extracting evaluation data for evaluating the classroom teaching quality from the tracking information.
2. The method for acquiring classroom teaching quality assessment data as claimed in claim 1, wherein the classroom includes a non-podium area and a podium area; detecting and tracking a human target from the image frame sequence, wherein obtaining tracking information of the human target comprises:
segmenting a first image frame sequence corresponding to the non-platform area and a second image frame sequence corresponding to the platform area from the image frame sequences;
the human target is detected and tracked from each image frame of the first image frame sequence and each image frame of the second image frame sequence, respectively, and the human target detected and tracked from each image frame of the first image frame sequence is determined to be a student target, and the human target detected and tracked from each image frame of the second image frame sequence is determined to be a teacher target.
3. The method for acquiring classroom teaching quality assessment data as claimed in claim 2, wherein the podium area does not include a video display area.
4. The method for acquiring classroom teaching quality assessment data as claimed in claim 1, wherein said tracking information includes location information; extracting evaluation data for evaluating the classroom teaching quality from the tracking information comprises the following steps:
extracting tracking information of the teacher target from the tracking information;
and generating position distribution cloud picture information of the teacher target according to the position information of the teacher target, wherein the evaluation data comprises the position distribution cloud picture information of the teacher target.
5. The method for acquiring classroom teaching quality evaluation data as claimed in claim 2, wherein a human target is detected and tracked from the image frame sequence, obtaining tracking information of the human target; extracting evaluation data for evaluating the classroom teaching quality from the tracking information comprises the following steps:
performing target detection on each image frame of the first image frame sequence, and generating tracking information of the human target, wherein the tracking information includes: a target frame and a tracking identifier for marking the character target;
determining the teacher target in the character targets according to the tracking identification, and inputting an image corresponding to a target frame of the teacher target in each image frame of the first image frame sequence into a state classification module to obtain a classification result of whether the teacher target is in a preset state, wherein the preset state comprises: a board writing state and/or a back-to-board writing state;
recording a writing event of the teacher target in a case where it is detected from a plurality of consecutive image frames in the first image frame sequence that the teacher target is in the preset state, wherein the evaluation data includes the writing event of the teacher target.
6. The method for acquiring classroom teaching quality evaluation data as claimed in claim 2, wherein a human target is detected and tracked from the image frame sequence, obtaining tracking information of the human target; extracting evaluation data for evaluating the classroom teaching quality from the tracking information comprises the following steps:
performing target detection on each image frame of the second image frame sequence, and generating tracking information of the human target, wherein the tracking information includes: a target frame and a tracking identifier for marking the character target;
determining the student targets in the character targets according to the tracking identification, and respectively inputting images corresponding to target frames of the student targets in each image frame of the second image frame sequence to a height calculation module to obtain height information of the student targets in each image frame;
recording a rising event of the student object in case that a variation amount of the height information of the student object is detected to be greater than a preset threshold from adjacent image frames in the second image frame sequence, wherein the evaluation data includes the rising event of the student object.
7. The method for acquiring classroom teaching quality evaluation data as claimed in claim 6, wherein the goal boxes of the student goal include a first goal box for labeling the head and shoulders of the student goal and a second goal box for labeling the human body below the shoulders of the student goal; wherein the height calculation module calculates height information of the student object based on the center coordinates of the first object frame.
8. The method for acquiring classroom teaching quality assessment data as claimed in claim 6, wherein after the event of standing up of the student goal is recorded, the method further comprises:
and sending the image frame corresponding to the standing event to an object associated with the student object.
9. The method for acquiring classroom teaching quality evaluation data according to claim 6,
the method further comprises the following steps: acquiring a left eye image sequence and a right eye image sequence of the classroom through a binocular camera, wherein a left eye image in the left eye image sequence and a corresponding right eye image in the right eye image sequence are shot synchronously; determining the spatial depth information of the corresponding student targets according to the parallax of the same student targets in the left eye image and the right eye image;
respectively inputting images corresponding to the target frame of each student target in each image frame of the image frame sequence into a height calculation module, and obtaining the height information of each student target in each image frame comprises the following steps: and inputting the image corresponding to the target frame of each student target in each image frame of the image frame sequence and the corresponding spatial depth information of the students into the height calculation module to obtain the real height information of each student target in each image frame.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for acquiring classroom teaching quality assessment data as claimed in any one of claims 1 to 9 when executing the computer program.
11. A computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing the method of acquiring classroom teaching quality assessment data according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010656229.6A CN112001944A (en) | 2020-07-09 | 2020-07-09 | Classroom teaching quality evaluation data acquisition method, computer equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010656229.6A CN112001944A (en) | 2020-07-09 | 2020-07-09 | Classroom teaching quality evaluation data acquisition method, computer equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112001944A true CN112001944A (en) | 2020-11-27 |
Family
ID=73466703
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010656229.6A Pending CN112001944A (en) | 2020-07-09 | 2020-07-09 | Classroom teaching quality evaluation data acquisition method, computer equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112001944A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113572965A (en) * | 2021-08-06 | 2021-10-29 | 广州市奥威亚电子科技有限公司 | Double-blackboard-writing tracking shooting method and device and shooting system |
| CN113657509A (en) * | 2021-08-19 | 2021-11-16 | 重庆晋才富熙科技有限公司 | Teaching training improving method and device, terminal and storage medium |
| CN113674312A (en) * | 2021-06-30 | 2021-11-19 | 浙江大华技术股份有限公司 | Target tracking close-up method and device |
| CN115205982A (en) * | 2022-09-08 | 2022-10-18 | 深圳市维海德技术股份有限公司 | Standing tracking detection method, electronic device, and medium |
| CN115545587A (en) * | 2022-12-05 | 2022-12-30 | 成都大学 | Classroom teaching scheme evaluation method and system based on multiple agents |
| WO2023035891A1 (en) * | 2021-09-08 | 2023-03-16 | 广州视源电子科技股份有限公司 | Behavior identification method and apparatus, and device and storage medium |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103327250A (en) * | 2013-06-24 | 2013-09-25 | 深圳锐取信息技术股份有限公司 | Method for controlling camera lens based on pattern recognition |
| US20160217321A1 (en) * | 2015-01-23 | 2016-07-28 | Shindig. Inc. | Systems and methods for analyzing facial expressions within an online classroom to gauge participant attentiveness |
| CN108022205A (en) * | 2016-11-04 | 2018-05-11 | 杭州海康威视数字技术股份有限公司 | Method for tracking target, device and recording and broadcasting system |
| CN108230392A (en) * | 2018-01-23 | 2018-06-29 | 北京易智能科技有限公司 | A kind of dysopia analyte detection false-alarm elimination method based on IMU |
| CN108648757A (en) * | 2018-06-14 | 2018-10-12 | 北京中庆现代技术股份有限公司 | A kind of analysis method based on various dimensions Classroom Information |
| CN110232346A (en) * | 2019-06-06 | 2019-09-13 | 南京睦泽信息科技有限公司 | A kind of video intelligent analysis system based on deep learning |
| CN110244923A (en) * | 2018-09-30 | 2019-09-17 | 浙江大华技术股份有限公司 | A kind of image display method and apparatus |
| CN110334610A (en) * | 2019-06-14 | 2019-10-15 | 华中师范大学 | A kind of various dimensions classroom based on computer vision quantization system and method |
| CN110781843A (en) * | 2019-10-29 | 2020-02-11 | 首都师范大学 | Classroom behavior detection method and electronic equipment |
| CN110837795A (en) * | 2019-11-04 | 2020-02-25 | 防灾科技学院 | Teaching condition intelligent monitoring method, device and equipment based on classroom monitoring video |
| CN110895819A (en) * | 2018-09-12 | 2020-03-20 | 长沙智能驾驶研究院有限公司 | Target tracking method, target tracking device, computer-readable storage medium and computer equipment |
| CN111046819A (en) * | 2019-12-18 | 2020-04-21 | 浙江大华技术股份有限公司 | Behavior recognition processing method and device |
| CN111091063A (en) * | 2019-11-20 | 2020-05-01 | 北京迈格威科技有限公司 | Living body detection method, device and system |
-
2020
- 2020-07-09 CN CN202010656229.6A patent/CN112001944A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103327250A (en) * | 2013-06-24 | 2013-09-25 | 深圳锐取信息技术股份有限公司 | Method for controlling camera lens based on pattern recognition |
| US20160217321A1 (en) * | 2015-01-23 | 2016-07-28 | Shindig. Inc. | Systems and methods for analyzing facial expressions within an online classroom to gauge participant attentiveness |
| CN108022205A (en) * | 2016-11-04 | 2018-05-11 | 杭州海康威视数字技术股份有限公司 | Method for tracking target, device and recording and broadcasting system |
| CN108230392A (en) * | 2018-01-23 | 2018-06-29 | 北京易智能科技有限公司 | A kind of dysopia analyte detection false-alarm elimination method based on IMU |
| CN108648757A (en) * | 2018-06-14 | 2018-10-12 | 北京中庆现代技术股份有限公司 | A kind of analysis method based on various dimensions Classroom Information |
| CN110895819A (en) * | 2018-09-12 | 2020-03-20 | 长沙智能驾驶研究院有限公司 | Target tracking method, target tracking device, computer-readable storage medium and computer equipment |
| CN110244923A (en) * | 2018-09-30 | 2019-09-17 | 浙江大华技术股份有限公司 | A kind of image display method and apparatus |
| CN110232346A (en) * | 2019-06-06 | 2019-09-13 | 南京睦泽信息科技有限公司 | A kind of video intelligent analysis system based on deep learning |
| CN110334610A (en) * | 2019-06-14 | 2019-10-15 | 华中师范大学 | A kind of various dimensions classroom based on computer vision quantization system and method |
| CN110781843A (en) * | 2019-10-29 | 2020-02-11 | 首都师范大学 | Classroom behavior detection method and electronic equipment |
| CN110837795A (en) * | 2019-11-04 | 2020-02-25 | 防灾科技学院 | Teaching condition intelligent monitoring method, device and equipment based on classroom monitoring video |
| CN111091063A (en) * | 2019-11-20 | 2020-05-01 | 北京迈格威科技有限公司 | Living body detection method, device and system |
| CN111046819A (en) * | 2019-12-18 | 2020-04-21 | 浙江大华技术股份有限公司 | Behavior recognition processing method and device |
Non-Patent Citations (1)
| Title |
|---|
| 邵奔驰: "基于深度学习的多目标检测在教室场景中的应用", 中国优秀硕士学位论文电子全文数据库信息科技, no. 6, 15 June 2020 (2020-06-15) * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113674312A (en) * | 2021-06-30 | 2021-11-19 | 浙江大华技术股份有限公司 | Target tracking close-up method and device |
| CN113572965A (en) * | 2021-08-06 | 2021-10-29 | 广州市奥威亚电子科技有限公司 | Double-blackboard-writing tracking shooting method and device and shooting system |
| CN113657509A (en) * | 2021-08-19 | 2021-11-16 | 重庆晋才富熙科技有限公司 | Teaching training improving method and device, terminal and storage medium |
| CN113657509B (en) * | 2021-08-19 | 2024-04-30 | 重庆晋才富熙科技有限公司 | Teaching training lifting method, device, terminal and storage medium |
| WO2023035891A1 (en) * | 2021-09-08 | 2023-03-16 | 广州视源电子科技股份有限公司 | Behavior identification method and apparatus, and device and storage medium |
| CN115205982A (en) * | 2022-09-08 | 2022-10-18 | 深圳市维海德技术股份有限公司 | Standing tracking detection method, electronic device, and medium |
| CN115545587A (en) * | 2022-12-05 | 2022-12-30 | 成都大学 | Classroom teaching scheme evaluation method and system based on multiple agents |
| CN115545587B (en) * | 2022-12-05 | 2023-03-24 | 成都大学 | Classroom teaching scheme evaluation method and system based on multiple intelligent agents |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112001944A (en) | Classroom teaching quality evaluation data acquisition method, computer equipment and medium | |
| CN109522815B (en) | Concentration degree evaluation method and device and electronic equipment | |
| CN107105310B (en) | Human image replacing method and device in video live broadcast and recording and broadcasting system | |
| CN107292318B (en) | Image significance object detection method based on center dark channel prior information | |
| CN105512627A (en) | Key point positioning method and terminal | |
| CN111814587B (en) | Human behavior detection method, teacher behavior detection method, and related systems and devices | |
| CN105228708A (en) | Body action scoring apparatus, dancing scoring apparatus, Caraok device and game device | |
| CN107798932A (en) | A kind of early education training system based on AR technologies | |
| CN104516501A (en) | Display device and content display method | |
| KR20150039252A (en) | Apparatus and method for providing application service by using action recognition | |
| CN112101123B (en) | Attention detection method and device | |
| US20220207266A1 (en) | Methods, devices, electronic apparatuses and storage media of image processing | |
| CN112116556B (en) | Passenger flow volume statistics method and device and computer equipment | |
| CN111160134A (en) | Human-subject video scene analysis method and device | |
| US20190304136A1 (en) | Gaze point estimation processing apparatus, gaze point estimation model generation apparatus, gaze point estimation processing system, and gaze point estimation processing method | |
| CN103020606A (en) | Pedestrian detection method based on spatio-temporal context information | |
| CN111325107B (en) | Detection model training method, device, electronic equipment and readable storage medium | |
| CN102096930A (en) | Student standing and sitting detection method for intelligent recorded broadcasting system for teaching | |
| CN112613780A (en) | Learning report generation method and device, electronic equipment and storage medium | |
| CN112818933A (en) | Target object identification processing method, device, equipment and medium | |
| CN111160277A (en) | Behavior recognition analysis method and system, and computer-readable storage medium | |
| CN113705510A (en) | Target identification tracking method, device, equipment and storage medium | |
| CN116266415A (en) | Action evaluation method, system and device based on body building teaching training and medium | |
| CN112580584A (en) | Method, device and system for detecting standing behavior and storage medium | |
| CN113496235A (en) | Image processing method, device and system, storage medium and computing equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |