CN114639157A - Bad learning behavior detection method, system, electronic device and storage medium - Google Patents
Bad learning behavior detection method, system, electronic device and storage medium Download PDFInfo
- Publication number
- CN114639157A CN114639157A CN202210536462.XA CN202210536462A CN114639157A CN 114639157 A CN114639157 A CN 114639157A CN 202210536462 A CN202210536462 A CN 202210536462A CN 114639157 A CN114639157 A CN 114639157A
- Authority
- CN
- China
- Prior art keywords
- area
- pen
- hand
- target object
- reference value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000006399 behavior Effects 0.000 title claims abstract description 83
- 238000001514 detection method Methods 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000012937 correction Methods 0.000 claims abstract description 20
- 238000004590 computer program Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims 1
- 230000001815 facial effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 208000027418 Wounds and injury Diseases 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006378 damage Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 208000014674 injury Diseases 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000033001 locomotion Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006748 scratching Methods 0.000 description 1
- 230000002393 scratching effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Educational Technology (AREA)
- Tourism & Hospitality (AREA)
- Educational Administration (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- General Health & Medical Sciences (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Human Resources & Organizations (AREA)
- General Business, Economics & Management (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application relates to the technical field of machine vision, and discloses a method and a system for detecting bad learning behaviors, electronic equipment and a storage medium, wherein the method comprises the following steps: carrying out target area detection on the acquired depth map of the front side of the target object, and determining a hand area and a face area of the target object; under the condition that the face area and the hand area are overlapped, acquiring the minimum value of depth values in the hand area as a first reference value, and acquiring the maximum value of the depth values in a non-overlapped area of the face area and the hand area as a second reference value; calculating a first difference between the first reference value and the second reference value; if the absolute value of the first difference is smaller than the first preset threshold, it is determined that the target object has bad learning behaviors, and a preset first correction prompt tone is played.
Description
Technical Field
The embodiment of the application relates to the technical field of machine vision, in particular to a method and a system for detecting bad learning behaviors, electronic equipment and a storage medium.
Background
Along with the rapid development of communication technology and the complexity and changeability of epidemic situation prevention and control situation, the online teaching becomes an inseparable part of students, teachers give lessons remotely through the online teaching platform, students join online classes at home to listen to lessons, the online teaching platform provides multiple functions for simulating real classes, such as student sign-in, remote questioning, assignment work, class tests and the like, the inherent datamation, tabulation and standardization attributes of the online teaching platform provide multiple conveniences for teacher teaching, the course selection of various famous students is provided for students, the distance barrier is broken through, and the learning breadth and the learning depth of the students are improved.
It can be understood that student's self-restraint ability has not been established completely yet, the student often can make subconsciously when in class, think, write homework and eat one's hands, grab the face, bite the nib, change bad behaviors such as pen, these behaviors are unclean, insanitary, unsightly, and can also cause the injury to student's health, under the condition in real classroom, the teacher can overlook whole class, and can walk around in giving lessons, in case find the student make these bad learning behaviors, the teacher can in time stop, correct, help the student to develop good custom, avoid the emergence of dangerous condition.
However, in the teacher interface of the online teaching platform, students become a small window, and the teacher usually can only see the facial pictures of the students through the window, and the teacher cannot frequently browse and switch tens of small windows in the online class, and the small window has low picture quality and is relatively fuzzy, so that the teacher cannot timely find whether the students make bad learning behaviors, parents also have their own work and cannot accompany the children to online the class at any moment, so that the bad learning behaviors of the students cannot be timely corrected, and the students can easily develop bad learning habits.
Disclosure of Invention
An object of the embodiments of the present application is to provide a method, a system, an electronic device, and a storage medium for detecting bad learning behaviors, which can automatically find and correct the bad learning behaviors of students in real time on the premise of protecting the privacy of the students, so as to guide the students to develop good learning habits.
In order to solve the above technical problem, an embodiment of the present application provides a method for detecting an unfavorable learning behavior, including the following steps: carrying out target area detection on the acquired depth map of the front side of the target object, and determining a hand area and a face area of the target object; wherein the depth map is periodically acquired based on a preset time interval; under the condition that the face area and the hand area are overlapped, acquiring the minimum value of the depth values in the hand area as a first reference value, and acquiring the maximum value of the depth values in the non-overlapped area of the face area and the hand area as a second reference value; calculating a first difference between the first reference value and the second reference value; and if the absolute value of the first difference is smaller than a first preset threshold, determining that the target object has bad learning behavior, and playing a preset first correction prompt tone.
An embodiment of the present application further provides a bad learning behavior detection system, including: the depth camera module is used for periodically acquiring a depth map of the front side of the target object based on a preset time interval; the target area detection module is used for carrying out target area detection on the acquired depth map and determining a hand area and a face area of the target object; the calculation module is used for acquiring the minimum value of the depth values in the hand area as a first reference value, acquiring the maximum value of the depth values in the non-overlapped area of the face area compared with the hand area as a second reference value and calculating a first difference value between the first reference value and the second reference value under the condition that the face area and the hand area are overlapped; and the execution module is used for determining that the target object has bad learning behaviors and playing a preset first correction prompt tone under the condition that the absolute value of the first difference is smaller than a first preset threshold value.
An embodiment of the present application further provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the above-described bad learning behavior detection method.
Embodiments of the present application further provide a computer-readable storage medium storing a computer program, which when executed by a processor implements the above-mentioned bad learning behavior detection method.
The method, the system, the electronic device and the storage medium for detecting the bad learning behavior provided by the embodiments of the present application periodically obtain a depth map of a front side of a target object based on a preset time interval, perform target area detection on the depth map, determine a hand area and a face area of the target object, obtain a minimum value of depth values in the hand area as a first reference value and a maximum value of depth values in a non-overlapping area of the face area compared to the hand area as a second reference value when the face area and the hand area of the target object overlap, calculate a first difference between the first reference value and the second reference value, determine that the target object has the bad learning behavior if an absolute value of the difference between the first reference value and the second reference value is smaller than a first preset threshold, play a preset first correction prompt tone, and consider that a teacher cannot frequently browse, a user, and a user can not frequently browse the bad learning behavior in an online class, The method comprises the steps that a picture window of a student is switched, whether the student makes bad learning behaviors or not can not be found in time, whether the face area of a target object is overlapped with the hand area or not is detected periodically, if the hand area is overlapped with the face area, whether the distance between the hand area and the face area is too close or not is judged, if the distance is too close, the target object makes the bad learning behaviors such as grabbing a face and eating a hand is shown, due to the inherent data security of a depth map, the bad learning behaviors of the student can be found and corrected automatically in real time on the premise of protecting the privacy of the student, the student is guided to form a good learning habit, meanwhile, the detection frequency can be guaranteed to be appropriate through the setting of periodic detection, and the phenomenon that the student is disturbed excessively to go to class normally is avoided.
In addition, after the determining the hand region and the face region of the target object, the method includes: under the condition that the human face area and the hand area are not overlapped, determining whether the target object is in a pen holding state or not according to the outline of the hand area; if the target object is in a pen holding state, determining a pen area in the depth map; under the condition that the face area is overlapped with the pen area, acquiring the minimum value of the depth values in the pen area as a third reference value, and acquiring the maximum value of the depth values in the non-overlapped area of the face area and the pen area as a fourth reference value; calculating a second difference between the third reference value and the fourth reference value; if the absolute value of the second difference is smaller than a second preset threshold, it is determined that the target object has bad learning behaviors, a preset second correction prompt tone is played, and in consideration of that behaviors such as pen-biting and pen-using rancour are also bad learning behaviors which are unclean, insanitary and unsightly, although hands of a student do not touch the face, the pen can be regarded as extension of the hands, and only whether a face region overlaps with the hand region or not and the bad learning behaviors such as pen-biting cannot be detected.
Additionally, after the determining the area of the pen in the depth map, further comprising: determining the angle of the pen relative to the plumb line according to the area of the pen under the condition that the human face area is not overlapped with the area of the pen; if the angle of the pen relative to the plumb line is larger than a third preset threshold value, determining that the target object has bad learning behaviors, playing a preset third correction prompt tone, considering that the pen is also an unclean, unsightly and dangerous bad learning behavior, scratching the user himself when the user is out of hand, therefore, the angle of the pen relative to the plumb line is obtained and detected under the condition that the target object is detected to be in the pen holding state and the human face area of the target object is not overlapped with the area of the pen, when the pen is held normally, the pen is closer to be parallel and the angle is smaller than the plumb line, and the pen approaches to be horizontal when the pen is rotated, i.e. the angle of the pen with respect to the plumb line is large, when the angle of the pen with respect to the plumb line is larger than a third predetermined threshold, this application can in time correct this kind of dangerous bad study action, prevents that the student from receiving the injury that this can avoid.
In addition, the hand region includes a left hand region and a right hand region, and when the face region overlaps the hand region, acquiring a minimum value of the depth values in the hand region as a first reference value includes: under the condition that the face area is overlapped with at least one of the left hand area and the right hand area, the minimum value of the depth value in the hand area overlapped with the face area is obtained to be a first reference value, in the depth map, the left hand and the right hand of a student are detected generally, the student can possibly only use one hand to scratch the face and grab the face, the other hand is normally put on a desk, and the hand which is not overlapped with the face possibly influences the hand which is overlapped with the face.
In addition, the minimum value of the depth values in the hand area is obtained as a first reference value, and specifically: the minimum value of the depth values in the overlap area of the hand area and the face area is obtained as a first reference value, generally speaking, children have good flexibility and may make various strange motions, and therefore the minimum value of the depth values searched in the whole area of the hand area may not be accurate enough as the first reference value, and therefore the accuracy of detecting the bad learning behavior can be further improved by only searching the minimum value of the depth values in the overlap area of the hand area and the face area as the first reference value.
In addition, before the target area detection is performed on the acquired depth map of the front of the target object, the method includes: periodically acquiring continuous M-frame depth maps based on a preset time interval; the target area detection of the acquired depth map of the front of the target object includes: the target area detection is respectively carried out on the continuous M-frame depth maps, generally speaking, the short-time face touching by hands is not a bad learning behavior, but the long-time face touching by hands can basically judge that the bad learning behavior is the bad learning behavior, so that the depth maps in a period of time, namely the continuous M-frame depth maps, are obtained at one time, the target area detection is carried out on the basis of the continuous M-frame depth maps, and the accuracy of the bad learning behavior detection can be further improved.
Drawings
One or more embodiments are illustrated by the corresponding figures in the drawings, which are not meant to be limiting.
Fig. 1 is a first flowchart of a bad learning behavior detection method according to an embodiment of the present application;
FIG. 2 is a schematic illustration of a student grabbing a face according to an embodiment of the present application;
fig. 3 is a flowchart ii of a bad learning behavior detection method according to an embodiment of the present application;
FIG. 4 is a schematic illustration of a student's pen rancour face provided by an embodiment of the present application;
fig. 5 is a flowchart three of a bad learning behavior detection method according to an embodiment of the present application;
FIG. 6 is a flow chart of determining an angle of a pen with respect to a plumb line based on a region of the pen in one embodiment of the present application;
fig. 7 is a schematic diagram of a bad learning behavior detection system of another embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to another embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that in the various embodiments of the present application, numerous technical details are set forth in order to provide a better understanding of the present application. However, the technical solution claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not constitute any limitation to the specific implementation manner of the present application, and the embodiments may be mutually incorporated and referred to without contradiction.
An embodiment of the present application relates to a method for detecting an undesired learning behavior, which is applied to an electronic device, where the electronic device may be a terminal or a server, and the electronic device in this embodiment and the following embodiments are described by taking the server as an example.
A specific flow of the bad learning behavior detection method of this embodiment may be as shown in fig. 1, and includes:
Specifically, the server may periodically perform one detection without completing one acquisition based on a depth map of the front of the target object at a preset time interval, that is, perform target area detection on the depth map of the front of the target object, and determine a hand area and a face area of the target object in the depth map, where the preset time interval may be set by a person skilled in the art according to actual needs, and the preset time interval is not set too short, and if the preset time interval is set too short, a large amount of shooting and calculation resources are occupied, and it is also easy to excessively interrupt a normal class of a student, and it is easy to adjust too much.
In a specific implementation, the server is in communication with the depth camera, the depth camera may obtain a depth map of a target scene, the depth camera is directly shooting the front of the target object, the server periodically wakes up and calls the depth camera based on a preset time interval, and the depth camera shoots the front of the target object to obtain the depth map and transmits the depth map to the server.
In one example, when the server performs target area detection on the acquired depth map on the front side of the target object, the server may traverse the depth values of the pixels in the depth map, perform connected domain detection based on the depth values of the pixels, thereby obtaining a plurality of basic areas, and then determine the hand area and the face area of the target object from the basic areas according to the contour shape of the basic areas.
In another example, when the server performs target area detection on the acquired depth map of the front of the target object, the server may input the depth map of the front of the target object into a pre-trained target area detection model, and acquire a hand area and a face area of the target object output by the target area detection model.
In an example, a server may periodically obtain consecutive M-frame depth maps based on a preset time interval, and the server may perform target region detection on the consecutive M-frame depth maps, where M is an integer greater than 1, and generally, when a face is touched by a hand briefly, it is not a bad learning behavior, but when a face is touched by a hand for a long time, it is basically determined that the face is a bad learning behavior.
And 102, under the condition that the face area and the hand area are overlapped, acquiring the minimum value of the depth values in the hand area as a first reference value, and acquiring the maximum value of the depth values in the non-overlapped area of the face area and the hand area as a second reference value.
Specifically, after the server determines the hand region and the face region of the target object in the depth map, it may detect whether the face region of the target object overlaps with the hand region of the target object, and in a case where it is determined that the face region of the target object overlaps with the hand region, the server may acquire a minimum value of the depth values in the hand region of the target object as a first reference value and a maximum value of the depth values in the non-overlapping region of the face region of the target object with respect to the hand region as a second reference value.
It can be understood that, for the front of the target object, the human face area overlaps with the hand area, and only if the target object blocks the face with a hand, for example, if the target object performs a face-grabbing operation, the hand is completely blocked by the head if the target object performs a head-holding operation, and the hand area cannot be detected in the depth map of the front of the target object.
In one example, the hand area includes a left hand area and a right hand area, the server may detect whether the face area of the target object overlaps at least one of the left hand area and the right hand area after determining the hand area and the face area of the target object, and in the case that the face area of the target object overlaps at least one of the left hand area and the right hand area, the server obtains a first reference value as a minimum value of depth values in the hand area overlapping the face area and obtains a second reference value as a maximum value of depth values in a non-overlapping area of the face area compared to the hand area, in the depth map, the left and right hands of the student are generally detected, and the student may scratch and grab the face with only one hand while the other hand is normally on a desk, and the hand not overlapping the face may affect the hand overlapping the face, therefore, the minimum value of the depth value is searched in the hand area corresponding to the hand with the overlapped face area, and therefore accuracy of detection of poor learning behaviors can be improved.
In one example, the server acquires that the minimum value of the depth values in the hand region of the target object is the first reference value, specifically, the minimum value of the depth values in the overlap region of the hand region of the target object with respect to the face region is the first reference value, generally speaking, children have good flexibility and may make various strange actions, so that the minimum value of the depth values searched in the whole area of the hand region may not be accurate enough as the first reference value, and therefore, the application searches the minimum value of the depth values in the overlap region of the hand region with respect to the face region as the first reference value, and further improves the accuracy of detection of poor learning behaviors.
And 104, if the absolute value of the first difference is smaller than a first preset threshold, determining that the target object has bad learning behavior, and playing a preset first correction prompt tone.
In a specific implementation, after the server obtains the first reference value and the second reference value, a first difference between the first reference value and the second reference value may be calculated, and it is determined whether an absolute value of the first difference between the first reference value and the second reference value is smaller than a first preset threshold, if the absolute value of the first difference is smaller than the first preset threshold, it is determined that the target object has undesirable learning behaviors such as face grasping, hand eating, and the like, and the server may play a preset first correction prompt tone to remind the target object, where the first preset threshold may be set by a technician in the art according to actual needs.
In one example, the target subject may make poor learning behaviors such as grabbing a face, eating a hand, etc. as shown in fig. 2.
In this embodiment, the server periodically obtains a depth map of the front side of the target object based on a preset time interval, performs target area detection on the depth map, determines a hand area and a face area of the target object, obtains a minimum value of depth values in the hand area as a first reference value when the face area and the hand area of the target object overlap, obtains a maximum value of depth values in the face area compared with a non-overlapping area of the hand area as a second reference value, calculates a first difference between the first reference value and the second reference value, determines that the target object has an undesirable learning behavior if an absolute value of the difference between the first reference value and the second reference value is smaller than a first preset threshold, plays a preset first correction prompt tone, and considers that a teacher cannot frequently browse and switch a student picture window during online lessons, i.e. cannot timely find whether a student makes an undesirable learning behavior, the method and the device for detecting the facial area of the target object periodically detect whether the facial area of the target object is overlapped with the hand area, if the facial area is overlapped, whether the distance between the hand area and the facial area is too close, if the distance is too close, the target object makes bad learning behaviors such as face grabbing and hand eating, and due to the inherent data security of the depth map, the method and the device for detecting the facial area of the target object can automatically find and correct the bad learning behaviors of students in real time on the premise of protecting the privacy of the students, guide the students to develop good learning habits, meanwhile, the setting of periodic detection can ensure that the detection frequency is proper, and avoid excessively disturbing the students to normally go to class.
In an embodiment, after determining the hand region and the face region of the target object, the server may perform the detection of the poor learning behavior on the target object through the steps shown in fig. 3, which specifically includes:
In a specific implementation, although there is no overlap between the face region and the hand region of the target object, that is, the target object does not perform undesirable learning behaviors such as catching a face and eating a hand, a tool such as a pen held by the target object may perform other undesirable learning behaviors such as biting a pen head and using a pen rancour as an extension of a hand, and the server may determine whether the target object is in a pen holding state according to the contour of the hand region when determining that there is no overlap between the face region and the hand region of the target object, so as to prepare for further detection of the undesirable learning behaviors.
In one example, the server may input the contour of the hand region of the target object into a pre-trained pen-holding state detection model, and obtain a detection result of the contour of the hand region of the target object, which is output by the pen-holding state detection model, where the detection result is a pen-holding state and a non-pen-holding state.
In a specific implementation, if the server determines that the target object is in a pen-holding state, the server may determine a pen region in the depth map, and since the lengths, sizes, and shapes of the pens are different, the server may obtain depth values of points within a certain search range with the hand region as a center, thereby determining the pen region in the depth map.
In step 203, under the condition that the face area and the pen area are overlapped, the minimum value of the depth values in the pen area is obtained as a third reference value, and the maximum value of the depth values in the non-overlapped area of the face area and the pen area is obtained as a fourth reference value.
In a specific implementation, after the server determines a pen region in the depth map, it may determine whether there is overlap between a face region of the target object and the pen region, and when it is determined that there is overlap between the face region of the target object and the pen region, the server may obtain a minimum value of depth values in the pen region as a third reference value, and obtain a maximum value of depth values in a non-overlapping region of the face region and the pen region as a fourth reference value, where the overlap between the face region of the target object and the pen region indicates that the target object blocks a face with the pen, and normal writing does not occur, that is, the target object is likely to perform an undesirable learning behavior.
In step 204, a second difference between the third reference value and the fourth reference value is calculated.
In step 205, if the absolute value of the second difference is smaller than the second preset threshold, it is determined that the target object has an adverse learning behavior, and a preset second correction prompt tone is played.
In a specific implementation, after obtaining the third reference value and the fourth reference value, the server may calculate a second difference between the third reference value and the fourth reference value, and judging whether the absolute value of a second difference between the third reference value and the fourth reference value is smaller than a second preset threshold value, if so, it is determined that there is an ill-learning behavior of the target object, such as pen-biting, face-with-pen rancour, etc., the server may play a preset second correction tone, thereby alerting the target object, the second preset threshold may be set by a person skilled in the art according to actual needs, which is not specifically limited in the embodiment of the present application, and the second correction prompt tone may be prerecorded by parents of students and teachers, and the second correction prompt tone is used to prevent the target object from continuing to make bad learning behaviors such as pen-biting, using pen rancour faces, and the like.
In one example, the target object may make an undesirable learning behavior such as a face with a pen rancour as shown in FIG. 4.
In the embodiment, considering that behaviors such as pen biting, pen using rancour and the like are unclean, unsanitary and unsightly bad learning behaviors, although hands of students do not touch the faces, the pen can be regarded as extension of the hands, and the bad learning behaviors such as pen biting cannot be detected only by detecting whether a face area overlaps with a hand area.
In an embodiment, after determining the pen region in the depth map, the server may perform poor learning behavior detection on the target object through the steps shown in fig. 5, which specifically include:
In the specific implementation, considering that the pen rotation is also an unclean, unsightly and dangerous bad learning behavior, once the pen is out of hand, the user is likely to scratch himself, so that the angle of the pen relative to the plumb line can be obtained and detected under the condition that the target object is detected to be in the pen holding state and the face region of the target object is not overlapped with the region of the pen, when the user holds the pen normally, the pen is closer to be parallel and smaller than the plumb line, and the pen approaches to be in a horizontal state when the user rotates the pen, that is, the angle of the pen relative to the plumb line is very large, when the angle of the pen relative to the plumb line is larger than a third preset threshold, the user can correct the dangerous bad learning behavior in time to prevent the student from being injured by the user, wherein the third preset threshold can be set by a technician in the field according to actual needs, and the embodiment of the user is not specifically limited to this user, the third correction prompt tone can be prerecorded by parents and teachers of students, and is used for preventing the target object from continuing to make bad learning behaviors such as pen turning and the like.
In an embodiment, the server determines the angle of the pen with respect to the plumb line according to the pen area, which may be implemented by the steps shown in fig. 6, which specifically include:
In a specific implementation, the server may determine the angle of the pen compared to the plumb line based on triangle principles, the server determines a first endpoint and a second endpoint of the pen in the area, and obtains coordinates of the first endpoint, a depth value of the first endpoint, coordinates of the second endpoint, and a depth value of the second endpoint, the first endpoint and the second endpoint being two points of the most marginal position in the area of the pen.
In a specific implementation, the server may calculate a two-dimensional distance between the first endpoint and the second endpoint according to the abscissa and the ordinate of the first endpoint and the abscissa of the second endpoint, and meanwhile, the server may calculate a horizontal distance between the abscissa and the ordinate of the first endpoint and the abscissa of the second endpoint according to the abscissa of the first endpoint and the abscissa of the second endpoint.
In a specific implementation, according to depth information carried by the depth map, the server may determine a three-dimensional distance between any two points in the depth map, and the server first calculates a third difference between the depth value of the first endpoint and the depth value of the second endpoint according to the depth value of the first endpoint and the depth value of the second endpoint, and then determines the three-dimensional distance between the first endpoint and the second endpoint according to the third difference and the two-dimensional distance between the first endpoint and the second endpoint.
In step 404, the angle of the pen with respect to the plumb line is determined based on the horizontal distance between the first endpoint and the second endpoint and the three-dimensional distance between the first endpoint and the second endpoint.
In a specific implementation, the horizontal distance between the first endpoint and the second endpoint is the opposite side of the included angle of the pen compared with the plumb line, the three-dimensional distance between the first endpoint and the second endpoint is the hypotenuse of the included angle of the pen compared with the plumb line, and the server can calculate the angle of the pen compared with the plumb line according to the right triangle principle.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.
Another embodiment of the present application relates to a bad learning behavior detection system, and the following describes implementation details of the bad learning behavior detection system of this embodiment in detail, where the following are provided only for facilitating understanding, and are not necessary for implementing this embodiment, and a schematic diagram of the bad learning behavior detection system of this embodiment may be as shown in fig. 7, and includes:
a depth camera module 501, configured to periodically obtain a depth map of the front of the target object based on a preset time interval.
And a target area detection module 502, configured to perform target area detection on the acquired depth map, and determine a hand area and a face area of the target object.
The calculating module 503 is configured to, when the face area and the hand area overlap, obtain a minimum value of depth values in the hand area as a first reference value, obtain a maximum value of depth values in a non-overlapping area of the face area and the hand area as a second reference value, and calculate a first difference between the first reference value and the second reference value.
The executing module 504 is configured to determine that the target object has an adverse learning behavior and play a preset first correction prompt tone when the absolute value of the first difference is smaller than a first preset threshold.
It should be noted that, all the modules involved in this embodiment are logic modules, and in practical application, one logic unit may be one physical unit, may also be a part of one physical unit, and may also be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present application, a unit that is not so closely related to solving the technical problem proposed by the present application is not introduced in the present embodiment, but this does not indicate that there is no other unit in the present embodiment.
Another embodiment of the present application relates to an electronic device, as shown in fig. 8, including: at least one processor 601; and a memory 602 communicatively coupled to the at least one processor 601; the memory 602 stores instructions executable by the at least one processor 601, and the instructions are executed by the at least one processor 601, so that the at least one processor 601 can execute the bad learning behavior detection method in the above embodiments.
Where the memory and processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting together one or more of the various circuits of the processor and the memory. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And the memory may be used to store data used by the processor in performing operations.
Another embodiment of the present application relates to a computer-readable storage medium storing a computer program. The computer program realizes the above-described method embodiments when executed by a processor.
That is, as can be understood by those skilled in the art, all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples for carrying out the present application, and that various changes in form and details may be made therein without departing from the spirit and scope of the present application in practice.
Claims (10)
1. A bad learning behavior detection method is characterized by comprising the following steps:
carrying out target area detection on the acquired depth map of the front side of the target object, and determining a hand area and a face area of the target object; wherein the depth map is periodically acquired based on a preset time interval;
under the condition that the face area and the hand area are overlapped, acquiring the minimum value of depth values in the hand area as a first reference value, and acquiring the maximum value of depth values in a non-overlapped area of the face area and the hand area as a second reference value;
calculating a first difference between the first reference value and the second reference value;
and if the absolute value of the first difference is smaller than a first preset threshold, determining that the target object has bad learning behavior, and playing a preset first correction prompt tone.
2. The method according to claim 1, wherein after the determining the hand region and the face region of the target object, the method further comprises:
under the condition that the human face area and the hand area are not overlapped, determining whether the target object is in a pen holding state or not according to the outline of the hand area;
if the target object is in a pen holding state, determining a pen area in the depth map;
under the condition that the face area is overlapped with the pen area, acquiring the minimum value of depth values in the pen area as a third reference value, and acquiring the maximum value of the depth values in the non-overlapped area of the face area and the pen area as a fourth reference value;
calculating a second difference between the third reference value and the fourth reference value;
and if the absolute value of the second difference is smaller than a second preset threshold, determining that the target object has bad learning behavior, and playing a preset second correction prompt tone.
3. The poor learning behavior detection method of claim 2, further comprising, after the determining the region of the pen in the depth map:
determining the angle of the pen relative to the plumb line according to the area of the pen under the condition that the human face area is not overlapped with the area of the pen;
and if the angle of the pen compared with the plumb line is larger than a third preset threshold value, determining that the target object has bad learning behaviors, and playing a preset third correction prompt tone.
4. The method according to claim 3, wherein the determining an angle of the pen with respect to a plumb line according to the area of the pen comprises:
determining a first endpoint and a second endpoint of the pen in the area of the pen, and acquiring a coordinate of the first endpoint, a depth value of the first endpoint, a coordinate of the second endpoint and a depth value of the second endpoint;
respectively calculating a two-dimensional distance and a horizontal distance between the first end point and the second end point according to the coordinates of the first end point and the coordinates of the second end point;
determining a three-dimensional distance between the first endpoint and the second endpoint according to a third difference between the depth value of the first endpoint and the depth value of the second endpoint and the two-dimensional distance;
and determining the angle of the pen relative to the plumb line according to the horizontal distance and the three-dimensional distance.
5. The method according to any one of claims 1 to 4, wherein the hand region includes a left-hand region and a right-hand region, and the obtaining a minimum value of depth values in the hand region as a first reference value when the face region overlaps the hand region includes:
and under the condition that the face area is overlapped with at least one of the left-hand area and the right-hand area, acquiring the minimum value of the depth values in the hand area overlapped with the face area as a first reference value.
6. The method according to any one of claims 1 to 4, wherein the obtaining of the minimum value of the depth values in the hand region is a first reference value, specifically: and acquiring the minimum value of the depth values in the overlapping area of the hand area and the face area as a first reference value.
7. The method according to any one of claims 1 to 4, wherein before the target region detection is performed on the acquired depth map of the front of the target object, the method comprises:
periodically acquiring continuous M-frame depth maps based on a preset time interval; wherein M is an integer greater than 1;
the target area detection of the acquired depth map of the front of the target object includes:
and respectively carrying out target area detection on the continuous M frames of depth maps.
8. A poor learning behavior detection system, comprising:
the depth camera module is used for periodically acquiring a depth map of the front of the target object based on a preset time interval;
the target area detection module is used for carrying out target area detection on the acquired depth map and determining a hand area and a face area of the target object;
the calculation module is used for acquiring the minimum value of the depth values in the hand area as a first reference value, acquiring the maximum value of the depth values in the non-overlapped area of the face area compared with the hand area as a second reference value and calculating a first difference value between the first reference value and the second reference value under the condition that the face area and the hand area are overlapped;
and the execution module is used for determining that the target object has bad learning behaviors and playing a preset first correction prompt tone under the condition that the absolute value of the first difference is smaller than a first preset threshold value.
9. An electronic device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of poor learning behavior detection as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the poor learning behavior detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210536462.XA CN114639157B (en) | 2022-05-18 | 2022-05-18 | Bad learning behavior detection method, system, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210536462.XA CN114639157B (en) | 2022-05-18 | 2022-05-18 | Bad learning behavior detection method, system, electronic device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114639157A true CN114639157A (en) | 2022-06-17 |
| CN114639157B CN114639157B (en) | 2022-11-22 |
Family
ID=81953171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210536462.XA Active CN114639157B (en) | 2022-05-18 | 2022-05-18 | Bad learning behavior detection method, system, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114639157B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107169456A (en) * | 2017-05-16 | 2017-09-15 | 湖南巨汇科技发展有限公司 | A kind of sitting posture detecting method based on sitting posture depth image |
| CN107169453A (en) * | 2017-05-16 | 2017-09-15 | 湖南巨汇科技发展有限公司 | A kind of sitting posture detecting method based on depth transducer |
| CN107948399A (en) * | 2017-10-31 | 2018-04-20 | 广东小天才科技有限公司 | Eye care method, device, mobile terminal and the storage medium of mobile terminal |
| CN110750160A (en) * | 2019-10-24 | 2020-02-04 | 京东方科技集团股份有限公司 | Drawing method and device for drawing screen based on gesture, drawing screen and storage medium |
| CN110859630A (en) * | 2019-11-26 | 2020-03-06 | 塔普翊海（上海）智能科技有限公司 | Posture corrector based on AR technology and correction method thereof |
| CN111240481A (en) * | 2020-01-10 | 2020-06-05 | 鄢家厚 | Read-write distance identification method based on smart watch |
| CN111862555A (en) * | 2019-04-30 | 2020-10-30 | 北京安云世纪科技有限公司 | Sitting posture correction control method and device, computer equipment and storage medium |
| CN111914667A (en) * | 2020-07-08 | 2020-11-10 | 浙江大华技术股份有限公司 | Smoking detection method and device |
| CN112307899A (en) * | 2020-09-27 | 2021-02-02 | 中国科学院宁波材料技术与工程研究所 | Facial posture detection and correction method and system based on deep learning |
| CN113347381A (en) * | 2021-05-24 | 2021-09-03 | 随锐科技集团股份有限公司 | Method and system for predicting inelegant lifting track |
| CN113658211A (en) * | 2021-07-06 | 2021-11-16 | 江汉大学 | User posture evaluation method and device and processing equipment |
| CN113657223A (en) * | 2021-08-05 | 2021-11-16 | 康佳集团股份有限公司 | Pen holding posture correction method and device, intelligent terminal and storage medium |
-
2022
- 2022-05-18 CN CN202210536462.XA patent/CN114639157B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107169456A (en) * | 2017-05-16 | 2017-09-15 | 湖南巨汇科技发展有限公司 | A kind of sitting posture detecting method based on sitting posture depth image |
| CN107169453A (en) * | 2017-05-16 | 2017-09-15 | 湖南巨汇科技发展有限公司 | A kind of sitting posture detecting method based on depth transducer |
| CN107948399A (en) * | 2017-10-31 | 2018-04-20 | 广东小天才科技有限公司 | Eye care method, device, mobile terminal and the storage medium of mobile terminal |
| CN111862555A (en) * | 2019-04-30 | 2020-10-30 | 北京安云世纪科技有限公司 | Sitting posture correction control method and device, computer equipment and storage medium |
| CN110750160A (en) * | 2019-10-24 | 2020-02-04 | 京东方科技集团股份有限公司 | Drawing method and device for drawing screen based on gesture, drawing screen and storage medium |
| CN110859630A (en) * | 2019-11-26 | 2020-03-06 | 塔普翊海（上海）智能科技有限公司 | Posture corrector based on AR technology and correction method thereof |
| CN111240481A (en) * | 2020-01-10 | 2020-06-05 | 鄢家厚 | Read-write distance identification method based on smart watch |
| CN111914667A (en) * | 2020-07-08 | 2020-11-10 | 浙江大华技术股份有限公司 | Smoking detection method and device |
| CN112307899A (en) * | 2020-09-27 | 2021-02-02 | 中国科学院宁波材料技术与工程研究所 | Facial posture detection and correction method and system based on deep learning |
| CN113347381A (en) * | 2021-05-24 | 2021-09-03 | 随锐科技集团股份有限公司 | Method and system for predicting inelegant lifting track |
| CN113658211A (en) * | 2021-07-06 | 2021-11-16 | 江汉大学 | User posture evaluation method and device and processing equipment |
| CN113657223A (en) * | 2021-08-05 | 2021-11-16 | 康佳集团股份有限公司 | Pen holding posture correction method and device, intelligent terminal and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 李彬等: "基于Kinect的课堂教学状态监测系统", 《传感器与微系统》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114639157B (en) | 2022-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10891873B2 (en) | Method and apparatus for monitoring learning and electronic device | |
| US20180079254A1 (en) | Digital pen with enhanced educational feedback | |
| WO2022166243A1 (en) | Method, apparatus and system for detecting and identifying pinching gesture | |
| CN108563780A (en) | Course content recommends method and apparatus | |
| CN109214962A (en) | Web education learning method, system, terminal and its storage medium | |
| CN110020628B (en) | Sitting posture detection method, system and equipment based on face detection and storage medium | |
| CN103903491A (en) | Method and device for achieving writing inspection | |
| CN107506162A (en) | Coordinate mapping method, computer-readable recording medium and projecting apparatus | |
| CN109240495A (en) | A kind of method and apparatus controlling automatic page turning | |
| CN108564833A (en) | Intelligent interaction dialog control method and device | |
| CN114639157B (en) | Bad learning behavior detection method, system, electronic device and storage medium | |
| CN103744971A (en) | Method and device for actively pushing information | |
| JP2018165760A (en) | Question control program, method for controlling question, and question controller | |
| CN112150777B (en) | Intelligent operation reminding device and method | |
| CN111401240B (en) | Classroom attention detection method, device, equipment and storage medium | |
| KR20220039440A (en) | Display apparatus and method for controlling the display apparatus | |
| CN113657223A (en) | Pen holding posture correction method and device, intelligent terminal and storage medium | |
| KR20190143742A (en) | The Device for improving the study concentration | |
| CN109408017A (en) | Image projecting method, device, terminal device and computer readable storage medium | |
| Patanasakpinyo | Flattening methods for adaptive location-based software to user abilities | |
| CN115100728A (en) | Method, apparatus, storage medium and program product for detecting visual acuity status | |
| US20220327956A1 (en) | Language teaching machine | |
| CN112203131B (en) | Prompting method and device based on display equipment and storage medium | |
| CN111178348B (en) | Method for tracking target object and sound box equipment | |
| CN113486789A (en) | Face area-based sitting posture detection method, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230823 Address after: Room 799-4, 7th Floor, Building A3A4, Zhong'an Chuanggu Science and Technology Park, No. 900 Wangjiang West Road, Gaoxin District, Hefei Free Trade Experimental Zone, Anhui Province, 230031 Patentee after: Anhui Lushenshi Technology Co.,Ltd. Address before: 230091 room 611-217, R & D center building, China (Hefei) international intelligent voice Industrial Park, 3333 Xiyou Road, high tech Zone, Hefei, Anhui Province Patentee before: Hefei lushenshi Technology Co.,Ltd. |