CN114639157B - Bad learning behavior detection method, system, electronic device and storage medium - Google Patents
Bad learning behavior detection method, system, electronic device and storage medium Download PDFInfo
- Publication number
- CN114639157B CN114639157B CN202210536462.XA CN202210536462A CN114639157B CN 114639157 B CN114639157 B CN 114639157B CN 202210536462 A CN202210536462 A CN 202210536462A CN 114639157 B CN114639157 B CN 114639157B
- Authority
- CN
- China
- Prior art keywords
- area
- pen
- hand
- target object
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000006399 behavior Effects 0.000 title claims abstract description 99
- 238000001514 detection method Methods 0.000 title claims abstract description 54
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000012937 correction Methods 0.000 claims abstract description 23
- 230000002411 adverse Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 2
- 230000020595 eating behavior Effects 0.000 claims 4
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 208000027418 Wounds and injury Diseases 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006378 damage Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 208000014674 injury Diseases 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000035622 drinking Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Educational Technology (AREA)
- Tourism & Hospitality (AREA)
- Educational Administration (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- General Health & Medical Sciences (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Human Resources & Organizations (AREA)
- General Business, Economics & Management (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application relates to the technical field of machine vision, and discloses a method and a system for detecting bad learning behaviors, electronic equipment and a storage medium, wherein the method comprises the following steps: performing target area detection on the acquired depth map of the front of the target object, and determining a hand area and a face area of the target object; under the condition that the face area and the hand area are overlapped, acquiring the minimum value of the depth values in the hand area as a first reference value, and acquiring the maximum value of the depth values in the non-overlapped area of the face area and the hand area as a second reference value; calculating a first difference between the first reference value and the second reference value; if the absolute value of the first difference is smaller than the first preset threshold, it is determined that the target object has bad learning behaviors, and a preset first correction prompt tone is played.
Description
Technical Field
The embodiment of the application relates to the technical field of machine vision, in particular to a method and a system for detecting bad learning behaviors, electronic equipment and a storage medium.
Background
Along with the rapid development of communication technology and the complexity and changeability of epidemic situation prevention and control situation, the online teaching becomes an inseparable part of students, teachers give lessons remotely through the online teaching platform, students join online classes at home to listen to lessons, the online teaching platform provides multiple functions for simulating real classes, such as student sign-in, remote questioning, assignment work, class tests and the like, the inherent datamation, tabulation and standardization attributes of the online teaching platform provide multiple conveniences for teacher teaching, the course selection of various famous students is provided for students, the distance barrier is broken through, and the learning breadth and the learning depth of the students are improved.
It can be understood that student's self-restraint ability has not been established completely yet, the student often can make subconsciously when in class, think, write homework and eat one's hands, grab the face, bite the nib, change bad behaviors such as pen, these behaviors are unclean, insanitary, unsightly, and can also cause the injury to student's health, under the condition in real classroom, the teacher can overlook whole class, and can walk around in giving lessons, in case find the student make these bad learning behaviors, the teacher can in time stop, correct, help the student to develop good custom, avoid the emergence of dangerous condition.
However, in the teacher interface of the online teaching platform, students become a small window, and the teacher usually can only see the facial pictures of the students through the window, and the teacher cannot frequently browse and switch tens of small windows in the online class, and the small window has low picture quality and is relatively fuzzy, so that the teacher cannot timely find whether the students make bad learning behaviors, parents also have their own work and cannot accompany the children to online the class at any moment, so that the bad learning behaviors of the students cannot be timely corrected, and the students can easily develop bad learning habits.
Disclosure of Invention
An object of the embodiments of the present application is to provide a method, a system, an electronic device, and a storage medium for detecting bad learning behaviors, which can automatically find and correct the bad learning behaviors of students in real time on the premise of protecting the privacy of the students, so as to guide the students to develop good learning habits.
In order to solve the above technical problem, an embodiment of the present application provides a method for detecting an unfavorable learning behavior, including the following steps: carrying out target area detection on the acquired depth map of the front side of the target object, and determining a hand area and a face area of the target object; wherein the depth map is periodically acquired based on a preset time interval; under the condition that the face area and the hand area are overlapped, acquiring the minimum value of depth values in the hand area as a first reference value, and acquiring the maximum value of depth values in a non-overlapped area of the face area and the hand area as a second reference value; calculating a first difference between the first reference value and the second reference value; and if the absolute value of the first difference is smaller than a first preset threshold, determining that the target object has bad learning behaviors, and playing a preset first correction prompt tone.
An embodiment of the present application further provides a bad learning behavior detection system, including: the depth camera module is used for periodically acquiring a depth map of the front side of the target object based on a preset time interval; the target area detection module is used for carrying out target area detection on the acquired depth map and determining a hand area and a face area of the target object; the calculation module is used for acquiring the minimum value of the depth values in the hand area as a first reference value, acquiring the maximum value of the depth values in the non-overlapped area of the face area compared with the hand area as a second reference value and calculating a first difference value between the first reference value and the second reference value under the condition that the face area and the hand area are overlapped; and the execution module is used for determining that the target object has bad learning behaviors and playing a preset first correction prompt tone under the condition that the absolute value of the first difference is smaller than a first preset threshold value.
An embodiment of the present application further provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the above-described method of bad learning behavior detection.
Embodiments of the present application further provide a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the method for detecting adverse learning behavior as described above is implemented.
The method, the system, the electronic device and the storage medium for detecting the bad learning behavior provided by the embodiment of the application periodically detect whether the target object is in a front depth map based on a preset time interval, detect a target area of the depth map, determine a hand area and a face area of the target object, when the face area and the hand area of the target object are overlapped, acquire a minimum value of depth values in the hand area as a first reference value, acquire a maximum value of depth values in a non-overlapped area of the face area compared with the hand area as a second reference value, calculate a first difference value between the first reference value and the second reference value, if an absolute value of the difference value between the first reference value and the second reference value is smaller than a first preset threshold value, determine that the target object has the bad learning behavior, play a preset first correction prompt tone, consider that a teacher cannot frequently browse and switch a picture window during online class, cannot find whether the bad learning behavior cannot be found or not, and automatically set a face area to be suitable for guiding students to take the bad learning behavior, and automatically detect whether the student's face area is too close to the student's learning behavior, and the student's habit can be detected in time.
In addition, after the determining the hand region and the face region of the target object, the method includes: under the condition that the human face area and the hand area are not overlapped, determining whether the target object is in a pen holding state or not according to the outline of the hand area; if the target object is in a pen holding state, determining a pen area in the depth map; under the condition that the face area is overlapped with the pen area, acquiring the minimum value of depth values in the pen area as a third reference value, and acquiring the maximum value of the depth values in the non-overlapped area of the face area and the pen area as a fourth reference value; calculating a second difference between the third reference value and the fourth reference value; if the absolute value of the second difference is smaller than a second preset threshold value, determining that the target object has bad learning behaviors, playing a preset second correction prompt tone, considering that behaviors such as pen-biting, pen-using 24636 and the like are also bad learning behaviors which are unclean, unhygienic and unsightly, and even though hands of students do not touch the faces, the pens can be regarded as extension of the hands, and only detecting whether a face region overlaps with the hand region and cannot detect the bad learning behaviors such as pen-biting or the like.
Additionally, after the determining the area of the pen in the depth map, further comprising: determining the angle of the pen relative to the plumb line according to the area of the pen under the condition that the human face area is not overlapped with the area of the pen; if the angle of the pen compared with the plumb line is larger than a third preset threshold value, it is determined that the target object has a bad learning behavior, a preset third correction prompt tone is played, the pen turning is also an unclean, unsightly and dangerous bad learning behavior is considered, and once the user is out of hand, the user can scratch the user, so that the pen is detected to be in a pen holding state when the user normally holds the pen, and under the condition that the face region of the target object is not overlapped with the region of the pen, the angle of the pen compared with the plumb line is obtained and detected, when the user normally holds the pen, the pen is closer to parallel with the plumb line and is smaller in angle, the pen approaches to a horizontal state when the pen is turned, namely the angle of the pen compared with the plumb line is larger, and when the angle of the pen compared with the plumb line is larger than the third preset threshold value, the dangerous bad learning behavior can be corrected in time, and the student is prevented from receiving injuries which can be avoided.
In addition, the hand region includes a left hand region and a right hand region, and when the face region overlaps the hand region, acquiring a minimum value of the depth values in the hand region as a first reference value includes: under the condition that the face area is overlapped with at least one of the left hand area and the right hand area, the minimum value of the depth value in the hand area overlapped with the face area is obtained to be a first reference value, in the depth map, the left hand and the right hand of a student are detected generally, the student can possibly scratch the face and grab the face with only one hand, the other hand is normally put on a desk, and at the moment, the hand not overlapped with the face possibly influences the hand overlapped with the face, so that the minimum value of the depth value is searched in the hand area corresponding to the hand overlapped with the face area, and the accuracy of detection of bad learning behaviors can be improved.
In addition, the acquiring of the minimum value of the depth values in the hand region is a first reference value, and specifically includes: the minimum value of the depth values in the overlap area of the hand area and the face area is obtained as a first reference value, generally speaking, children have good flexibility and may make various strange motions, and therefore the minimum value of the depth values searched in the whole area of the hand area may not be accurate enough as the first reference value, and therefore the accuracy of detecting the bad learning behavior can be further improved by only searching the minimum value of the depth values in the overlap area of the hand area and the face area as the first reference value.
In addition, before the target area detection is performed on the acquired depth map of the front of the target object, the method includes: periodically acquiring continuous M-frame depth maps based on a preset time interval; the target area detection of the acquired depth map of the front of the target object includes: the target area detection is respectively carried out on the continuous M-frame depth maps, generally speaking, the short-time face touching by hands is not a bad learning behavior, but the long-time face touching by hands can basically judge that the bad learning behavior is the bad learning behavior, so that the depth maps in a period of time, namely the continuous M-frame depth maps, are obtained at one time, the target area detection is carried out on the basis of the continuous M-frame depth maps, and the accuracy of the bad learning behavior detection can be further improved.
Drawings
One or more embodiments are illustrated by the corresponding figures in the drawings, which are not meant to be limiting.
Fig. 1 is a first flowchart of a bad learning behavior detection method according to an embodiment of the present application;
FIG. 2 is a schematic illustration of a student grabbing a face according to an embodiment of the present application;
fig. 3 is a flowchart ii of a bad learning behavior detection method according to an embodiment of the present application;
FIG. 4 is a schematic illustration of a student pen 24636 with a face provided by an embodiment of the present application;
fig. 5 is a flowchart three of a bad learning behavior detection method according to an embodiment of the present application;
FIG. 6 is a flow chart of determining an angle of a pen with respect to a plumb line based on a region of the pen in one embodiment of the present application;
FIG. 7 is a schematic diagram of a bad learning behavior detection system of another embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to another embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that in the examples of the present application, numerous technical details are set forth in order to provide a better understanding of the present application. However, the technical solution claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not constitute any limitation to the specific implementation manner of the present application, and the embodiments may be mutually incorporated and referred to without contradiction.
An embodiment of the present application relates to a method for detecting an undesired learning behavior, which is applied to an electronic device, where the electronic device may be a terminal or a server, and the electronic device in this embodiment and the following embodiments are described by taking the server as an example.
The specific flow of the bad learning behavior detection method of this embodiment may be as shown in fig. 1, and includes:
Specifically, the server may periodically perform, based on a preset time interval, detection once if the acquisition is not completed once, that is, perform target area detection on the depth map of the front side of the acquired target object, and determine a hand area and a face area of the target object in the depth map, where the preset time interval may be set by a person skilled in the art according to actual needs, and the preset time interval is not set too short, and if the preset time interval is set too short, a large amount of shooting and computing resources are occupied, and it is easy to excessively interrupt a normal lesson of a student, and it is easy to make a overuse of the lesson.
In a specific implementation, the server is in communication with the depth camera, the depth camera may obtain a depth map of a target scene, the depth camera is directly shooting the front of the target object, the server periodically wakes up and calls the depth camera based on a preset time interval, and the depth camera shoots the front of the target object to obtain the depth map and transmits the depth map to the server.
In one example, when the server performs target area detection on the acquired depth map of the front side of the target object, the server may traverse the depth value of each pixel in the depth map, perform connected domain detection based on the depth value of each pixel, thereby obtaining a plurality of basic areas, and then determine the hand area and the face area of the target object from each basic area according to the contour shape of each basic area.
In another example, when the server performs target area detection on the acquired depth map of the front of the target object, the server may input the depth map of the front of the target object into a pre-trained target area detection model, and acquire a hand area and a face area of the target object output by the target area detection model.
In an example, the server may periodically obtain continuous M-frame depth maps based on a preset time interval, and the server may perform target region detection on the continuous M-frame depth maps respectively, where M is an integer greater than 1, and generally, when a face is touched by a hand briefly, it is not a bad learning behavior, but when a face is touched by a hand for a long time, it may be basically determined that the learning behavior is a bad learning behavior, so that the accuracy of detecting the bad learning behavior may be further improved by obtaining the depth maps within a period of time, that is, the continuous M-frame depth maps at a time, and performing the target region detection based on the continuous M-frame depth maps.
And 102, under the condition that the face area and the hand area are overlapped, acquiring the minimum value of the depth values in the hand area as a first reference value, and acquiring the maximum value of the depth values in the non-overlapped area of the face area and the hand area as a second reference value.
Specifically, after the server determines the hand region and the face region of the target object in the depth map, it may detect whether the face region of the target object overlaps with the hand region of the target object, and when determining that the face region of the target object overlaps with the hand region, the server may acquire a minimum value of depth values in the hand region of the target object as a first reference value and a maximum value of depth values in a non-overlapping region of the face region of the target object compared with the hand region as a second reference value.
It can be understood that, for the front of the target object, the human face area overlaps with the hand area, and only the target object blocks the face with the hand, for example, when the target object performs a motion of grabbing the face, if the target object holds the head with the hand, the hand is completely blocked by the head, and the hand area cannot be detected in the depth map of the front of the target object.
In one example, the hand area includes a left hand area and a right hand area, the server may detect whether the face area of the target object overlaps at least one of the left hand area and the right hand area after determining the hand area and the face area of the target object, and in a case that the face area of the target object overlaps at least one of the left hand area and the right hand area, the server obtains a minimum value of depth values in the hand area overlapping the face area as a first reference value and obtains a maximum value of depth values in the face area compared to a non-overlapping area of the hand area as a second reference value.
In one example, the server acquires that the minimum value of the depth values in the hand region of the target object is the first reference value, specifically, the minimum value of the depth values in the overlap region of the hand region of the target object with respect to the face region is the first reference value, generally speaking, children have good flexibility and may make various strange actions, so that the minimum value of the depth values searched in the whole area of the hand region may not be accurate enough as the first reference value, and therefore, the application searches the minimum value of the depth values in the overlap region of the hand region with respect to the face region as the first reference value, and further improves the accuracy of detection of poor learning behaviors.
And 104, if the absolute value of the first difference is smaller than a first preset threshold, determining that the target object has bad learning behavior, and playing a preset first correction prompt tone.
In specific implementation, after the server acquires the first reference value and the second reference value, a first difference between the first reference value and the second reference value may be calculated, and whether an absolute value of the first difference between the first reference value and the second reference value is smaller than a first preset threshold is judged, if the absolute value of the first difference is smaller than the first preset threshold, it is determined that the target object has poor learning behaviors such as face catching and hand eating, and the server may play a preset first correction prompt tone to remind the target object, where the first preset threshold may be set by a technician in the field according to actual needs.
In one example, the target subject may make poor learning behaviors such as grabbing a face, eating a hand, etc. as shown in fig. 2.
In this embodiment, the server periodically obtains a depth map of the front side of the target object based on a preset time interval, performs target area detection on the depth map, determines a hand area and a face area of the target object, obtains a minimum value of depth values in the hand area as a first reference value and obtains a maximum value of depth values in a non-overlapping area of the face area compared with the hand area as a second reference value when the face area and the hand area of the target object overlap, calculates a first difference between the first reference value and the second reference value, determines that the target object has an unfavorable learning behavior if an absolute value of the difference between the first reference value and the second reference value is smaller than a first preset threshold, and plays a preset first correction prompt tone, considering that a teacher cannot frequently browse and switch picture windows of students in online class, whether the students make bad learning behaviors or not cannot be timely found, the embodiment of the application periodically detects whether the face area of a target object is overlapped with the hand area, if the overlap exists, the hand area is too close to the face area, and if the overlap exists, the target object makes the bad learning behaviors such as face grabbing and hand eating.
In an embodiment, after determining the hand region and the face region of the target object, the server may perform the detection of the poor learning behavior on the target object through the steps shown in fig. 3, which specifically includes:
In a specific implementation, although there is no overlap between the face region and the hand region of the target object, that is, the target object does not make undesirable learning behaviors such as grabbing a face and eating a hand, a tool held by the target object, for example, a pen, can be used as an extension of the hand to perform other undesirable learning behaviors such as biting a pen head and using the pen 24636 to make a face, and the server can determine whether the target object is in a pen-grabbing state according to the contour of the hand region under the condition that it is determined that there is no overlap between the face region and the hand region of the target object, so as to prepare for further detection of the undesirable learning behaviors.
In one example, the server may input the contour of the hand region of the target object into a pre-trained pen-holding state detection model, and obtain a detection result of the contour of the hand region of the target object, which is output by the pen-holding state detection model, where the detection result is a pen-holding state and a non-pen-holding state.
In a specific implementation, if the server determines that the target object is in a pen-holding state, the server may determine a pen region in the depth map, and since the lengths, sizes, and shapes of the pens are different, the server may obtain depth values of points within a certain search range with the hand region as a center, thereby determining the pen region in the depth map.
In step 203, under the condition that the face area and the pen area are overlapped, the minimum value of the depth values in the pen area is obtained as a third reference value, and the maximum value of the depth values in the non-overlapped area of the face area and the pen area is obtained as a fourth reference value.
In a specific implementation, after the server determines a pen region in the depth map, it may determine whether there is overlap between a face region of the target object and the pen region, and when it is determined that there is overlap between the face region of the target object and the pen region, the server may obtain a minimum value of depth values in the pen region as a third reference value, and obtain a maximum value of depth values in a non-overlapping region of the face region and the pen region as a fourth reference value, where the overlap between the face region of the target object and the pen region indicates that the target object blocks a face with the pen, and normal writing does not occur, that is, the target object is likely to perform an undesirable learning behavior.
In step 204, a second difference between the third reference value and the fourth reference value is calculated.
In step 205, if the absolute value of the second difference is smaller than the second preset threshold, it is determined that the target object has an adverse learning behavior, and a preset second correction prompt tone is played.
In specific implementation, after the server acquires the third reference value and the fourth reference value, a second difference value between the third reference value and the fourth reference value can be calculated, whether the absolute value of the second difference value between the third reference value and the fourth reference value is smaller than a second preset threshold value or not is judged, if the absolute value of the second difference value is smaller than the second preset threshold value, it is determined that undesirable learning behaviors such as pen biting and face touching exist in the target object, and the server can play a preset second correction prompt tone to remind the target object, wherein the second preset threshold value can be set by technicians in the field according to actual needs, and the embodiment of the application is not specifically limited to this, and the second correction prompt tone can be prerecorded by students parents and teachers, and is used for drinking the target object to continue to make undesirable learning behaviors such as pen biting and face touching.
In one example, the target object may make an undesirable learning behavior with a pen \24636.
In the embodiment, considering that behaviors such as pen-biting, face-using, etc. are also poor learning behaviors such as unclean, unsanitary and unsightly, although hands of students do not touch the face, the pen can be regarded as extension of the hands, and poor learning behaviors such as pen-biting can not be detected only by detecting whether a face region overlaps with a hand region, and the application detects whether a target object holds the pen under the condition that the face region does not overlap with the hand region and detects whether the face region overlaps with the pen region, so that the poor learning behaviors of the students can be further found and corrected, and the students can be better guided to develop good learning habits.
In an embodiment, after determining the pen region in the depth map, the server may perform poor learning behavior detection on the target object through the steps shown in fig. 5, which specifically include:
In a specific implementation, considering that the pen turning is also an unclean, unsightly and dangerous bad learning behavior, once the user is out of hand, the user can scratch himself, so that the pen can acquire and detect the angle of the pen relative to the plumb line when the user detects that the target object is in a pen holding state and the face area of the target object is not overlapped with the area of the pen, when the user normally holds the pen, the pen is closer to be parallel to the plumb line and has a smaller angle, and when the user turns the pen, the pen approaches to a horizontal state, that is, the angle of the pen relative to the plumb line is very large.
In an embodiment, the server determines the angle of the pen with respect to the plumb line according to the pen area, which may be implemented by the steps shown in fig. 6, which specifically include:
In a specific implementation, the server may determine the angle of the pen compared to the plumb line based on triangle principles, the server determines a first endpoint and a second endpoint of the pen in the area, and obtains coordinates of the first endpoint, a depth value of the first endpoint, coordinates of the second endpoint, and a depth value of the second endpoint, the first endpoint and the second endpoint being two points of the most marginal position in the area of the pen.
In a specific implementation, the server may calculate a two-dimensional distance between the first endpoint and the second endpoint according to the abscissa and the ordinate of the first endpoint and the abscissa of the second endpoint, and meanwhile, the server may calculate a horizontal distance between the abscissa and the ordinate of the first endpoint and the abscissa of the second endpoint according to the abscissa of the first endpoint and the abscissa of the second endpoint.
In step 403, a three-dimensional distance between the first endpoint and the second endpoint is determined according to a third difference between the depth value of the first endpoint and the depth value of the second endpoint and the two-dimensional distance between the first endpoint and the second endpoint.
In a specific implementation, according to depth information carried by the depth map, the server may determine a three-dimensional distance between any two points in the depth map, and the server first calculates a third difference between the depth value of the first endpoint and the depth value of the second endpoint according to the depth value of the first endpoint and the depth value of the second endpoint, and then determines the three-dimensional distance between the first endpoint and the second endpoint according to the third difference and the two-dimensional distance between the first endpoint and the second endpoint.
In a specific implementation, the horizontal distance between the first end point and the second end point is the opposite side of the included angle of the pen compared with the plumb line, the three-dimensional distance between the first end point and the second end point is the hypotenuse of the included angle of the pen compared with the plumb line, and the server can calculate the angle of the pen compared with the plumb line according to the right triangle principle.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are within the scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.
Another embodiment of the present application relates to a bad learning behavior detection system, and the following describes implementation details of the bad learning behavior detection system of this embodiment in detail, where the following are provided only for facilitating understanding, and are not necessary for implementing this embodiment, and a schematic diagram of the bad learning behavior detection system of this embodiment may be as shown in fig. 7, and includes:
a depth camera module 501, configured to periodically obtain a depth map of the front of the target object based on a preset time interval.
A target area detection module 502, configured to perform target area detection on the acquired depth map, and determine a hand area and a face area of the target object.
The calculating module 503 is configured to, when the face area and the hand area overlap, obtain a minimum value of depth values in the hand area as a first reference value, obtain a maximum value of depth values in a non-overlapping area of the face area and the hand area as a second reference value, and calculate a first difference between the first reference value and the second reference value.
The executing module 504 is configured to determine that the target object has an adverse learning behavior and play a preset first correction prompt tone when the absolute value of the first difference is smaller than a first preset threshold.
It should be noted that, all modules involved in this embodiment are logic modules, and in practical application, one logic unit may be one physical unit, may also be a part of one physical unit, and may also be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present application, a unit which is not so closely related to solve the technical problem proposed by the present application is not introduced in the present embodiment, but this does not indicate that no other unit exists in the present embodiment.
Another embodiment of the present application relates to an electronic device, as shown in fig. 8, including: at least one processor 601; and a memory 602 communicatively coupled to the at least one processor 601; the memory 602 stores instructions executable by the at least one processor 601, and the instructions are executed by the at least one processor 601, so that the at least one processor 601 can execute the bad learning behavior detection method in the above embodiments.
Where the memory and processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting together one or more of the various circuits of the processor and the memory. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over a wireless medium through an antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. While the memory may be used to store data used by the processor in performing operations.
Another embodiment of the present application relates to a computer-readable storage medium storing a computer program. The computer program realizes the above-described method embodiments when executed by a processor.
That is, as can be understood by those skilled in the art, all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples for carrying out the present application, and that various changes in form and details may be made therein without departing from the spirit and scope of the present application in practice.
Claims (9)
1. A method for detecting poor learning behavior, comprising:
performing target area detection on the acquired depth map of the front of the target object, and determining a hand area and a face area of the target object; wherein the depth map is periodically acquired based on a preset time interval;
under the condition that the face area and the hand area are overlapped, acquiring the minimum value of depth values in the hand area as a first reference value, and acquiring the maximum value of depth values in a non-overlapped area of the face area and the hand area as a second reference value;
calculating a first difference between the first reference value and the second reference value;
if the absolute value of the first difference is smaller than a first preset threshold, determining that the target object has bad learning behavior, and playing a preset first correction prompt tone; wherein the adverse learning behaviors comprise a face grabbing behavior and a hand eating behavior, and the first correction prompt tone is used for preventing the target object from continuing to make the face grabbing behavior and the hand eating behavior;
after the determining the hand region and the face region of the target object, the method includes:
under the condition that the human face area and the hand area are not overlapped, determining whether the target object is in a pen holding state or not according to the outline of the hand area;
if the target object is in a pen holding state, determining a pen area in the depth map;
under the condition that the face area is overlapped with the pen area, acquiring the minimum value of the depth values in the pen area as a third reference value, and acquiring the maximum value of the depth values in the non-overlapped area of the face area and the pen area as a fourth reference value;
calculating a second difference between the third reference value and the fourth reference value;
if the absolute value of the second difference is smaller than a second preset threshold, determining that the target object has an adverse learning behavior, and playing a preset second correction prompt tone, wherein the adverse learning behavior further comprises a pen-biting behavior and a pen-using behavior of 24636, and the second correction prompt tone is used for preventing the target object from continuing to make the pen-biting behavior and the pen-using behavior of 24636.
2. The bad learning behavior detection method according to claim 1, further comprising, after the determining the region of the pen in the depth map:
determining the angle of the pen relative to the plumb line according to the area of the pen under the condition that the human face area is not overlapped with the area of the pen;
and if the angle of the pen compared with the plumb line is larger than a third preset threshold value, determining that the target object has bad learning behaviors, and playing a preset third correction prompt tone.
3. The method according to claim 2, wherein the determining an angle of the pen with respect to a plumb line according to the area of the pen comprises:
determining a first end point and a second end point of the pen in the area of the pen, and acquiring a coordinate of the first end point, a depth value of the first end point, a coordinate of the second end point and a depth value of the second end point;
respectively calculating a two-dimensional distance and a horizontal distance between the first end point and the second end point according to the coordinate of the first end point and the coordinate of the second end point;
determining a three-dimensional distance between the first endpoint and the second endpoint according to a third difference between the depth value of the first endpoint and the depth value of the second endpoint and the two-dimensional distance;
and determining the angle of the pen relative to the plumb line according to the horizontal distance and the three-dimensional distance.
4. The method according to any one of claims 1 to 3, wherein the hand region includes a left-hand region and a right-hand region, and the acquiring a minimum value of the depth values in the hand region as a first reference value when the face region overlaps the hand region includes:
and under the condition that the face area is overlapped with at least one of the left-hand area and the right-hand area, acquiring the minimum value of the depth values in the hand area overlapped with the face area as a first reference value.
5. The method for detecting an ill-learned behavior according to any one of claims 1 to 3, wherein the obtaining of the minimum value of the depth values in the hand region is a first reference value, specifically: and acquiring the minimum value of the depth values in the overlapping area of the hand area and the face area as a first reference value.
6. The method according to any one of claims 1 to 3, wherein before the target region detection is performed on the acquired depth map of the front of the target object, the method comprises:
periodically acquiring continuous M-frame depth maps based on a preset time interval; wherein M is an integer greater than 1;
the target area detection of the acquired depth map of the front of the target object includes:
and respectively carrying out target area detection on the continuous M-frame depth maps.
7. A bad learning behavior detection system, comprising:
the depth camera module is used for periodically acquiring a depth map of the front of the target object based on a preset time interval;
the target area detection module is used for carrying out target area detection on the acquired depth map and determining a hand area and a face area of the target object;
the calculation module is used for acquiring the minimum value of the depth values in the hand area as a first reference value, acquiring the maximum value of the depth values in the non-overlapped area of the face area compared with the hand area as a second reference value and calculating a first difference value between the first reference value and the second reference value under the condition that the face area and the hand area are overlapped;
the execution module is used for determining that the target object has bad learning behaviors and playing a preset first correction prompt tone under the condition that the absolute value of the first difference value is smaller than a first preset threshold value; wherein the adverse learning behaviors comprise a face grabbing behavior and a hand eating behavior, and the first correction prompt tone is used for preventing the target object from continuing to make the face grabbing behavior and the hand eating behavior;
after the determining the hand region and the face region of the target object, the method includes:
under the condition that the human face area and the hand area are not overlapped, determining whether the target object is in a pen holding state or not according to the outline of the hand area;
if the target object is in a pen-holding state, determining a pen area in the depth map;
under the condition that the face area is overlapped with the pen area, acquiring the minimum value of the depth values in the pen area as a third reference value, and acquiring the maximum value of the depth values in the non-overlapped area of the face area and the pen area as a fourth reference value;
calculating a second difference between the third reference value and the fourth reference value;
if the absolute value of the second difference value is smaller than a second preset threshold value, determining that the target object has poor learning behaviors, and playing a preset second correction prompt tone, wherein the poor learning behaviors further comprise a pen-biting behavior and a pen-using 24636, and the second correction prompt tone is used for preventing the target object from continuing to make the pen-biting behavior and the pen-using 24636.
8. An electronic device, comprising:
at least one processor; and (c) a second step of,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of bad learning behavior detection according to any of claims 1 to 6.
9. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the poor learning behavior detection method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210536462.XA CN114639157B (en) | 2022-05-18 | 2022-05-18 | Bad learning behavior detection method, system, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210536462.XA CN114639157B (en) | 2022-05-18 | 2022-05-18 | Bad learning behavior detection method, system, electronic device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114639157A CN114639157A (en) | 2022-06-17 |
| CN114639157B true CN114639157B (en) | 2022-11-22 |
Family
ID=81953171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210536462.XA Active CN114639157B (en) | 2022-05-18 | 2022-05-18 | Bad learning behavior detection method, system, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114639157B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118214828A (en) * | 2024-01-29 | 2024-06-18 | 南京科瀚教育科技有限公司 | Intelligent key monitoring image selection system for remote education |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110750160A (en) * | 2019-10-24 | 2020-02-04 | 京东方科技集团股份有限公司 | Drawing method and device for drawing screen based on gesture, drawing screen and storage medium |
| CN111862555A (en) * | 2019-04-30 | 2020-10-30 | 北京安云世纪科技有限公司 | Sitting posture correction control method and device, computer equipment and storage medium |
| CN111914667A (en) * | 2020-07-08 | 2020-11-10 | 浙江大华技术股份有限公司 | Smoking detection method and device |
| CN113347381A (en) * | 2021-05-24 | 2021-09-03 | 随锐科技集团股份有限公司 | Method and system for predicting inelegant lifting track |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107169453B (en) * | 2017-05-16 | 2020-07-17 | 湖南巨汇科技发展有限公司 | Sitting posture detection method based on depth sensor |
| CN107169456B (en) * | 2017-05-16 | 2019-08-09 | 湖南巨汇科技发展有限公司 | A kind of sitting posture detecting method based on sitting posture depth image |
| CN107948399A (en) * | 2017-10-31 | 2018-04-20 | 广东小天才科技有限公司 | Eye protection method and device for mobile terminal, mobile terminal and storage medium |
| CN110859630B (en) * | 2019-11-26 | 2022-07-19 | 塔普翊海（上海）智能科技有限公司 | Posture corrector based on AR technology and correction method thereof |
| CN111240481B (en) * | 2020-01-10 | 2021-02-09 | 鄢家厚 | Read-write distance identification method based on smart watch |
| CN112307899A (en) * | 2020-09-27 | 2021-02-02 | 中国科学院宁波材料技术与工程研究所 | Facial posture detection and correction method and system based on deep learning |
| CN113658211B (en) * | 2021-07-06 | 2024-02-09 | 江汉大学 | User gesture evaluation method and device and processing equipment |
| CN113657223A (en) * | 2021-08-05 | 2021-11-16 | 康佳集团股份有限公司 | Pen holding posture correction method and device, intelligent terminal and storage medium |
-
2022
- 2022-05-18 CN CN202210536462.XA patent/CN114639157B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111862555A (en) * | 2019-04-30 | 2020-10-30 | 北京安云世纪科技有限公司 | Sitting posture correction control method and device, computer equipment and storage medium |
| CN110750160A (en) * | 2019-10-24 | 2020-02-04 | 京东方科技集团股份有限公司 | Drawing method and device for drawing screen based on gesture, drawing screen and storage medium |
| CN111914667A (en) * | 2020-07-08 | 2020-11-10 | 浙江大华技术股份有限公司 | Smoking detection method and device |
| CN113347381A (en) * | 2021-05-24 | 2021-09-03 | 随锐科技集团股份有限公司 | Method and system for predicting inelegant lifting track |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114639157A (en) | 2022-06-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2018233398A1 (en) | Method, device, and electronic apparatus for monitoring learning | |
| Jones et al. | Students' accuracy of measurement estimation: Context, units, and logical thinking | |
| CN104035557B (en) | Kinect action identification method based on joint activeness | |
| CN114639157B (en) | Bad learning behavior detection method, system, electronic device and storage medium | |
| CN110020628B (en) | Sitting posture detection method, system and equipment based on face detection and storage medium | |
| CN103426337A (en) | Information processing device and information processing method | |
| CN103778818A (en) | Method and device for prompting writing errors | |
| CN109214962A (en) | Web education learning method, system, terminal and its storage medium | |
| CN111428686A (en) | Student interest preference evaluation method, device and system | |
| Morash et al. | A review of haptic spatial abilities in the blind | |
| CN107506162A (en) | Coordinate mapping method, computer-readable recording medium and projecting apparatus | |
| Larrue et al. | Influence of body-centered information on the transfer of spatial learning from a virtual to a real environment | |
| KR20190143742A (en) | The Device for improving the study concentration | |
| CN112150777B (en) | Intelligent operation reminding device and method | |
| Kayukawa et al. | Smartphone-based assistance for blind people to stand in lines | |
| CN108090119A (en) | Method, device, mobile terminal and storage medium for displaying answers to questions | |
| CN113989832A (en) | Gesture recognition method and device, terminal equipment and storage medium | |
| KR20220039440A (en) | Display apparatus and method for controlling the display apparatus | |
| JP2018165760A (en) | Question control program, method for controlling question, and question controller | |
| CN113657223A (en) | Pen holding posture correction method and device, intelligent terminal and storage medium | |
| Patanasakpinyo | Flattening methods for adaptive location-based software to user abilities | |
| US20220327956A1 (en) | Language teaching machine | |
| CN113486789A (en) | Face area-based sitting posture detection method, electronic equipment and storage medium | |
| CN112714328A (en) | Live course student posture prompting method and device and electronic equipment | |
| CN113920530A (en) | Monitoring method, system and storage medium for intelligently supervising student learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230823 Address after: Room 799-4, 7th Floor, Building A3A4, Zhong'an Chuanggu Science and Technology Park, No. 900 Wangjiang West Road, Gaoxin District, Hefei Free Trade Experimental Zone, Anhui Province, 230031 Patentee after: Anhui Lushenshi Technology Co.,Ltd. Address before: 230091 room 611-217, R & D center building, China (Hefei) international intelligent voice Industrial Park, 3333 Xiyou Road, high tech Zone, Hefei, Anhui Province Patentee before: Hefei lushenshi Technology Co.,Ltd. |