CN112651865A

CN112651865A - Behavior state prompting method and device, electronic equipment and storage medium

Info

Publication number: CN112651865A
Application number: CN202011601725.8A
Authority: CN
Inventors: 时占; 闫研
Original assignee: Beijing Sensetime Technology Development Co Ltd
Current assignee: Beijing Sensetime Technology Development Co Ltd
Priority date: 2020-12-30
Filing date: 2020-12-30
Publication date: 2021-04-13

Abstract

The disclosure provides a behavior state prompting method, a behavior state prompting device, electronic equipment and a storage medium, wherein the prompting method comprises the following steps: acquiring a video stream by a video acquisition component, wherein the video stream comprises a target management object in a first state; detecting a behavior state of the target management object based on the video stream; and generating management prompt information aiming at the target management object according to the detected behavior state of the target management object and the time planning information.

Description

Behavior state prompting method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of computer vision technologies, and in particular, to a method and an apparatus for prompting a behavior state, an electronic device, and a storage medium.

Background

Considering that most people have no time idea in the child stage, if the children are not effectively guided in the learning stage, the children may develop poor learning habits, such as short duration of learning, excessive movement, inattention or incorrect sitting posture.

At present, for the development of good learning habits of primary and secondary school students, the education mainly depends on the language of parents or teachers, for example, the parents or teachers need to supervise and manage the learning process of the primary and secondary school students, and the process consumes time and human resources.

Disclosure of Invention

The embodiment of the disclosure provides at least one prompting scheme of behavior states.

In a first aspect, an embodiment of the present disclosure provides a method for prompting a behavior state, including:

acquiring a video stream by a video acquisition component, wherein the video stream comprises a target management object in a first state;

detecting a behavior state of the target management object based on the video stream;

and generating management prompt information aiming at the target management object according to the detected behavior state of the target management object and the time planning information.

In the embodiment of the disclosure, a video stream for a target management object may be collected, so that a behavior state of the target management object may be detected based on the video stream, for example, when the target management object is a target student, taking a scene of prompting for the behavior state of the target student in a learning environment as an example, various behavior states of the target student in a certain time period may be detected, for example, whether there are behaviors such as a good learning state, whether there are hyperactivity and incorrect sitting posture in a learning process, and further, according to the detected behavior state of the target management object and time planning information, management prompting information for the target management object may be generated, so that time management may be performed on the target management object based on the management prompting information in a later period, and the target management object may be assisted to develop a good behavior habit.

In one possible embodiment, the management prompt includes at least one of:

prompting the target management object to prolong or shorten the time length in the first state;

and prompting the target management object to switch from the first state to a second state.

In the embodiment of the present disclosure, a plurality of kinds of management prompt information for the target management object may be generated, for example, taking a scene of prompting for behavior states of the target students in a learning environment as an example, if the first state represents a learning state and the second state represents a rest state, the target students may be prompted to lengthen or shorten the learning time, or the learning state may be shifted to the rest state, and the target management object may be helped to develop a good learning habit by adjusting the learning duration and/or prompting to reasonably switch between the learning state and the rest state.

In a possible implementation manner, after the generating the management prompt information for the target management object, the prompting method further includes:

outputting the management prompt information through a video acquisition component;

and/or sending the management prompt information to a target client corresponding to the target management object, and controlling the target client to output the management prompt information.

In the embodiment of the disclosure, after the management prompt information for the target management object is generated, the management prompt information may be directly output through the video acquisition component, so that the target management object may adjust the behavior state according to the management prompt information, and the management prompt information may also be sent to the target client corresponding to the target management object, so that a user may assist the target management object to adjust the behavior state according to the management prompt information.

In a possible embodiment, the time schedule information includes a first specified duration, and the first specified duration is used to indicate a duration that the target management object is in the first state;

the generating of the management prompt information for the target management object according to the detected behavior state of the target management object and the time planning information includes:

according to the detected behavior state of the target management object and the first specified duration, determining whether the duration of the behavior state of the target management object including the first state reaches the first specified duration or not;

and in response to the behavior state of the target management object comprising the duration of being in the first state reaching the first specified duration, generating first prompt information indicating that the target management object is switched from the first state to a second state.

In the embodiment of the disclosure, when it is detected that the duration of the first state of the target management object reaches the first specified duration, the indication indicating that the target management object enters the state switching may be generated, for example, the first state is a learning state, and the second state is a rest state, and after it is detected that the duration of the continuous learning of the target management object reaches the first specified duration, the target management object may be prompted to rest, so that the target management object may be helped to reasonably arrange the learning duration, and a good learning habit is developed by combining labor and ease.

In a possible implementation manner, after the generating the first prompting information indicating that the target management object is switched from the first state to the second state, the prompting method further includes:

responding to the first prompt information output by the video acquisition component and/or a target client corresponding to the target management object, and detecting whether the target management object is in the second state;

and in response to the target management object not being in the second state, prolonging the first specified time until the duration of the target management object being in the first state reaches the prolonged first specified time, and outputting the first prompt information again through the video acquisition component and/or the target client.

In the embodiment of the disclosure, after the video acquisition component and/or the target client outputs the first prompt information for prompting the target management object to switch from the first state to the second state, if it is detected that the target management object does not enter the second state, the first specified time duration corresponding to the first state may be flexibly extended, and when it is detected that the duration of the target management object in the first state reaches the extended first specified time duration, the target management object is prompted by the first prompt information again to prompt the target management object to enter the second state, and the target management object is helped to develop good behavior habits through time management of different states of the target management object.

In a possible implementation manner, the time-plan information includes a second specified duration, where the second specified duration is used to indicate a duration that the target management object is in the second state, and the prompting method further includes:

in response to extending the first prescribed duration, extending the second prescribed duration based on the extended first prescribed duration.

In the embodiment of the disclosure, it is proposed that, under the condition that the first specified time is prolonged, the second specified time is prolonged based on the prolonged first specified time, taking a scene of prompting for behavior states of target students in a learning environment as an example, if the first specified time corresponding to the first state is the learning time and the second specified time specified in the second state is the rest time, the learning time and the rest time of the target management object under the learning scene can be flexibly adjusted, so as to help the target management object to reasonably learn and rest, and develop a good learning habit.

responding to the second state of the target management object, and determining a response time length of the target management object entering the second state, wherein the response time length is a time length corresponding to a time period taking the output of the first prompt message as a starting point and taking the target management object entering the second state as a stopping point;

updating or not updating the second prescribed duration based on the response duration.

In the embodiment of the disclosure, after the target management object is prompted to enter the second state, the target management object can be flexibly updated according to the response time from the output of the first prompt information to the entry of the target management object into the second state, so as to help the target management object to reasonably distribute learning and rest and form a good learning habit.

In one possible embodiment, when the first state is a learning state and the second state is a rest state, the generating management guidance information for the target management object further includes:

performing behavior detection on the target management object in the first state to obtain a behavior detection result;

and generating second prompt information for correcting the abnormal behavior in response to the condition that the behavior detection result indicates that the target management object has the abnormal behavior.

In the embodiment of the present disclosure, it is proposed that when the target management object is in the learning state, the target management object may be subjected to behavior detection to detect whether the target management object has an abnormal behavior, and when it is determined that the target management object has the abnormal behavior, second prompt information for correcting the abnormal behavior is generated, so as to help the target management object correct the abnormal behavior in the learning state, thereby developing a good learning habit.

In one possible embodiment, the abnormal behavior includes at least one of inattention and sitting posture abnormality, and the prompting method further includes at least one of:

the second prompting information is used for prompting the attention of the target management object when the abnormal behavior comprises the attention of the target management object is not focused;

and when the abnormal behavior comprises that the target management object is abnormal in sitting posture, the second prompt message is used for prompting the target management object to adjust the sitting posture.

In the embodiment of the disclosure, the attention and the sitting posture of the target management object in the learning state can be detected, so that the target management object is helped to develop a learning habit of concentration and good sitting posture in the learning state.

In a possible implementation, the capturing, by the video capturing component, a video stream includes:

collecting the video stream by the video collection component in response to occurrence of a preset mode trigger event;

the preset mode trigger event comprises one or more of the following:

receiving an instruction for controlling the video acquisition component to enter the preset mode;

reaching the preset moment when the preset mode starts;

the target management object is in the first state matched with the preset mode.

In the embodiment of the disclosure, a management stage of entering the learning time of the target management object can be triggered in multiple ways, so that convenience in management of the target management object is improved.

In a possible implementation manner, the prompting method further includes:

generating a behavior state report of the target management object according to the detected behavior state of the target management object and the time planning information;

and receiving a behavior state report acquisition request sent by a target client corresponding to the target management object, and generating the behavior state report to the target client.

In the embodiment of the disclosure, a behavior state report capable of reflecting behavior state conditions in a history stage of a target management object is generated according to a detected behavior state of the target management object and time planning information, and the behavior state report can be displayed to a target client, so that the behavior state conditions of the target management object can be reflected to a user intuitively, and the user can analyze and guide the behavior state of the target management object conveniently.

In a possible implementation manner, the prompting method further includes:

and receiving a video viewing request aiming at the behavior state sent by a target client corresponding to the target management object, and sending a video clip associated with the behavior state to the target client.

In the embodiment of the disclosure, the video stream associated with the behavior state is sent to the target client, so that a user of the target client can conveniently and timely master the behavior state of the target management object, for example, the behavior state of a student in a learning process can be timely mastered, and the student is assisted to develop a good learning habit.

In a second aspect, an embodiment of the present disclosure provides a device for prompting a behavior state, including:

the acquisition module is used for acquiring a video stream through the video acquisition component, wherein the video stream comprises a target management object in a first state;

the detection module is used for detecting the behavior state of the target management object based on the video stream;

and the generating module is used for generating management prompt information aiming at the target management object according to the detected behavior state of the target management object and the time planning information.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the hinting method according to the first aspect.

In a fourth aspect, the present disclosure provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the prompting method according to the first aspect.

In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.

Fig. 1 is a flowchart illustrating a behavior state prompting method provided by an embodiment of the present disclosure;

FIG. 2 is a flowchart illustrating a first method for generating management prompt information according to an embodiment of the present disclosure;

FIG. 3 is a flowchart illustrating a second method for generating management prompt information according to an embodiment of the disclosure;

FIG. 4 is a flowchart illustrating a third method for generating management prompt information according to an embodiment of the disclosure;

FIG. 5 is a flowchart illustrating a fourth method for generating management prompt information according to an embodiment of the disclosure;

FIG. 6 is a flow chart illustrating another behavior state prompting method provided by embodiments of the present disclosure;

fig. 7a is a schematic illustration showing a learning report provided by an embodiment of the present disclosure;

FIG. 7b is a schematic illustration of another learning report provided by the embodiments of the present disclosure;

fig. 8 is a schematic structural diagram illustrating a behavior state prompting device provided by an embodiment of the present disclosure;

fig. 9 shows a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

The term "and/or" herein merely describes an associative relationship, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

In an independent learning environment, students in middle and primary schools may have poor learning habits, such as short duration of learning, incorrect sitting posture and inattentive attention, and therefore how to help the students in middle and primary schools develop good learning habits is a problem to be solved urgently.

Based on the above research, the present disclosure provides a method for prompting a behavior state, which may collect a video stream for a target management object, so that the behavior state of the target management object may be detected based on the video stream, for example, a scene for prompting a behavior state of a target student in a learning environment is taken as an example, whether the target student has a good learning state in a certain time period, whether there are behaviors such as hyperactivity and incorrect sitting posture in a learning process, and further, management prompt information for the target management object may be generated according to the detected behavior state of the target management object and time planning information, so that time management may be performed on the target management object based on the management prompt information in a later period, and the target management object is assisted to form a good behavior habit.

To facilitate understanding of the present embodiment, first, a method for prompting a behavior state disclosed in the embodiments of the present disclosure is described in detail, where an execution subject of the method for prompting a behavior state provided in the embodiments of the present disclosure is generally a computer device with certain computing capability, and the computer device includes, for example: the terminal device may be a video acquisition component, or a processing device corresponding to the video acquisition component, or the like. In some possible implementations, the behavior state prompting method may be implemented by a processor calling computer readable instructions stored in a memory.

Referring to fig. 1, a flowchart of a method for prompting a behavior state provided in an embodiment of the present disclosure is shown, where the method for prompting a behavior state includes the following steps S101 to S103:

s101, a video stream is collected through a video collecting component, wherein the video stream comprises a target management object in a first state.

Illustratively, the video capturing component may have a function of capturing images and/or videos, including but not limited to a pan-tilt camera, a smart phone with a video recording function, a tablet, a notebook, etc., and is not limited herein.

For example, taking time management in a learning scene for a target management object as an example, the target management object may be a target student who needs time management, and the first state may be a preset learning state in the learning scene.

S102, detecting the behavior state of the target management object based on the video stream.

Illustratively, the behavior state may include a corresponding behavior state of the target student in a learning state, such as a sitting posture standard state, a sitting posture abnormal state, a concentration and a non-concentration, etc.

And S103, generating management prompt information aiming at the target management object according to the detected behavior state of the target management object and the time planning information.

For example, taking the goal management object as the goal student, the time schedule information may include when the goal student enters the learning state in the learning scenario, and a preset learning duration threshold value, such as 45 minutes, for keeping the learning state in the learning scenario; the method can also comprise the steps of when the target students enter a rest state in the learning scene, keeping a preset rest duration threshold value of the rest state in the learning scene, such as 10 minutes, and generating management prompt information aiming at the target students through the detected behavior states of the target students and the time planning information.

Specifically, the management prompt information includes at least one of:

(1) prompting the target management object to delay or shorten the duration of the first state;

(2) and prompting the target management object to switch from the first state to the second state.

For example, the first state may be a learning state in a learning scene, and the second state may be a rest state in the learning scene, and in the process of detecting the behavior state of the target management object in the learning scene, the target management object may be prompted to switch between the learning state and the rest state through the management prompt information, and the target management object may be prompted to delay or shorten the duration of the learning state.

Or, the first state may be a rest state in a learning scene, the second state may be a learning state in the learning scene, and in the process of detecting the behavior state of the target management object in the learning scene, the target management object may be prompted to switch between the rest state and the learning state through the management prompting information, and the target management object is prompted to delay or shorten the time length in the rest state.

For example, the target student is prompted to enter a rest state when the continuous learning time is long, and then the target student returns to the learning state again after the rest time, in addition, whether a bad behavior state exists or not can be detected when the target student is in the learning state, for example, behaviors such as inattention and/or incorrect sitting posture and the like can be detected, and the target management object can be prompted to reasonably switch between the learning state and the rest state through the generated management prompt information for the target management object, so that the target management object can be helped to develop a good learning habit.

In the embodiment of the disclosure, a video stream for a target management object may be collected, so that a behavior state of the target management object may be detected based on the video stream, for example, a scene of prompting a behavior state of a target student in a learning environment is taken as an example, whether the target student has a good learning state in a certain time period, whether there are behaviors such as hyperactivity and incorrect sitting posture in a learning process, and further, a management prompt message for the target management object may be generated according to the detected behavior state of the target management object and time planning information, so that time management may be performed on the target management object based on the management prompt message at a later stage, and a good behavior habit of the target management object may be assisted.

The above-mentioned S101 to S103 will be explained with reference to specific embodiments.

For the above S101, capturing a video stream by a video capturing component includes:

in response to sending the preset mode trigger event, a video stream is captured by a video capture component.

Illustratively, the preset mode trigger event includes one or more of:

(1) receiving an instruction for controlling a video acquisition component to enter a preset mode;

(2) reaching the preset moment when the preset mode starts;

(3) the target management object is in a first state matching a preset pattern.

For example, before controlling the video capturing component to capture a video stream including a target management object in a first state, the video capturing component may be in a video stream capturing state in a closed or normal mode, which is equivalent to the role of a security camera, and after detecting a preset mode trigger event, the video capturing component needs to perform video stream capturing on the target management object.

For the case (1), for example, the instruction for controlling the video capturing component to enter the preset mode may be sent by a target client corresponding to the target management object, and is used to instruct the video capturing component to start video stream capturing on the target management object, where the target client corresponding to the target management object may be a smartphone, a tablet, a notebook, and the like of a parent of the target student, and is not specifically limited herein.

For the case (2), for example, the time when the preset mode starts may be preset, for example, 3: 00-5: 00 pm every day is set as the time period corresponding to the preset mode, and when it is detected that the current time reaches 3:00 pm, the preset time when the preset mode starts may be determined.

For the case of the (3), the preset mode may be, for example, an operating mode for performing behavior state detection on the target management object in the learning scene, so that after the target management object is in the first state matched with the preset mode, switching to the preset mode may also be triggered, for example, it may be detected that the target management object sits at the front end of a desk based on a video stream in a normal mode, and after the target management object is kept for a certain period of time, the target management object may also be triggered to enter the preset mode.

In a real-time manner, after generating management prompt information for a target management object, the prompt method provided by the embodiment of the present disclosure further includes:

the management prompt information is output through a video acquisition component;

and/or sending management prompt information to a target client corresponding to the target management object, and controlling the target client to output the management prompt information.

For example, the video capture component may output the management prompt information through one or more of voice, message prompt, light effect, and vibration, such as the generated management prompt information for prompting the target management object to enter a rest state, and the target management object may be prompted through voice: "your learning time has reached 40 minutes, please rest"; the preset ring for indicating the target management object to enter the rest state can be played in a message prompting mode; the light connected with the video acquisition component can be controlled to flash according to the set frequency in a light effect prompting mode, and the vibration can be sent out in a vibrating prompting mode to prompt a target management object to enter a rest state.

For example, the generated management prompt information may also be sent to a target client of the target management object, and the target client may be controlled to output the management prompt information, for example, the target client may be controlled to output the management prompt information through one or more of voice playing, text message prompting, light effect, and vibration, so as to request a parent to manage the behavior state of the target student in the learning scene.

Illustratively, the management prompt information can be output through the video acquisition component, and the management prompt information can be sent to the target client corresponding to the target management.

For the above S102, the behavior state of the target management object may be detected based on the video stream in the following manner:

the method comprises the steps of performing framing processing on a video stream, performing pose detection on the video frame to obtain pose data of a target management object in the video frame, and determining the behavior state of the target management object according to the pose data of the video frame in the video stream.

Illustratively, the pose data may include the position and pose of the target management object in the video frame, so that the behavior of the target management object in a single frame of the video frame may be marked, such as leaning forward, leaning backward, leaving the desk, lying prone, head-to-side shifting (to the left side, to the right side compared to the central area of the video frame), passing-over head (the face of the target management object is not detected in the video frame captured by the video capturing part), normal pose (pose where the above does not occur), and the like.

Further, based on the detected posture data of the target management object in the video frame, the behavior state of the target management object can be further determined, for example, the target management object can be determined to be in a learning state when the target management object is continuously detected to be located in front of the desk, and the posture data of the target management object in the video frame is continuously detected to be forward leaning in the learning state, so that the target management object can be determined to have sitting posture abnormality in the learning state; for another example, if it is continuously detected that the head included in the pose data in the video frames is shifted in different directions, it can be determined that the target management object includes inattention (hyperactivity) during the learning process.

For example, the mode of performing pose detection on the video frame to obtain the pose data of the target management object in the video frame may be determined by a pre-trained neural network, for example, a plurality of feature points included in the video frame may be extracted to obtain position information corresponding to the plurality of feature points in the video frame, and then the position information corresponding to the plurality of feature points is input to the neural network for determining the pose data to obtain the pose data of the target management object included in the video frame.

For example, the posture data of the target management object can be determined according to the position information of the feature points of different parts of the target management object in the video frame, for example, the steering angle of the face of the target management object can be determined according to the position information of the feature points of the face in the video frame and the preset position information of the feature points of the face in the video frame under various steering angles; and determining whether the sitting posture of the target management object is an upright sitting posture, a forward leaning sitting posture or a backward leaning sitting posture according to the position information of the characteristic points of the trunk contour of the target management object in the video frame and the position information of the characteristic points of the trunk contour of the human body in the upright sitting posture, the forward leaning sitting posture and the backward leaning sitting posture, which are acquired in advance.

For example, in different application scenarios, when feature point extraction is performed on a video frame, in addition to performing feature point right extraction on a target management object, feature point extraction may be performed on a surrounding environment, for example, in a process of detecting a behavior state of a target student in a learning scene, position information of a feature point of a desk in the video frame may be extracted, according to a plurality of feature points of the target student and position information of a plurality of feature points of the desk in the video frame, it may be assisted to determine whether the target student leans prone on the desk, for example, it may be detected that a distance between a facial feature point of the target student and a feature point of the desk is smaller than a preset distance threshold, and the target student in the video frame leans prone on the desk may be marked.

In another embodiment, when detecting the posture data of the target management object in the video frame, the target detection may be performed on the video frame, the position information of the detection frame of the target management object in the video frame is determined, and the position information of the vertex of the detection frame in the video frame, which is when the human body is in the upright sitting posture, the forward leaning sitting posture or the backward leaning sitting posture, is collected in advance through the position information of the vertex of the detection frame in the video frame, so as to determine the posture data of the target management object.

For example, it may also be determined whether the head of the target management object is shifted to the left side, or shifted to the right side, or the like, according to the relative position information of the head detection frame and the central region of the video frame.

For example, the posture data of the target management objects detected by a single frame of video frames includes forward inclination of the target management objects, but may be an accident, and it cannot be accurately described that the target management objects have a behavior state with incorrect sitting posture, so in order to improve the accuracy of determining the behavior state of the target management objects, it is proposed that the behavior states of the target management objects can be determined together according to the posture data of the target management objects detected by multiple frames of video frames.

In one embodiment, the time plan information includes a first specified duration, the first specified duration being used to indicate a duration that the target management object is in the first state; in the step S103, when the management guidance information for the target management object is generated based on the detected behavior state of the target management object and the time plan information, as shown in fig. 2, the following steps S201 to S202 may be included:

s201, according to the detected behavior state of the target management object and the first planned time length, whether the duration of the behavior state of the target management object including the first state reaches a first specified time length is determined.

For example, when the target management object is a target student, the first planned time period may be set by a guardian of the target management object, such as a parent of the target student.

For example, taking the first state as the learning state, the behavior state included when the target student is in the learning state may be preset, for example, the behavior state including the target student sitting on a seat in front of a desk with a hand on the desk may be set; or may set a behavior state including learning materials in which the subject student sits on a seat in front of the desk with hands on the desk and looks toward the desk, which is not particularly limited herein.

S202, in response to the behavior state of the target management object including that the duration of the target management object in the first state reaches a first specified duration, generating first prompt information indicating that the target management object is switched from the first state to the second state.

For example, in a case where the first state is a learning state and the second state is a rest state, in a case where it is detected that the duration of the learning state of the target management object reaches a first prescribed duration, first prompt information for instructing the target management object to shift to the rest state may be generated.

In one embodiment, after generating the first prompting information indicating that the target management object is switched from the first state to the second state, as shown in fig. 3, the prompting method provided by the embodiment of the present disclosure further includes the following steps S301 to S302:

s301, responding to the first prompt information output by the video acquisition component and/or the target client corresponding to the target management object, and detecting whether the target management object is in a second state;

s302, responding to the fact that the target management object is not in the second state, prolonging the first specified time length until the duration time that the target management object is in the first state reaches the prolonged first specified time length, and outputting the first prompt information through the video acquisition component and/or the target client again.

For example, the manner of outputting the first prompt information by the video acquisition component refers to the manner of outputting the management prompt information by the video acquisition component mentioned above, and the manner of outputting the first prompt information by the target client corresponding to the target management object refers to the manner of outputting the management prompt information by the target client mentioned above, which is not described herein again.

For example, after the first prompt information is output through the video capturing part and/or the target client, whether the target management object is in the second state may be detected according to the video stream, and for example, the second state is a rest state, a behavior state included when the target student is in the rest state may be preset, for example, a seat before the target student leaves a desk may be set; or the head of the user lies on the desk, which is not limited herein.

If it is detected that the target management object does not enter the rest state yet, considering that the target management object may continue to maintain the learning state, the first predetermined time duration corresponding to the first state (learning state) may be extended, for example, the initially set first predetermined time duration may be extended from 40 minutes to 50 minutes, then the behavior state of the target management object is continuously detected until the duration that the target management object is in the learning state is detected to reach the extended first predetermined time duration, and the first prompt information for prompting the target management object to switch to the second state is output again through the video acquisition component and/or the target client.

In an embodiment, the time schedule information includes a second specified duration, where the second specified duration is used to indicate a duration that the target management object is in the second state, and the prompting method provided in the embodiment of the present disclosure further includes:

in response to extending the first prescribed duration, the second prescribed duration is extended based on the extended first prescribed duration.

For example, the first state is a learning state, the first predetermined time period is used to indicate the time period for which the target management object is in the learning state, and the second state is a rest state, and the second predetermined time period is used to indicate the time period for which the target management object is in the rest state.

When the second predetermined time length is extended based on the extended first predetermined time length, the following modes may be included:

(1) adjusting the rest duration based on the extended total learning duration;

(2) adjusting a rest duration based on the extended learning duration;

for the case of the (1) and when the rest duration is adjusted based on the extended total learning duration, the ratio of the total learning duration to the rest duration before and after the extension may be the same, for example, the initial total learning duration is 40 minutes, the initial rest duration is 10 minutes, and if the extended total learning duration becomes 60 minutes, the rest duration is adjusted to 15 minutes.

For the case (2), for example, the initial total learning duration is 40 minutes, and the initial rest duration is 10 minutes, if the extended total learning duration becomes 60 minutes, that is, the extended learning duration is 20 minutes, the rest duration may be adjusted according to the 20 minutes, and when the rest duration is adjusted, the rest duration may be increased by a multiple of 20 minutes, for example, by 0.5 times of 20 minutes, and the rest duration may be adjusted to 20 minutes.

In an implementation manner, the time schedule information includes a second specified duration, and the second specified duration is used to indicate a duration that the target management object is in the second state, as shown in fig. 4, the prompting method provided in the embodiment of the present disclosure further includes the following steps S401 to S402:

s401, responding to the target management object in the second state, determining the response time length of the target management object entering the second state, wherein the response time length is the time length corresponding to the time period taking the first prompt information as the starting point and taking the target management object entering the second state as the stopping point;

s402, updating or not updating the second specified time length based on the response time length.

For example, taking the second state as the rest state and the second specified duration as the duration for indicating that the target management object is in the rest state as an example, after the first prompt information for prompting that the target management object enters the rest state is output and after the target management object is determined to be in the rest state, a duration that passes from the output of the first prompt information to the determination that the target management object enters the rest state may be determined, if the response duration is less than the set response duration, the rest duration may not be updated, and if the response duration is greater than or equal to the set response duration, the second specified duration may be updated based on the response duration.

In one embodiment, when the first state is the learning state and the second state is the rest state, and when the management guidance information for the target management object is generated, as shown in fig. 5, the method further includes the following steps S501 to S502:

s501, performing behavior detection on the target management object in the first state to obtain a behavior detection result;

and S502, generating second prompt information for correcting the abnormal behavior in response to the condition that the behavior detection result indicates that the target management object has the abnormal behavior.

For example, the process of performing behavior detection on the target management object in the first state to obtain the behavior detection result may refer to the above-described detection process of the behavior state, and the behavior states corresponding to the plurality of abnormal behaviors may be preset, so that in a case where it is detected that the target management object has a behavior state corresponding to at least one abnormal behavior in the learning state, the second prompt information for correcting the abnormal behavior may be generated.

Further, after the second prompt information is generated, the second prompt information may be output through the video acquisition component and/or the target client corresponding to the target management object, and the manner of outputting the second prompt information is similar to that of outputting the first prompt information, which is not described herein again.

In one embodiment, the abnormal behavior includes at least one of inattention and sitting posture abnormality, and the prompting method further includes at least one of:

(1) the second prompt message is used for prompting the attention of the target management object when the abnormal behavior comprises that the target management object is not focused;

(2) in the case where the abnormal behavior includes an abnormality in sitting posture of the target management object, the second prompt information is used to prompt the target management object to adjust the sitting posture.

Illustratively, the inattention may include hyperactivity, shaking of the head, etc., and the specific detection manner is detailed above in the detection process for the behavioral state described above.

Illustratively, the sitting posture abnormality may include forward inclination, backward inclination, humpback, etc., and the specific detection manner is detailed in the detection process for the behavior state described above.

In an implementation manner, as shown in fig. 6, the prompting method provided by the embodiment of the present disclosure further includes the following steps S601 to S602:

s601, generating a behavior state report of the target management object according to the detected behavior state of the target management object and the time planning information;

and S602, receiving a behavior state report acquisition request sent by a target client corresponding to the target management object, and sending a behavior state report to the target client.

For example, by taking detection of the behavior state of the target student in the learning scene as an example, after the behavior state of the target student included in the video stream is obtained, a behavior state report for reflecting the behavior state of the target student in the learning scene may be generated according to the behavior state of the target student and the time planning information, and after a behavior state report request sent by a target client corresponding to the target student is received, the behavior state report is sent to the target client for display.

Taking the behavioral state report as a learning report, as shown in fig. 7a, which is a schematic diagram of a learning report that is displayed at a target client and indicates whether a target student is too comfortable during learning, the learning report may include a learning information display area 21, in which information such as a student name, generation time of the learning report, learning duration, and learning location corresponding to the learning report may be displayed; the learning report can learn a time feedback region 22, in which the ratio of the learning duration and the rest duration of the student in the learning process can be shown by a ratio graph; in addition, the learning time feedback area may further include a trigger button for triggering acquisition of a video segment in a learning state or a rest state, and the target client user may acquire the corresponding video segment through the trigger button.

Still taking the behavioral state report as an example of a learning report, as shown in fig. 7b, the learning report may include a learning information display area 31 in which information such as a student name, a generation time of the learning report, a learning duration, and a learning location corresponding to the learning report may be displayed.

The learning report can also comprise a learning posture analysis result display area 32, and the proportion of different postures of the student in the learning process in the total learning time length can be displayed through a proportion graph in the learning posture analysis result display area; in addition, the learning gesture analysis result display area may further include a trigger button for triggering acquisition of a video segment with an abnormal gesture, and through the trigger button, the target client user may acquire the video segment corresponding to the abnormal gesture.

A learning state result display area 33 can be further included in the learning report, and the proportion of different learning states (such as an attentive state and an inattentive state) existing in the learning process of the student in the total learning time length can be displayed through a proportion graph in the learning state result display area; in addition, the learning state result display area may further include a trigger button for triggering acquisition of a video segment in the inattentive state, and the target client user may acquire the video segment corresponding to the inattentive state through the trigger button.

In addition, the prompting method provided by the embodiment of the disclosure further includes:

and receiving a video viewing request aiming at the behavior state sent by a target client corresponding to the target management object, and sending the video clip associated with the behavior state to the target client.

For example, the behavior state may include a behavior state with problems such as sitting posture abnormality, attention abnormality, and the like, and after the behavior state report is sent to the target client for presentation, the target user corresponding to the target client may further trigger to view a video clip corresponding to the behavior state, for example, click and view a related problem video in the target client, and further select a video stream associated with the behavior state requested to be viewed.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Based on the same technical concept, a behavior state prompting device corresponding to the behavior state prompting method is further provided in the embodiment of the disclosure, and as the principle of solving the problem of the device in the embodiment of the disclosure is similar to the prompting method in the embodiment of the disclosure, the implementation of the device can refer to the implementation of the method, and repeated parts are not described again.

Referring to fig. 8, a schematic diagram of a behavior state prompting device 700 provided in an embodiment of the present disclosure is shown, where the behavior state prompting device includes:

a collecting module 701, configured to collect, by a video collecting component, a video stream, where the video stream includes a target management object in a first state;

a detection module 702, configured to detect a behavior state of a target management object based on a video stream;

the generating module 703 is configured to generate management prompt information for the target management object according to the detected behavior state of the target management object and the time planning information.

In one possible embodiment, the management hint information includes at least one of:

and prompting the target management object to switch from the first state to the second state.

In a possible implementation manner, the prompting apparatus further includes a sending module 704, and after the generating module 703 generates the management prompting information for the target management object, the sending module 704 is configured to:

In one possible embodiment, the time schedule information includes a first specified duration, and the first specified duration is used to indicate a duration that the target management object is in the first state;

the generating module 703, when configured to generate the management prompt information for the target management object according to the detected behavior state of the target management object and the time plan information, includes:

determining whether the duration of the behavior state of the target management object including the state in the first state reaches a first specified duration or not according to the detected behavior state of the target management object and the first specified duration;

and generating first prompt information indicating that the target management object is switched from the first state to the second state in response to the behavior state of the target management object comprising duration of being in the first state reaching a first specified time length.

In a possible implementation manner, after generating the first prompt information indicating that the target management object is switched from the first state to the second state, the generating module 703 is further configured to:

responding to the first prompt information output by a video acquisition component and/or a target client corresponding to the target management object, and detecting whether the target management object is in a second state;

and responding to the fact that the target management object is not in the second state, prolonging the first specified time length until the duration of the target management object in the first state reaches the prolonged first specified time length, and outputting the first prompt information through the video acquisition component and/or the target client again.

In a possible implementation manner, the time schedule information includes a second specified duration, where the second specified duration is used to indicate a duration that the target management object is in the second state, and the generating module 703 is further configured to:

responding to the second state of the target management object, and determining the response time length of the target management object entering the second state, wherein the response time length is the time length corresponding to the time period taking the output of the first prompt message as a starting point and taking the second state of the target management object as a stopping point;

the second prescribed duration is updated or not updated based on the response duration.

In a possible embodiment, in the case that the first state is a learning state and the second state is a rest state, the generating module 703, when being configured to generate the management guidance information for the target management object, includes:

In one possible embodiment, the abnormal behavior includes at least one of inattention and sitting posture abnormality, and in a case where the abnormal behavior includes inattention of the target management object, the second prompting information is for prompting the target management object to concentrate on attention;

in the case where the abnormal behavior includes an abnormality in sitting posture of the target management object, the second prompt information is used to prompt the target management object to adjust the sitting posture.

In one possible implementation, the capture module 701 captures a video stream through a video capture component, including:

collecting a video stream through a video collection component in response to the occurrence of a preset mode trigger event;

the preset mode triggering event comprises one or more of the following:

receiving an instruction for controlling a video acquisition component to enter a preset mode;

reaching the preset moment when the preset mode starts;

the target management object is in a first state matching a preset pattern.

In a possible implementation, the generating module 703 is further configured to:

and receiving a behavior state report acquisition request sent by a target client corresponding to the target management object, and sending a behavior state report to the target client.

In a possible implementation manner, the prompting apparatus further includes a sending module 704, and the sending module 704 is configured to:

The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.

Corresponding to the prompting method of the behavior state in fig. 1, an embodiment of the present disclosure further provides an electronic device 800, as shown in fig. 9, a schematic structural diagram of the electronic device 800 provided in the embodiment of the present disclosure includes:

a processor 81, a memory 82, and a bus 83; the memory 82 is used for storing execution instructions and includes a memory 821 and an external memory 822; the memory 821 herein is also referred to as an internal memory, and is used for temporarily storing operation data in the processor 81 and data exchanged with the external memory 822 such as a hard disk, and the processor 81 exchanges data with the external memory 822 through the memory 821, and when the electronic device 800 operates, the processor 81 communicates with the memory 82 through the bus 83, so that the processor 81 executes the following instructions: acquiring a video stream by a video acquisition component, wherein the video stream comprises a target management object in a first state; detecting a behavior state of the target management object based on the video stream; and generating management prompt information aiming at the target management object according to the detected behavior state of the target management object and the time planning information.

The embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for prompting a behavior state described in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

The embodiment of the present disclosure further provides a computer program product, where the computer program product carries a program code, and an instruction included in the program code may be used to execute the step of the method for prompting a behavior state in the foregoing method embodiment.

The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A method for prompting a behavior state is characterized by comprising the following steps:

2. The prompting method of claim 1, wherein the management prompting information comprises at least one of:

3. The prompting method according to claim 1 or 2, wherein after the generating of the management prompting information for the target management object, the prompting method further comprises:

outputting the management prompt information through the video acquisition component;

4. The prompting method according to any one of claims 1 to 3, wherein the time schedule information includes a first prescribed time duration, and the first prescribed time duration is used to indicate a time duration in which the target management object is in the first state;

and in response to the behavior state of the target management object comprising the duration of the first state reaching the first specified time, generating first prompt information indicating that the target management object is switched from the first state to a second state.

5. The prompting method according to claim 4, wherein after the generating first prompting information indicating that the target management object is switched from the first state to the second state, the prompting method further comprises:

6. The prompting method of claim 5, wherein the time plan information comprises a second specified duration, the second specified duration being used to indicate a duration that the target management object is in the second state, the prompting method further comprising:

7. The prompting method of claim 5, wherein the time plan information comprises a second specified duration, the second specified duration being used to indicate a duration that the target management object is in the second state, the prompting method further comprising:

8. The presentation method according to claim 2, wherein the generating of the management presentation information for the target management object when the first state is a learning state and the second state is a rest state includes:

9. A method as claimed in claim 8, wherein the abnormal behavior comprises at least one of inattention and sitting posture abnormalities, the method further comprising at least one of:

10. A prompting method according to any one of claims 1 to 9, wherein the capturing of the video stream by the video capturing component comprises:

the preset mode trigger event comprises one or more of the following:

reaching the preset moment when the preset mode starts;

11. The prompting method according to any one of claims 1 to 10, further comprising:

and receiving a behavior state report acquisition request sent by a target client corresponding to the target management object, and sending the behavior state report to the target client.

12. The prompting method according to any one of claims 1 to 11, further comprising:

13. A device for prompting a behavior state, comprising:

14. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the hinting method of any one of claims 1 to 12.

15. A computer-readable storage medium, having stored thereon a computer program for performing, when executed by a processor, the steps of the prompting method according to any one of claims 1 to 12.