WO2020015439A1

WO2020015439A1 - Monitoring method and apparatus, monitoring device, and storage medium

Info

Publication number: WO2020015439A1
Application number: PCT/CN2019/085945
Authority: WO
Inventors: 赵海杰; 秦林婵; 黄通兵
Original assignee: 北京七鑫易维信息技术有限公司
Priority date: 2018-07-18
Filing date: 2019-05-08
Publication date: 2020-01-23
Also published as: CN108922085B; CN108922085A

Abstract

A monitoring method and apparatus, a monitoring device, and a storage medium, wherein same can realize the identity authentication of a person to be monitored by matching a facial image of the person to be monitored with a preset image so as to improve the calling accuracy, and can perform targeted calling based on the direction of the gaze of the person to be monitored and a physical condition of the person to be monitored, so as to improve the use experience of the person to be monitored and medical staff. The method comprises: acquiring a facial image of a person in a monitoring area (S101); if the facial image matches a preset image, determining that the person corresponding to the facial image is a person to be monitored (S102); determining the gaze of the person to be monitored according to an image of the eye in the facial image of the person to be monitored (S103); and if the duration for which the person to be monitored gazes at any subarea in a preset area exceeds a warning duration, determining a calling level according to the gazed-at subarea or according to the gazed-at subarea and the physical condition of the the person to be monitored and correspondingly performing calling (S104).

Description

Monitoring method, device, monitoring equipment and storage medium

Technical field

Embodiments of the present invention relate to the field of communications technologies, and in particular, to a monitoring method, device, monitoring device, and storage medium.

Background technique

At present, with the acceleration of people's pace of life, mental and physical energy consumption increases, and the incidence of various diseases is getting higher and higher. When patients who have lost their language ability and mobility after surgery or other reasons are hospitalized or rehabilitated, they often need to be guarded by medical staff throughout the process so that they can be rescued in time when the patient is unwell.

In the related art, the monitoring method adopted for patients with inconvenient vocalizations and restricted behavioral capabilities. In the process of monitoring a user, an alarm is usually performed by judging the length of time that a person to be monitored gazed at a preset area to notify medical personnel. This monitoring method lacks pertinence in the process of monitoring, and cannot effectively call the medical staff, thereby increasing the workload of the medical staff.

Summary of the invention

At least part of the embodiments of the present invention provide a monitoring method, device, monitoring device, and storage medium to at least partially solve the lack of pertinence of related technologies in the process of monitoring, unable to effectively call medical staff, thereby increasing medical staff The technical issue of the workload.

In one embodiment of the present invention, a monitoring method is provided, including:

Obtain facial images of people in the monitoring area;

If the face image matches a preset image, determining that the person corresponding to the face image is a person to be monitored;

Determining the line of sight of the person to be monitored at least according to an eye image in a face image of the person to be monitored;

If the length of time that the person to be monitored looks at any of the sub-regions in the preset area exceeds the warning period, the call is determined according to the sub-region of the gaze, or the sub-region of the gaze and the physical state of the person Level and make corresponding calls, where the preset area includes multiple sub-areas, and different sub-areas correspond to different call levels.

In one embodiment of the present invention, a monitoring device is further provided, including:

An acquisition module configured to acquire a facial image of a person in a monitoring area;

A person determination module configured to determine that a person corresponding to the face image is a person to be monitored when the face image matches a preset image;

A fixation point determining module, configured to determine a line of sight of the person to be monitored based on at least an eye image in a face image of the person to be monitored;

A calling module configured to, if the length of time that the line of sight of the person to be monitored looks at any sub-region in the preset area exceeds the warning period, according to the sub-region of the fixation, or according to the sub-region of the fixation and the person to be monitored Physical state, determining a call level and making a corresponding call, wherein the preset area includes multiple sub-areas, and different sub-areas correspond to different call levels.

One or more processors;

A storage device configured to store one or more programs;

The one or more programs are executed by the one or more processors, so that the one or more processors implement the monitoring method described above.

In one embodiment of the present invention, a computer-readable storage medium is also provided, storing one or more programs, and the one or more programs are executed by a processor to implement the monitoring method.

Through the above technical solution, before the line-of-sight analysis of the person to be monitored, the face image of the person in the monitoring area can be identified, the call caused by the person who is not to be monitored is avoided, and the accuracy of the call is improved. In addition, after the person to be monitored looks at any of the sub-regions included in the preset area and the gaze duration exceeds the warning period, the target is based on the sub-region of the gaze, or the sub-region of the person to be guarded and the physical status of the person to be monitored Sexual calling reduces the workload of the medical staff and the pain of the medical staff, and effectively improves the experience of the medical staff and the medical staff.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart of a monitoring method according to an embodiment of the present invention.

FIG. 2a is a flowchart of a monitoring method according to an exemplary embodiment of the present invention.

FIG. 2b is a schematic diagram of an application scenario of a monitoring method according to an exemplary embodiment of the present invention.

FIG. 2c is a schematic diagram of a preset image according to an exemplary embodiment of the present invention.

FIG. 3 is a structural block diagram of a monitoring device according to an embodiment of the present invention.

FIG. 4 is a structural block diagram of a monitoring device according to an embodiment of the present invention.

detailed description

The present invention will be described in detail in an optional embodiment of the present invention with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. It should also be noted that, for the convenience of description, only some parts related to the present invention are shown in the drawings instead of all the structures.

Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although flowcharts describe operations (or steps) as sequential processing, many of these operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of operations can be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the drawings. The processing may correspond to methods, functions, procedures, subroutines, subroutines, and so on.

Example one

FIG. 1 is a flowchart of a monitoring method according to an embodiment of the present invention. The method may be applicable to a scenario in which a person to be monitored is monitored, such as monitoring a person who has a language barrier and a mobility disability, thereby When a guardian makes a call when there is a call request. This method may be performed by a monitoring device provided by one embodiment of the present invention. The device may be implemented by at least one of software and hardware and integrated on a monitoring device. In this embodiment, the monitoring equipment may be integrated into existing equipment, such as a computer, a television, or a medical instrument; it may also be a monitoring equipment that implements the monitoring method in this embodiment. The monitoring equipment includes, but is not limited to, cameras, speakers, and processors.

As shown in FIG. 1, the monitoring method provided by this embodiment includes the following steps:

In step S101, a facial image of a person in a monitoring area is acquired.

In this embodiment, the monitoring area can be understood as a collectible area of the image acquisition device. In this embodiment, only face images of persons in the monitoring area can be processed, so that the time of image processing can be effectively reduced, and the efficiency of subsequent recognition can be improved.

It should be noted that the monitoring method in this embodiment may be targeted at those who are to be guarded who have language barriers and mobility problems. Correspondingly, the position of the person to be monitored can be constant or changed relative to the monitoring equipment. When the user's position changes relative to the monitoring device, the monitoring area may include the maximum range that the person to be monitored can move.

In an optional embodiment of the present invention, an image acquisition device (such as a camera) may be used to obtain a face image of a person in the monitoring area. In this embodiment, after obtaining a face image, monitoring analysis can be performed based on the face image.

In an optional embodiment of the present invention, an image of a person's face in the monitoring area can be obtained directly through an image acquisition device, or a full-body image of the person in the monitoring area containing environmental information can be obtained through the image acquisition device, and then the face can be extracted. Department image. It can be understood that the facial image may include background environment information (such as the surrounding environment of the person to be monitored).

In step S102, if the facial image matches a preset image, it is determined that a person corresponding to the facial image is a person to be monitored.

In this embodiment, the preset image may be understood as a face image of a person to be monitored that is entered in advance. The preset image can be used to authenticate the person to be guarded.

After obtaining a face image of a person in the monitoring area, the obtained face image can be matched with a preset image to complete the identity authentication of the person in the monitoring area, thereby determining whether the person in the monitoring area is a person to be monitored, To prevent the supervision of non-guardians. In this embodiment, it can be considered that when the face image is consistent with the preset image, the obtained face image matches the preset image.

For example, each feature information in a face image is consistent with corresponding feature information in a preset image, main feature information in a face image is consistent with corresponding feature information in a preset image, or feature information in a face image is consistent with that in a preset image A certain threshold was reached. The feature information can be understood as the features that identify the facial image, such as eye features, mouth features, or face shape features. The main feature information can be understood as features that can uniquely identify a face image, such as eye features.

Specifically, in an optional embodiment of the present invention, the feature information in the facial image of the person in the monitoring area and the feature information in the preset image may be matched to determine whether the facial image matches the Set the image to match. If they match, the person corresponding to the face image can be determined as the person to be monitored, so that the person to be monitored can be monitored, wherein the feature information can include eye characteristics such as eye spots, filaments, crowns, and / or crypts And other characteristics.

Take identity recognition based on eye features as an example: when at least one of the following conditions is met: the number and size of spots in the eye image (the eye image is included in the face image) and the preset image are similar, When the shape and size of the crowns in the eye image are similar to those in the preset image, and when the size and shape of the crypts in the eye image are similar to those in the preset image, the eye image and the Set the image to match. It can be understood that when determining whether the face image matches the preset image, a preset deviation range is allowed.

In step S103, a line of sight of the person to be monitored is determined based on at least an eye image in a face image of the person to be monitored.

After the person to be monitored is determined, the line of sight corresponding to the eye image in the face image of the person to be monitored can be determined to determine whether the person to be monitored has a call request. It should be noted that, in an optional embodiment of the present invention, the face image may include eyes of the user, so as to determine the sight line of the person to be monitored based on the eye image in the eye image.

In an optional embodiment of the present invention, the position of the eye image in the face image may be located according to the face image of the person to be monitored, and then the sight line may be determined according to the eye image. When determining the sight line based on the eye image, the eye movement characteristics in the eye image may be analyzed to obtain the sight line of the person to be monitored.

In an optional embodiment of the present invention, when an ordinary camera is used to obtain the facial image of the person to be monitored, the eye characteristics of the eye image in the facial image of the person to be monitored and the facial characteristics of the person to be monitored when facing the preset area may be based on Eye characteristics determine the line of sight of the person to be monitored; when using an infrared camera (configured with infrared lights, set to form spot information in the eye of the person to be monitored) to obtain an image of the person's face, you can compare the center of the spot with the center of the pupil The positional relationship determines the sight of the person to be monitored.

Eye movement characteristics can be understood as the characteristic information of the eyes. The eye movement feature may be determined by at least one of physiological information of the eyeball and eyeball information in the eye diagram. Eye characteristics may include, but are not limited to, pupil position, pupil shape, iris position, iris shape, eyelid position, eye corner position, and / or spot (also known as Pulcian spot) position.

When the position of the person to be monitored relative to the monitoring device is fixed, the eye features in the eye image can be analyzed. When the position of the person to be monitored changes relative to the monitoring equipment, the position information of the person to be monitored can also be analyzed to accurately determine the line of sight of the person to be monitored. The position information of the person to be monitored can be obtained by analyzing the facial image. In an optional embodiment of the present invention, the position of the image acquisition device is fixed, and when the person to be monitored is at different positions, there may be a certain difference in the face image acquired by the image acquisition device.

In step S104, if the length of time that the line of sight of the person to be monitored gazed at any sub-region in the preset area exceeds the warning period, according to the sub-region of the fixation, or according to the sub-region of the fixation and the Physical state, determining a call level and making a corresponding call, wherein the preset area includes multiple sub-areas, and different sub-areas correspond to different call levels.

In this embodiment, the preset area can be understood as a designated area on the monitoring device. The sub-region can be understood as a region included in the preset region. The warning duration can be understood as the time threshold of the person to be monitored watching the preset area. The call level can be understood as the priority set according to the physical status of the person to be monitored and the location information of the sub-area. The physical state may be, for example, a cold state of a person to be monitored, a state after surgery, a state of fracture rehabilitation, or the like.

It can be understood that the preset area may include multiple sub-areas, and different sub-areas may correspond to different call levels. Here, the specific position and shape of the preset area are not limited, and the number and shape of the sub-areas included in the preset area are not limited. Those skilled in the art can set it according to actual needs.

In an optional embodiment of the present invention, when the person to be monitored has a call request, he can watch the preset area to realize the alarm. The monitoring device can be understood as a device that runs the monitoring method provided in this embodiment. If the length of time that the person to be monitored looks at any of the sub-areas in the preset area is greater than the warning period, follow-up monitoring processing can be performed; if it is not greater than, the eye image of the person in the monitoring area can continue to be acquired until the person to be monitored looks at the preset The duration of any sub-area in the area exceeds the warning duration.

Specifically, by matching the gaze area corresponding to the line of sight of the person to be monitored with each sub-area in the preset area, it is determined whether the person to be monitored looks at any sub-area in the preset area. If the person to be guarded pays attention to any sub-region in the preset area, it can be further determined whether the length of time that the person to be guarded pays attention to the sub-region exceeds the warning period.

If the to-be-supervised person watches any of the sub-areas in the preset area for more than the warning period, it can be considered that the to-be-supervised person has a call request, so that the call level can be determined based on the sub-area to be-supervised and the physical status of the to-be-monitor And make a corresponding call based on the determined call level. If the length of time that the person to be monitored looks at any of the sub-areas in the preset area does not exceed the warning period or the person to be monitored does not look at any of the sub-areas in the preset area, you can continue to obtain facial images of the person in the monitored area for monitoring Guardianship of personnel.

After determining the sub-area to be monitored by the person to be monitored, a corresponding call level is determined based on the sub-area to be monitored by the person to be monitored and the physical state of the person to be monitored. The physical status of the person to be monitored may be determined based on the medical record information of the person to be monitored, or may be obtained through a monitoring device for monitoring the physical state. In this embodiment, when the call level is determined, the physical status of the person to be monitored is combined to ensure that each call is a valid call to prevent the person to be monitored from making a malicious call.

In an optional embodiment of the present invention, after a call level is determined according to a sub-area watched by a person to be monitored and a physical state of the person to be monitored, a corresponding call may be further performed based on the determined call level. It can be understood that different call levels can correspond to different call methods, such as ringing or lighting; it can also correspond to different medical staff, such as if the call level is higher, call the doctor, if the call level is lower, call the nurse .

In an optional embodiment of the present invention, before the line of sight analysis of the person to be monitored, the face image of the person in the monitoring area can be identified to avoid calls caused by the person who is not to be monitored, and the accuracy of the call is improved. Sex. In addition, after the person to be monitored looks at any subregion contained in the preset area and the gaze duration exceeds the warning period, a targeted call is made according to the subregion to be monitored and the physical status of the person to be monitored, reducing medical care. The workload of the personnel also slows down the illness of the person to be monitored and effectively improves the experience of the person to be monitored and the medical staff.

Example two

FIG. 2a is a flowchart of a monitoring method according to an exemplary embodiment of the present invention. As shown in FIG. 2a, if the face image matches a preset image, it is determined that the person corresponding to the face image is The person to be monitored includes: extracting feature information from the face image; if the feature information matches the preset feature information in the preset image, determining that the person corresponding to the face image is the to-be-monitored personnel.

In an optional embodiment of the present invention, determining the sight line of the person to be monitored at least according to an eye image in the face image of the person to be monitored includes: according to the face image and the reference image of the person to be monitored , Determining position information of the person to be monitored; and determining the line of sight of the person to be monitored according to the position information and eye movement characteristics in the eye image.

In an optional embodiment of the present invention, determining a call level and making a corresponding call according to the sub-area of fixation and the physical state of the person to be monitored includes: searching a preset level data table, determining the sub-area and fixation, and the subject The call level corresponding to the physical status of the monitor and make the corresponding call. Of course, the preset level data table can also be simplified to include the mapping relationship between the sub-area of the fixation and the call level, so as to find the call level corresponding to the sub-area of the fixation by looking up the preset level data table and perform the The corresponding call.

In an optional embodiment of the present invention, before the obtaining a face image of a person in the monitoring area, the method further includes: entering a face image containing environmental information when the person to be monitored faces the reference point as a reference image.

In an optional embodiment of the present invention, if the length of time that the line of sight of the person to be monitored looks at any sub-region in the preset area exceeds the warning period, the determination is made according to the sub-region of the gaze and the physical state of the person to be monitored After calling the level and making a corresponding call, the method further includes: sending a voice reminder corresponding to the call level to a medical staff who cares for the person to be monitored.

As shown in FIG. 2a, the monitoring method includes the following steps:

In step S201, a face image containing environmental information when a person to be monitored faces the reference point is input as a reference image.

In this embodiment, the reference point can be understood as a point displayed on the monitoring device for the attention of a person to be monitored. The reference image can be understood as an image obtained when the person to be monitored looks at the reference point, and the position information of the person to be monitored can be determined based on the reference image.

In an optional embodiment of the present invention, before the person to be monitored is monitored, a face image when the person to be monitored faces the reference point may be pre-registered as the reference image. It should be noted that the facial image may include the background environment of the face of the person to be monitored. By using a face image containing environmental information as a reference image, the user's position information can be determined more effectively.

In step S202, a facial image of a person in the monitoring area is acquired.

In step S203, feature information is extracted from the face image.

In this embodiment, after obtaining a face image of a person in a monitoring area, feature information may be extracted from the face image by using a feature extraction technology to implement identity identification of a person to be monitored based on the extracted feature information. The feature information may include eye features, mouth features, or facial features.

In step S204, it is determined whether the feature information matches the preset feature information. If so, step S205 is performed; if not, step S202 is performed.

In this embodiment, the preset feature information can be understood as the feature data in the preset image.

After the feature information is extracted, it can be further judged whether the feature information matches the preset feature information. If they match, the monitoring operation may be further performed, that is, step S205 is performed; if they do not match, the facial image of the person in the monitoring area may continue to be obtained, that is, step S202 is performed.

Step S205: Determine that the person corresponding to the facial image is a person to be monitored.

In this embodiment, if the feature information matches the preset feature information, a person corresponding to the face image may be determined as a person to be monitored to monitor the person to be monitored.

Step S206: Determine position information of the person to be monitored according to a face image and a reference image of the person to be monitored.

In this embodiment, when determining the line of sight of the person to be monitored based on the eye image in the face image of the person to be monitored, a feature comparison between the face image of the person to be monitored and the reference image may be performed first to determine the current status of the person to be monitored. Position information to determine the sight of the person to be monitored based on the position information.

Specifically, based on the face image of the person to be monitored, the position deviation of the face image compared to the reference image may be calculated to determine the position information of the person to be monitored. When determining the position deviation, the position deviation between the feature information in the face image and the feature information in the reference image may be determined.

In step S207, the sight line of the person to be monitored is determined according to the position information and an eye movement feature in an eye image of the person to be monitored.

In an optional embodiment of the present invention, the eye features (such as the position of the center of the pupil) and the position information in the eye image of the person to be monitored may be analyzed to determine the line of sight of the person to be monitored.

Specifically, when determining the line of sight of the person to be monitored, the line of sight of the person to be monitored may first be determined based on the eye movement characteristics in the eye image, and then the line of sight to be corrected is corrected based on the determined position information to obtain the line of sight to be monitored. The sight of personnel. In addition, the line of sight of the person to be monitored may also be determined directly based on the determined position information, eye movement characteristics in the eye image, and a predetermined line of sight model. The predetermined sight line model establishes a correspondence between position information, eye movement characteristics, and sight lines, and can determine sight lines directly based on the position information and eye movement features.

It can be understood that the image acquisition device may be an ordinary camera or an infrared camera (the infrared camera is provided with an infrared lamp). When an ordinary camera is used to acquire a face image, the line of sight can be determined based on the eye features and the reference pupil center in the face image. The reference pupil center may be obtained by the person to be monitored before using the monitoring device, or may be pre-stored reference data obtained based on the iris characteristics of the person to be monitored.

In step S208, it is determined whether the line of sight of the person to be monitored is looking at any of the sub-areas in the preset area and whether the gaze duration exceeds the warning duration. If yes, step S209 is performed; otherwise, step S202 is performed.

In this embodiment, after the line of sight of the person to be monitored is determined, it can be determined whether the line of sight is looking at any sub-area of the preset area. If the to-be-guarded person's gaze focuses on any sub-area in the preset area, it can be further determined whether the gaze duration exceeds the warning duration, and if so, step S209 can be performed. Correspondingly, if the line of sight of the person to be monitored does not look at any of the sub-areas in the preset area, the process may return to step S202. If the person to be monitored looks at a sub-area in the preset area, but the gaze duration does not exceed the warning duration, the process may return to step S202 to continue monitoring.

In step S209, the preset level data table is searched, a call level corresponding to the sub-area of the gaze and the physical state of the person to be monitored is determined, and a corresponding call is made.

In this embodiment, the preset level data table can be understood as a correspondence between a sub-area, a body state, and a call level in the preset area.

After determining that the line of sight of the person to be monitored looks at any sub-region in the preset area and exceeding the warning period, the preset level data table can be searched according to the sub-region of the person to be monitored and the physical status of the person to be monitored, and the corresponding Call level and make a corresponding call based on the call level.

The call level can be set according to different physical states and different sub-areas, and different call levels can correspond to different docking personnel. After the call level is determined, the corresponding call can be sent to different terminals to complete the call of the corresponding person. The communication method is not limited herein, and those skilled in the art may use a wired method for data transmission or a wireless method for data transmission according to specific requirements.

In step S210, a corresponding voice reminder is sent to the person to be monitored according to the determined call level.

In this embodiment, after a corresponding call is made, a voice reminder corresponding to the call level may be issued to a person to be monitored through a voice playback device (such as a speaker) according to the determined call level. For example, the person to be monitored is notified by voice to the corresponding medical staff, and the call level is used to remind the person to be monitored how to cope with the current physical condition alone, such as taking a deep breath and closing their eyes. It can be understood that the corresponding reminder can be displayed on the display in the form of text.

FIG. 2b is a schematic diagram of an application scenario of a monitoring method according to an embodiment of the present invention. The person in the monitoring area (in the collection area of the image acquisition device 221b) in FIG. 2b is the person to be monitored 21b. As shown in FIG. 2b, the monitoring device 22b acquires the face image of the person 21b to be monitored in real time through the image acquisition device 221b, and then analyzes the acquired face image to obtain a call level. The wireless communication module 222b then sends calls corresponding to different call levels to the corresponding terminals. Finally, a corresponding voice reminder is given to the monitoring person 21b through the speaker 223b. It should be noted that the positional relationship between the devices in this application scenario is only for illustrative purposes, and is not limited to specific locations. In addition, the wireless communication module 222b only shows one communication method, and the specific communication method is not limited herein.

The number of terminals shown in FIG. 2b is not limited, and different terminals can call different personnel. If the call level is low, the monitoring device 22b can send a call request to the first terminal 231b through the wireless communication module 222b, and the first terminal 231b can call the nurse responsible for the person 21b to be monitored; if the call level is high, the monitoring device 22b The call request can be sent to the first terminal 231b and the second terminal 232b through the wireless communication module 222b, where the second terminal 232b can call the doctor in charge of the person 21b to be monitored. If the call level reaches the highest level, the monitoring device 22b may send a call request to the first terminal 231b, the second terminal 232b, and the third terminal 233b through the wireless communication module 222b, and the third terminal 233b may call the family of the person to be monitored 21b.

FIG. 2c is a schematic diagram of a preset image according to an exemplary embodiment of the present invention. FIG. 2c shows one eye. The number of eyes is not limited herein. In this embodiment, two images may be included in the eye image. Eyes only to improve the accuracy of judgment. Here, the preset image is taken as an eye image, and it can be understood that the preset image may be a facial image. As shown in FIG. 2c, the preset image may be an eye image of a person to be monitored, which is pre-registered, and the preset feature information in the eye image may include pupil features and iris features.

Among them, in the eye image, the iris 22c is located in the middle layer of the eyeball (the annular portion between the black pupil and the white sclera); the small circular hole (the innermost circle of the eye) at the center of the iris 22c is the pupil 21c.

In an optional embodiment of the present invention, the specific process of the monitoring method provided by this embodiment is as follows:

First, a face image when a person to be monitored faces the reference point can be input as a reference image, and the face image includes environmental information. Then the monitoring device acquires the face image of the person in the monitoring area in real time through the image acquisition device (such as a camera), extracts feature information from the obtained face image, and compares the extracted feature information with preset feature information. If the matching degree of the feature information and the preset feature information is within a certain range, it is considered that the feature information matches the preset feature information.

If the feature information matches the preset feature information, the person corresponding to the face image may be determined as the person to be monitored, and the position information of the person to be monitored may be obtained by analyzing the face image and the reference image of the person to be monitored. After determining the position information of the person to be monitored, based on the position information and the eye movement characteristics in the eye image of the person to be monitored, the line of sight of the person to be monitored is determined. If the length of time that the person to be monitored looks at any sub-region in the preset area exceeds the warning duration, the call level is determined according to the sub-region of the gaze and the physical status of the person to be monitored and a corresponding call is made.

The monitoring method provided in this embodiment further restricts the operation to be monitored by the person to be monitored, the sight-line determination operation, and the calling operation, and further defines the reference image entry operation and the voice reminder operation. With this embodiment, it is possible to identify persons in a monitoring area based on a preset image, and improve the accuracy of monitoring. In addition, during the determination of the call request of the person to be monitored, the position information of the person to be monitored can be determined based on the reference image and the face image of the person to be monitored, so as to determine based on the position information and the eye movement characteristics of the person to be monitored Outgoing call level, thus making targeted calls, effectively reducing the workload of medical staff. After the call is made, a corresponding voice reminder is sent to the person to be monitored to inform the person to be monitored that the medical staff has been notified so as to reduce the discomfort of the person to be monitored and effectively improve the experience of the person to be monitored and the medical staff.

Example three

FIG. 3 is a structural block diagram of a monitoring device according to an embodiment of the present invention. This embodiment may be applicable to a scenario in which a person to be guarded is monitored, such as a person who has a language barrier and a mobility disability. The situation where the call is made when the person to be monitored has a call request. The device may be implemented by at least one of software and hardware and integrated on a monitoring device.

As shown in FIG. 3, the monitoring device includes: an acquisition module 31, a person determination module 32, a sight determination module 33, and a call module 34.

The acquisition module 31 is configured to acquire a facial image of a person in a monitoring area;

The person determination module 32 is configured to determine that a person corresponding to the face image is a person to be monitored when the face image matches a preset image;

The sight line determining module 33 is configured to determine the sight line of the person to be monitored based on at least the eye image in the face image of the person to be monitored;

The calling module 34 is configured to: if the length of time that the line of sight of the person to be monitored looks at any sub-region in the preset area exceeds the warning period, according to the sub-region of the fixation, or the sub-region of the fixation and the person to be monitored To determine the call level and make a corresponding call, wherein the preset area includes multiple sub-areas, and different sub-areas correspond to different call levels.

In this embodiment, the monitoring device first obtains a face image of a person in the monitoring area through the acquisition module 31; secondly, the person determination module 32 determines that the face image corresponds to when the face image matches a preset image The person to be monitored is the person to be monitored; then the line of sight determination module 33 is used to determine the line of sight of the person to be monitored based on the eye image in the face image of the person to be monitored; finally, the calling module 34 is the line of sight of the person to be monitored When the duration of gazing at any sub-area in the preset area exceeds the warning duration, a call level is determined according to the gazing sub-area and the physical state of the person to be monitored and a corresponding call is made.

A monitoring device provided in this embodiment can identify the face image of a person in a monitoring area before analyzing the sight of a person to be monitored, thereby avoiding calls caused by persons who are not to be monitored, and improving the accuracy of the call. . In addition, after the person to be monitored looks at any subregion contained in the preset area and the gaze duration exceeds the warning period, a targeted call is made according to the subregion to be monitored and the physical status of the person to be monitored, reducing medical care. The workload of the personnel also slows down the illness of the person to be monitored and effectively improves the experience of the person to be monitored and the medical staff.

In an optional embodiment of the present invention, the person determination module 32 includes: a feature extraction unit configured to extract feature information from the face image; and a person determination unit configured to when the feature information and the pre- When the preset feature information in the image matches, it is determined that the person corresponding to the facial image is the person to be monitored.

In an optional embodiment of the present invention, the sight determining module 33 is configured to determine position information of the person to be monitored according to a face image and a reference image of the person to be monitored; based on the position information and the Eye movement characteristics in the eye image determine the sight line of the person to be monitored.

Based on the above technical solution, the call module 34 is configured to search the preset level data table to determine the subarea and location of the gaze when the length of time that the person to be monitored looks at any subarea in the preset area exceeds the warning duration. Describe the call level corresponding to the physical status of the person to be monitored and make corresponding calls.

In an optional embodiment of the present invention, the monitoring device further includes: a reference image input module 35, configured to include environmental information when entering the face image of the person to be monitored before the face image of the person in the monitoring area is acquired. As the reference image.

In an optional embodiment of the present invention, the monitoring device further includes: a voice reminder module 36 configured to, if the duration of the person to be monitored looks at any sub-area in the preset area exceeds the warning duration, Then, a call level is determined according to the sub-area of the gaze and the physical state of the person to be monitored and a corresponding call is made, and then a voice reminder corresponding to the call level is sent to the medical staff who cares for the person to be monitored.

The above-mentioned monitoring device can execute the monitoring method provided by any embodiment of the present invention, and has corresponding function modules and beneficial effects of executing the method.

Embodiment 4

FIG. 4 is a structural block diagram of a monitoring device according to an embodiment of the present invention. As shown in FIG. 4, the monitoring device includes: one or more processors 41 and a storage device 42. The monitoring device may include one or more processors 41 (take one processor 41 as an example in FIG. 4). The storage device 42 is configured to store one or more programs; the one or more programs are executed by the one or more processors 41, so that the one or more processors 41 implement the monitoring method described above.

The monitoring device may further include a camera 43, an output device 44, and a communication device 45.

The processor 41, the storage device 42, the camera 43, the output device 44, and the communication device 45 in the monitoring device may be connected through a bus or other methods. In FIG. 4, the connection through the bus is taken as an example.

The storage device 42 in the monitoring device serves as a computer-readable storage medium and can be used to store one or more programs. The programs may be software programs, computer-executable programs, and modules, as in the first or second embodiment of the present invention. Provide program instructions / modules corresponding to the monitoring method (for example, the modules in the monitoring device shown in FIG. 3 include: an acquisition module 31, a personnel determination module 32, a sight determination module 33, and a call module 34, and further include: reference image entry Module 35 and voice reminder module 36). The processor 41 runs software programs, instructions, and modules stored in the storage device 42 to execute various functional applications and data processing of the monitoring device, that is, to implement the monitoring method in the foregoing method embodiment.

The storage device 42 may include a storage program area and a storage data area, wherein the storage program area may store an operating system and application programs required for at least one function; the storage data area may store data created according to the use of the device, and the like. In addition, the storage device 42 may include a high-speed random access memory, and may further include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some examples, the storage device 42 may further include memories remotely provided with respect to the processor 41, and these remote memories may be connected to the device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The camera 43 may be used to record an eye image of a person to be monitored as a preset image, and acquire an eye image of a person in a monitoring area. The output device 44 may include a display device such as a display screen, a speaker, or an indicator light. The communication device 45 may send calls corresponding to different call levels to different terminals.

In addition, when the one or more programs included in the monitoring device are executed by the one or more processors 41, the program performs the following operations: obtaining a face image of a person in the monitoring area; if the face image and a preset Image matching, it is determined that the person corresponding to the face image is a person to be monitored; at least according to the eye image in the face image of the person to be monitored, the line of sight of the person to be monitored is determined; Gaze at any sub-area in the preset area for longer than the pre-warning time, then determine the call level and make the corresponding call based on the sub-area of the fixation, or the sub-area of the fixation and the physical status of the person to be monitored The preset area includes multiple sub-areas, and different sub-areas correspond to different call levels.

In addition, an embodiment of the present invention also provides a computer-readable storage medium having stored thereon a computer program for executing a monitoring method when the program is executed by a processor, the method including: obtaining a face of a person in a monitoring area Image; if the face image matches a preset image, determine that the person corresponding to the face image is a person to be monitored; at least according to an eye image in the face image of the person to be monitored, determine the person to be monitored Line of sight of the person; if the length of time that the line of sight of the person to be monitored gazed at any sub-region in the preset area exceeds the pre-warning time, then State, determine a call level and make a corresponding call, wherein the preset area includes multiple sub-areas, and different sub-areas correspond to different call levels.

Optionally, when the program is executed by the processor, the program may also be used to execute a technical solution of a monitoring method provided by any embodiment of the present invention. Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by software and necessary general hardware, and of course, can also be implemented by hardware, but in many cases the former is a better implementation . Based on such an understanding, the technical solution of the present invention that is essentially or contributes to related technologies can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a computer floppy disk, Read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including several instructions to make a computer device (can be a personal computer, A server, or a network device, etc.) perform the methods described in the embodiments of the present invention.

Note that the above are only the preferred embodiments of the present invention and the applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and those skilled in the art can make various obvious changes, readjustments and substitutions without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more equivalent embodiments may be included, and the present invention The scope is determined by the scope of the appended claims.

Claims

A method of monitoring, including:

Obtain facial images of people in the monitoring area;

If the face image matches a preset image, determining that the person corresponding to the face image is a person to be monitored;

Determining the line of sight of the person to be monitored at least based on an eye image in a face image of the person to be monitored;

If the length of time that the person to be monitored looks at any of the sub-regions in the preset area exceeds the warning period, the call is determined according to the sub-region of the gaze, or the sub-region of the gaze and the physical state of the person Level and make corresponding calls, where the preset area includes multiple sub-areas, and different sub-areas correspond to different call levels.
The method according to claim 1, wherein if the face image matches the preset image, determining the person corresponding to the face image as the person to be monitored comprises:

Extracting feature information from the face image;

If the feature information matches the preset feature information in the preset image, it is determined that the person corresponding to the face image is the person to be monitored.
The method according to claim 1, wherein determining the line of sight of the person to be monitored based on at least an eye image in a face image of the person to be monitored includes:

Determining position information of the person to be monitored according to a face image and a reference image of the person to be monitored;

Determine the line of sight of the person to be monitored according to the position information and eye movement characteristics in the eye image.
The method according to claim 1, wherein determining the call level and making a corresponding call according to the sub-area of gaze and the physical state of the person to be monitored comprises:

Look up the preset level data table, determine the call level corresponding to the gaze subregion and the body state of the person to be monitored, and make a corresponding call.
The method according to claim 1, further comprising: before acquiring a facial image of a person in the monitoring area:

A face image containing environmental information when the person to be monitored faces the reference point is entered as a reference image.
The method according to claim 1, wherein, if the length of time that the line of sight of the person to be monitored looks at any sub-region in the preset area exceeds the warning period, the determination is based on the sub-region of the gaze and the physical state of the person After calling the level and making the corresponding call, it also includes:

Send a voice reminder corresponding to the call level to a medical staff who cares for the person to be monitored.
A monitoring device includes:

An acquisition module configured to acquire a facial image of a person in a monitoring area;

A person determination module configured to determine that a person corresponding to the face image is a person to be monitored when the face image matches a preset image;

A sight line determining module configured to determine a sight line of the person to be monitored based on at least an eye image in a face image of the person to be monitored;

A calling module configured to, if the length of time that the line of sight of the person to be monitored looks at any sub-region in the preset area exceeds the warning period, according to the sub-region of the fixation, or according to the sub-region of the fixation and the person to be monitored Physical state, determining a call level and making a corresponding call, wherein the preset area includes multiple sub-areas, and different sub-areas correspond to different call levels.
The apparatus according to claim 7, wherein the person determination module comprises:

A feature extraction unit configured to extract feature information from the face image;

The person determination unit is configured to determine, when the feature information matches the preset feature information in the preset image, that a person corresponding to the face image is the person to be monitored.
A monitoring device includes:

One or more processors;

A storage device configured to store one or more programs;

The one or more programs are executed by the one or more processors, so that the one or more processors implement the monitoring method according to any one of claims 1-6.
A computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the monitoring method according to any one of claims 1-6 is implemented.