WO2021179624A1

WO2021179624A1 - Monitoring method and system, electronic device, and storage medium

Info

Publication number: WO2021179624A1
Application number: PCT/CN2020/124151
Authority: WO
Inventors: 方志军; 李若岱; 罗彬�; 田士民
Original assignee: 深圳市商汤科技有限公司
Priority date: 2020-03-13
Filing date: 2020-10-27
Publication date: 2021-09-16
Also published as: JP2022526207A; SG11202106842PA; CN111414831B; CN111414831A; TW202134940A; TWI768641B

Abstract

A monitoring method and system, an electronic device, and a storage medium. The method comprises: carrying out target region recognition on a visible light image of a monitoring region, and obtaining a first region where a target object is located and a second region where a preset part of the target object is located in the visible light image (S11); performing living body detection on the target object according to size information of the first region, and a third region, in an infrared image of the monitoring region, corresponding to the first region, to obtain a living body detection result (S12); when the living body detection result shows a living body, carrying out identity recognition on the first region to obtain identity information of the target object, and carrying out temperature measurement on a fourth region, in the infrared image, corresponding to the second region, to obtain temperature information of the target object (S13); and generating first early warning information according to the target object and the identity information thereof under the condition that the temperature information is greater than or equal to a preset temperature threshold (S14). According to the present method, living body detection and temperature measurement can be carried out on the basis of an infrared image, the body temperature of a target object in the image can be quickly measured, the body temperature measurement efficiency is improved, and the monitoring method is suitable for places having large pedestrian volume.

Description

Monitoring method and system, electronic equipment and storage medium

Cross-references to related applications

This disclosure is based on a Chinese patent application with an application number of 202010177537.0 and an application date of March 13, 2020, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated by reference into this disclosure.

Technical field

The present disclosure relates to the field of computer vision technology, and relates to a monitoring method and system, electronic equipment, and storage medium.

Background technique

In the prevention and control of respiratory viruses, early detection and early isolation are the most primitive and extremely effective measures. Early detection refers to the early detection of suspicious persons by means of medical diagnosis and testing, which may include temperature detection. Early isolation refers to reducing the contact between virus-carrying patients and others, and reducing the risk of cross-infection, which can include non-contact temperature testing, wearing masks and other measures.

In related technologies, body temperature can be detected by means of thermometers, thermal imaging, etc., but the physical examination and detection process is cumbersome, it is difficult to adapt to places with a large flow of people, and it is difficult to verify the identity of the suspected case when a suspected case with fever symptoms is detected information.

Summary of the invention

The embodiments of the present disclosure propose a monitoring method and system, electronic equipment, and storage medium.

According to an aspect of the embodiments of the present disclosure, a monitoring method is provided, which includes: performing target area recognition on a visible light image of a monitoring area, and obtaining a first area where a target object is located in the visible light image and a preset of the target object The second area where the part is located; according to the size information of the first area and the third area corresponding to the first area in the infrared image of the monitoring area, the target object is detected in vivo to obtain the in vivo detection Result; in the case that the result of the living body detection is a living body, the first area is subjected to identification processing to obtain the identity information of the target object, and the fourth area corresponding to the second area in the infrared image is obtained Perform temperature detection processing to obtain temperature information of the target object; in the case that the temperature information is greater than or equal to a preset temperature threshold, generate first warning information according to the identity information of the target object.

According to the monitoring method of the embodiment of the present disclosure, living body detection and temperature detection can be performed based on infrared images, the body temperature of the target object in the image can be quickly detected, the efficiency of body temperature detection can be improved, and it is suitable for places with a large flow of people. And the identification information of the target object can be identified through the visible light image, which helps to determine the identity information of the suspected case and improve the efficiency of epidemic prevention.

In a possible implementation manner, according to the size information of the first area and the third area corresponding to the first area in the infrared image, the target object is subjected to live detection to obtain a live detection result , Including: determining the distance between the target object and the image acquisition device that acquires the visible light image according to the size information of the first area; determining the living body detection strategy according to the distance; For the position in the visible light image, the position of the third region is determined in the infrared image; according to the living body detection strategy, the living body detection processing is performed on the third region in the infrared image to obtain the living body detection result.

In this way, the living body detection strategy can be determined by the size of the first region, and the living body detection can be performed based on the living body detection strategy, and isothermal analysis can be used to improve the living body detection accuracy in the far-infrared image.

In a possible implementation manner, determining the living body detection strategy according to the distance includes one of the following: when the distance is greater than or equal to a first distance threshold, determining the living body detection strategy to be based on A living body detection strategy based on the shape of the human body; when the distance is greater than or equal to the second distance threshold and less than the first distance threshold, determining the living body detection strategy as a living body detection strategy based on the shape of the head and neck; If it is less than the second distance threshold, the living body detection strategy is determined as a living body detection strategy based on facial morphology.

In a possible implementation manner, performing a living detection process on the third region according to the living body detection strategy to obtain the living body detection result includes: performing a living body detection process on the third region according to the living body detection strategy Morphology detection processing to obtain a morphology detection result; in the case that the morphology detection result is a living body shape, perform isothermal analysis processing on the face area in the third area to obtain an isothermal analysis result; determine according to the living body detection strategy The first weight of the shape detection result and the second weight of the isothermal analysis result; the living body detection result is determined according to the first weight, the second weight, the shape detection result, and the isothermal analysis result.

In a possible implementation manner, the method further includes, in a case where the living body detection result is a living body, detecting whether a preset target is included in the first area, and obtaining the first detection result, wherein the The preset target includes an object that occludes a partial area of the face. In the case that the live body detection result is a live body, the identity recognition process is performed on the first area to obtain the identity information of the target object, including: in the live body detection result In the case of a living body, according to the first detection result, an identity recognition process is performed on the first area to obtain the identity information of the target object.

In this way, the identification method can be selected according to whether there is a preset target, and the accuracy of identification can be improved.

In a possible implementation manner, detecting whether a preset target is included in the first area, and obtaining the first detection result includes: performing detection processing on a face area in the first area to determine the face The feature missing result of the region; in the case that the feature missing result is a preset feature missing, detecting whether the face region includes a preset target, and obtaining the first detection result.

In a possible implementation manner, performing identity recognition processing on the first area according to the first detection result to obtain the identity information of the target object includes one of the following: If the result is that the preset target does not exist, perform first identity recognition processing on the face area in the first area to obtain the identity information of the target object; when the first detection result is that the In the case of a preset target, a second identity recognition process is performed on the face area in the first area to obtain the identity information of the target object, wherein the features of the unoccluded area of the face in the second identity recognition process The weight of is greater than the weight of the feature of the corresponding area in the first identification process.

In a possible implementation manner, the method further includes: in a case where the first detection result is that the preset target does not exist, generating second early warning information.

In a possible implementation manner, the method further includes: acquiring gender information of the target object; and determining the preset temperature threshold according to the gender information.

In a possible implementation, the method further includes: comparing the position information of the first area or the third area, the temperature information of the target object, and the identity information of the target object with the The visible light image and/or the infrared image are superimposed to obtain a detection image.

According to an aspect of the embodiments of the present disclosure, a monitoring system is provided, including: a visible light image acquisition part, an infrared image acquisition part, and a processing part, the processing part is configured to: The image performs target area recognition to obtain the first area where the target object is located in the visible light image and the second area where the preset part of the target object is located; according to the size information of the first area, and the first area In the third area corresponding to the infrared image acquired by the infrared image acquisition part, perform live detection on the target object to obtain the live detection result; in the case where the live detection result is a live body, perform identity recognition processing on the first area , Obtain the identity information of the target object, and perform temperature detection processing on the fourth area corresponding to the second area in the infrared image to obtain the temperature information of the target object; when the temperature information is greater than or equal to In the case of a preset temperature threshold, the first warning information is generated according to the identity information of the target object.

In a possible implementation, the processing part is further configured to: determine the distance between the target object and the image acquisition device that acquires the visible light image according to the size information of the first region; According to the distance, determine the living body detection strategy; according to the position of the first region in the visible light image, determine the position of the third region in the infrared image; according to the living body detection strategy, determine the position of the third region in the infrared image In the third area, a living body detection process is performed to obtain the living body detection result.

In a possible implementation manner, the processing part is further configured to: in a case where the distance is greater than or equal to a first distance threshold, determine the living body detection strategy as a living body detection strategy based on human body shape; or When the distance is greater than or equal to the second distance threshold and less than the first distance threshold, the living body detection strategy is determined to be a living body detection strategy based on the head and neck shape; or when the distance is smaller than the second distance In the case of a threshold value, the living body detection strategy is determined as a living body detection strategy based on facial morphology.

In a possible implementation manner, the processing part is further configured to: perform morphological detection processing on the third region according to the living body detection strategy to obtain a morphological detection result; when the morphological detection result is a living body morphology In the case of performing isothermal analysis processing on the face area in the third area to obtain an isothermal analysis result; determining the first weight of the morphological detection result and the second weight of the isothermal analysis result according to the living body detection strategy; According to the first weight, the second weight, the morphological detection result, and the isothermal analysis result, the living body detection result is determined.

In a possible implementation manner, the processing part is further configured to: in a case where the living body detection result is a living body, detect whether a preset target is included in the first area, and obtain the first detection result, wherein , The preset target includes an article that occludes a partial area of the face, and the processing part is further configured to: in a case where the result of the detection of a living body is a living body, according to the result of the first detection, the first area is Perform identity recognition processing to obtain the identity information of the target object.

In a possible implementation manner, the processing part is further configured to: perform detection processing on the facial region in the first region to determine the feature missing result of the facial region; If the feature is missing, it is detected whether a preset target is included in the face area, and the first detection result is obtained.

In a possible implementation manner, the processing part is further configured to: in the case where the first detection result is that the preset target does not exist, perform a first operation on the face area in the first area. Identity recognition processing to obtain the identity information of the target object; or in the case where the first detection result is that the preset target exists, the second identity recognition processing is performed on the face area in the first area to obtain In the identity information of the target object, the weight of the feature of the unoccluded area of the face in the second identity recognition processing is greater than the weight of the feature of the corresponding area in the first identity recognition processing.

In a possible implementation manner, the processing part is further configured to: combine the position information of the first area or the third area, the temperature information of the target object, and the identity information of the target object, Perform superposition processing with the visible light image and/or infrared image to obtain a detection image.

In a possible implementation manner, the processing part is further configured to: obtain gender information of the target object; and determine the preset temperature threshold according to the gender information.

In a possible implementation manner, the processing part is further configured to generate second warning information when the first detection result is that the preset target does not exist.

According to an aspect of the embodiments of the present disclosure, there is provided an electronic device including: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to execute the above-mentioned monitoring method.

According to an aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the foregoing monitoring method is implemented.

According to an aspect of the present disclosure, there is provided a computer program including computer readable code, and when the computer readable code runs in an electronic device, a processor in the electronic device executes for realizing the above-mentioned monitoring method.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, rather than limiting the embodiments of the present disclosure.

According to the following detailed description of exemplary embodiments with reference to the accompanying drawings, other features and aspects of the present disclosure will become clear.

Description of the drawings

The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments that conform to the embodiments of the present disclosure, and are used together with the specification to describe the technical solutions of the embodiments of the present disclosure.

Fig. 1 shows a flowchart of an image processing method according to an embodiment of the present disclosure;

FIG. 2 shows a schematic diagram of living body detection according to an embodiment of the present disclosure;

Fig. 3 shows a schematic diagram of identity recognition processing according to an embodiment of the present disclosure;

4A and 4B show schematic diagrams of application of a monitoring method according to an embodiment of the present disclosure;

Figure 5 shows a block diagram of a monitoring system according to an embodiment of the present disclosure;

FIG. 6 shows a block diagram of an electronic device according to an embodiment of the present disclosure;

FIG. 7 shows a block diagram of an electronic device according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the drawings. The same reference numerals in the drawings indicate elements with the same or similar functions. Although various aspects of the embodiments are shown in the drawings, unless otherwise noted, the drawings are not necessarily drawn to scale.

The dedicated word "exemplary" here means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" need not be construed as being superior or better than other embodiments.

The term "and/or" in this article is only an association relationship describing the associated objects, which means that there can be three relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, exist alone B these three situations. In addition, the term "at least one" in this document means any one of a plurality of or any combination of at least two of the plurality, for example, including at least one of A, B, and C, may mean including A, Any one or more elements selected in the set formed by B and C.

In addition, in order to better illustrate the embodiments of the present disclosure, numerous details are given in the following embodiments. Those skilled in the art should understand that the embodiments of the present disclosure can also be implemented without certain details. In some instances, the methods, means, elements, and circuits well-known to those skilled in the art have not been described in detail, so as to highlight the gist of the embodiments of the present disclosure.

Fig. 1 shows a flowchart of an image processing method according to an embodiment of the present disclosure. As shown in Fig. 1, the method includes:

In step S11, perform target area recognition on the visible light image of the monitoring area, and obtain the first area where the target object is located in the visible light image and the second area where the preset part of the target object is located in the visible light image;

In step S12, according to the size information of the first area and the third area corresponding to the first area in the infrared image of the monitoring area, perform a live detection on the target object to obtain a live detection result;

In step S13, in the case that the result of the living body detection is a living body, the identity recognition processing is performed on the first area to obtain the identity information of the target object, and the corresponding information of the second area in the infrared image is obtained. The fourth area performs temperature detection processing to obtain temperature information of the target object;

In step S14, when the temperature information is greater than or equal to a preset temperature threshold, first warning information is generated according to the identity information of the target object.

In a possible implementation manner, the monitoring method may be executed by a terminal device or other processing equipment, where the terminal device may be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, or a cordless Telephones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. Other processing equipment can be servers or cloud servers. In some possible implementations, the monitoring method can be implemented by a processor invoking computer-readable instructions stored in the memory.

In a possible implementation, the monitoring method can be used in front-end monitoring equipment. For example, the front-end monitoring equipment may be a device that integrates a processor, a camera, and other parts into a whole. The processor can control the camera and other parts to obtain The image of the monitoring area is monitored, and the monitoring method is executed by the processor. In a possible implementation, the monitoring method can also be used in a server (for example, the processor that executes the monitoring method is located in the server, and the processor and the camera are not packaged as a whole, but are distributed in the monitoring area and the back end. In the processor, the monitoring system is composed, and the server can receive the image taken by the camera in the monitoring area, and execute the monitoring method on the image.

In a possible implementation manner, the processor may be a System on Chip (SoC), and the camera may include an infrared camera for acquiring infrared images and a camera for acquiring visible light images. In a possible implementation, special sensors such as vanadium oxide uncooled infrared focal plane detectors can be used to capture spatial light signals. The sensors can be used in far-infrared cameras to obtain life light and perform temperature monitoring.

In a possible implementation, the camera can capture a video of the monitored area, and the video frames of the video are the visible light image and the infrared image. The infrared camera and the visible light camera can be arranged in close positions, for example, the infrared camera and the visible light camera can be arranged adjacently, side by side, etc., or the two cameras can be arranged as a whole, etc. The embodiment of the present disclosure does not limit the setting method. Therefore, in the images captured by the two cameras at the same time, the position of each target object is similar, that is, the position deviation of the same target object in the optical image and the corresponding infrared image is small, and the deviation can be based on The positional relationship between the two cameras is corrected. For example, the position of the target object in the third area in the infrared image can be determined according to the position of the first area in the visible light image (that is, the area where the target object is located).

In a possible implementation manner, the image captured by the camera may have poor image quality, for example, the image is blurred, the focal length error, and the camera is contaminated or blocked. The image quality can be checked first. For example, the image quality of the visible light image can be detected, for example, the texture and the definition of the boundary of the visible light image can be detected. In the case where the sharpness is greater than or equal to the sharpness threshold, the image quality can be considered to be good, and the next position detection and other processing can be performed. When the sharpness is less than the sharpness threshold, the image quality can be considered to be poor, and the visible light image and the infrared image corresponding to the visible light image can be deleted at the same time.

In a possible implementation manner, in step S11, the visible light image can be identified by the target area to obtain the first area where the target object is located. The first area may be the face area and/or the target object of the target object. The human body area of the subject. This processing can obtain the position coordinates of the first area where the target object is located, or the position coordinates of the detection frame containing the first area.

In a possible implementation manner, in the process of temperature detection, the temperature of the forehead and other areas can be measured. For example, the temperature of the forehead area can be determined by the pixel value of the infrared image, and then the body temperature of the target object can be obtained. Therefore, the second area where the preset part (for example, forehead and other areas) is located in the visible light image can be detected, and then the fourth area of the preset part in the infrared image can be determined, and the pixel value and temperature of the infrared image can be established in advance. According to the corresponding relationship between the two, the body temperature of the target object can be determined according to the pixel value in the fourth area.

In a possible implementation, a living body is a biologically active, real biological modality exhibited by a human body with vital characteristics. The prosthesis is a model that is similar to the living body (for example, photos, masks, headgear, etc.) made by imitating the biological characteristics of the living body. Generally, in the process of living body detection, a near-infrared image of a human face is used for living body detection. It is mainly based on the difference in imaging characteristics between the living body and the prosthesis under near-infrared light, such as the difference in optical flow, texture, color and other characteristics, so as to realize the distinction between the living body and the prosthesis. However, in the process of temperature monitoring, the infrared image obtained by the far-infrared camera is a far-infrared image (ie, an image based on the light of life), which is not a commonly used near-infrared image. Therefore, it can pass through the first area where the target object is located. Size, determine the living body detection strategy, and improve the accuracy of living body detection.

In a possible implementation manner, step S12 may include: determining the distance between the target object and the image acquisition device that acquires the visible light image according to the size information of the first region; A living body detection strategy; according to the position of the first region in the visible light image, the position of the third region in the infrared image is determined; according to the living body detection strategy, the third region in the infrared image is performed The living body detection processing obtains the living body detection result.

In a possible implementation, the first area is the area where the target object in the visible light image is located. For example, it may be the face area of the target object. The distance between the devices (visible light cameras) is negatively correlated, that is, the larger the area size, the smaller the distance, and the smaller the area size, the greater the distance. Further, a living body is characterized in an infrared image with unique vital characteristics (for example, a far-infrared image based on the light of life). The blood vessels of the human body are all over the body (for example, the human body, the face, the neck and shoulders, etc.), but in response to the close position of the target object in the visible light image, only part of the body is shown in the visible light image, for example, only It presents the face, head and neck, etc., but the facial features are clearer. The farther the distance, the fewer facial features and the more human features. Based on different distances, different live detection strategies are combined to improve the accuracy of live detection, which is suitable for use at different distances.

The corresponding relationship between the reference size and the reference distance in the image can be established in advance, and the distance between the target object and the camera can be determined according to the proportional relationship between the size of the area where the target object is located and the reference size, and the reference distance.

Fig. 2 shows a schematic diagram of living body detection according to an embodiment of the present disclosure. As shown in Fig. 2, the distance between the target object and the camera can be determined according to the size information of the first area, and the living body detection strategy can be selected according to the distance. The corresponding third area in the infrared image can be determined according to the first area in the visible light image. For example, the first area is the face area of the target object A in the visible light image (for example, the target object A is relatively close to the camera, and only the face area of the target object A can be captured), and the third area is an infrared image The face area of target A in the middle. The first area is the head and neck area of the target object B in the visible light image (for example, the face and shoulders and neck areas of the target object B can be photographed, or the upper body area, etc.), and the third area is the head and neck area of the target object B in the infrared image area. The first area is the human body area of the target object C in the visible light image (for example, the target object C is far away from the camera, and the human body area of the target object C can be photographed), and the third area is the human body area of the target object C in the infrared image .

In a possible implementation manner, determining the living body detection strategy according to the distance of the target object includes one of the following: when the distance is greater than or equal to a first distance threshold, the living body detection strategy is Determined as a living body detection strategy based on the shape of the human body; in a case where the distance is greater than or equal to a second distance threshold and less than the first distance threshold, determining the living body detection strategy as a living body detection strategy based on the shape of the head and neck; In a case where the distance is less than the second distance threshold, the living body detection strategy is determined as a living body detection strategy based on facial morphology.

In a possible implementation manner, the first distance threshold and the second distance threshold may be determined according to parameters such as the range and focal length of the detection area of the camera. For example, when the distance between the target object and the camera is greater than or equal to the first distance threshold, the image captured by the camera can include the whole body of the target object, or most of the body, and a living body detection strategy based on the shape of the human body can be selected . In the case that the distance between the target object and the camera is less than the first distance threshold and greater than or equal to the second distance threshold, the picture captured by the camera cannot include the whole body of the target object, but may include the face, shoulders and neck of the target object In the upper body area, or the upper body area, you can choose a live detection strategy based on the shape of the head and neck. In the case where the distance between the target object and the camera is less than the second distance threshold, the image captured by the camera only includes the face area of the target object, and a living body detection strategy based on facial morphology can be selected.

In a possible implementation manner, the position of the third area is determined in the infrared image according to the position of the first area in the visible light image. In an example, the visible light image may include one or more target objects, that is, include one or more first regions. According to the positional relationship between the infrared camera and the visible light camera quality inspection, and the position of the first region, the third The location of the area in the infrared image.

In a possible implementation, since the infrared camera can acquire far-infrared images used to detect life rays, instead of near-infrared images used to detect image features, directly detect the third area of the target object in the far-infrared image. The accuracy rate may be low. And when there are multiple target objects, if the third area where the target object is located is directly detected in the infrared image, the correspondence between each third area and each first area in the visible light image is difficult to clarify. Therefore, the position of the third area corresponding to the first area can be obtained according to the position of the first area in the visible light image and the position relationship between the two cameras. Similarly, the position of the corresponding fourth area in the infrared image can be obtained through the position of the second area in the visible light image.

In a possible implementation manner, after the living body detection strategy is determined, the selected living body detection strategy can be used to perform the living body detection processing on the third region in the infrared image. According to the living body detection strategy, performing living body detection processing on the third area to obtain the living body detection result includes: performing morphological detection processing on the third area according to the living body detection strategy to obtain a morphological detection result; In the case that the shape detection result is a living body shape, isothermal analysis processing is performed on the face area in the third area to obtain the isothermal analysis result; according to the living body detection strategy, the first weight and the total weight of the shape detection result are determined The second weight of the isothermal analysis result; the living body detection result is determined according to the first weight, the second weight, the morphological detection result, and the isothermal analysis result.

In a possible implementation manner, the morphological detection of the third region may be performed based on the living body detection strategy.

In a possible implementation, in the living body detection strategy based on the human body shape, the third area may include the face area and the human body area of the target object. There are many human body features, which can be extracted through neural networks. The processing is to perform a shape detection process to obtain a human body feature amount to determine whether the human body shape of the target object is a living body shape (for example, whether the shape of the target object is a living body shape can be determined according to the characteristics of the target object's posture, movement, and shape).

In a possible implementation, in the life detection strategy based on the head and neck shape, the third area may include the face, shoulder and neck area of the target object, or the upper body area, and the feature extraction process may be performed through neural networks, etc. , To obtain the features of the face and shoulders and neck regions for shape detection processing to determine whether the head and neck shape of the target object is a living shape (for example, it can be judged whether the target object is based on the characteristics of the target object's posture, movement, shape Living form).

In a possible implementation, in the life detection strategy based on facial morphology, the third area may include the face area of the target object, with fewer human body features and more facial features, which can be achieved through neural networks, etc. Method to perform feature extraction processing to obtain the feature amount of the face area for shape detection processing to determine whether the facial shape of the target object is a living form (for example, it can be judged whether the target object is a living form based on the facial expression, texture and other characteristics of the target object ).

In a possible implementation manner, the shape detection result of the above-mentioned shape detection processing may be a result in the form of a score. For example, in the case where the score of the morphology detection is greater than or equal to the first score threshold, the morphology detection result can be considered as a living body shape. The embodiment of the present disclosure does not limit the form of the morphological detection result.

In a possible implementation manner, in a case where the shape detection result is a living body shape, isothermal analysis processing is performed on the face region in the third region to obtain an isothermal analysis result.

In a possible implementation manner, isothermal analysis can be used to determine whether the face area is a living body. For example, the real face of the target object has a uniform distribution of blood vessels, and if the target object wears a mask, headgear, etc. There is no blood vessel distribution in the body, and its isotherm is different from that of the real face. The isotherm can be used to determine whether the facial area is alive or not.

In a possible implementation manner, the result of the isothermal analysis may be a result in the form of a score. For example, when the score of the isothermal analysis is greater than or equal to the second score threshold, the result of the isothermal analysis may be considered to be a living body. The embodiment of the present disclosure does not limit the form of the isothermal analysis result.

In a possible implementation manner, because in different living body detection strategies, the proportion of the feature amount of the face area is different (for example, in the living body detection strategy based on human body shape, the third area includes the face area and the human body. Area, the proportion of the face area is small, and in the life detection strategy based on facial morphology, the third area only includes the face area, and the proportion of the face area is larger). The first weight of the morphological detection result and the second weight of the isothermal analysis result can be determined according to the living body detection strategy.

For example, in a living body detection strategy based on human body shape, since the proportion of the facial area is small, the second weight based on the isothermal analysis result of the facial area may be smaller, and the first weight of the shape detection result may be larger. For example, in a living body detection strategy based on facial morphology, since the proportion of the facial region is relatively large, the second weight of the isothermal analysis result based on the facial region may be larger, and the first weight of the morphological detection result may be smaller. For example, in a living body detection strategy based on the shape of the head and neck, the second weight and the first weight may be close or equal. The embodiments of the present disclosure do not limit the values of the first weight and the second weight.

In a possible implementation manner, the morphological detection result and the isothermal analysis result may be weighted and summed according to the first weight and the second weight, respectively, to obtain the living body detection result. In a possible implementation, the live body detection result can be a result in the form of a score. When the live body detection result is greater than or equal to the live body score threshold, the target object can be considered a live body; in the case where the live body detection result is less than the live body score threshold Next, it can be considered that the target object is a prosthesis or a non-human body, and no subsequent processing is required.

In a possible implementation manner, in step S13, in the case that the target object is a living body, the identity recognition processing may be performed on the target object. In a possible implementation, in medical and health places (such as hospitals, operating rooms, etc.) or in public places during an epidemic, it is usually necessary to wear protective equipment (such as masks, goggles, etc.). A certain degree of occlusion is caused to the facial area. Therefore, it is necessary to monitor whether the target object is wearing a mask and other preset items, and to determine the way of identification according to the detection result.

Fig. 3 shows a schematic diagram of identity recognition processing according to an embodiment of the present disclosure. The method further includes: in a case where the living body detection result is a living body, detecting whether a preset target is included in the first area, and obtaining the first detection result, wherein the preset target includes a partial area of the face For the object to be shielded, the first detection result is whether the target object wears a mask or other objects capable of shielding the facial area. Further, identity recognition may be performed according to the first detection result, and step S13 may include: in the case that the living body detection result is a living body, according to the first detection result, performing identity recognition processing on the first area to obtain the The identity information of the target object.

In a possible implementation manner, it may first be determined whether there is a preset target (for example, an item such as a mask) on the face of the target object. Detecting whether a preset target is included in the first area, and obtaining the first detection result includes: performing detection processing on a face area in the first area to determine a feature missing result of the face area; In the case that the feature missing result is a preset feature missing, detecting whether the face area includes a preset target, and obtaining the first detection result.

In a possible implementation, the facial region in the first region can be detected and processed through a neural network. For example, a feature map of the facial region can be obtained through a convolutional neural network, and a mask can be worn on the target object that can block the face In the case of an item (preset target), there are missing features in the feature map of the facial area. For example, if the target object wears a mask, the features of the target object's nose and mouth cannot be detected, that is, the features of the nose and mouth are missing. When the target object wears sunglasses, the eye features cannot be detected, that is, the eye features are missing.

In a possible implementation, in the absence of preset features (for example, nose and mouth features or eye features, etc.), the target object may wear masks, sunglasses and other items, and further detection may be performed, for example, , The face of the target object can be monitored by convolutional neural networks and other methods to detect whether there are preset targets such as masks. For example, the shape and texture of the mask can be detected to identify masks and other items, and the target object can be excluded from covering the face by hand or due to Other people’s occlusion, etc.

In a possible implementation, if the preset features (for example, nose and mouth features, eye features, etc.) are not missing (ie, facial features are complete), there is no preset target such as a mask. If the surveillance area is a hospital or other medical and health place, or a public place during the epidemic, not wearing a mask may be a violation of regulations. The method further includes: generating second warning information when the first detection result is that the preset target does not exist. Further, a second warning message can also be output through a warning device or a display to remind the target object to wear a mask. For example, the warning device may include an audio device, a warning lamp, etc., and may output the second warning information in a manner of sound or light, or may display information that the target object is not wearing a mask on the display. The embodiment of the present disclosure does not limit the output mode of the second early warning information.

In a possible implementation manner, the method of identity recognition may also be determined according to the first detection result (that is, whether the target object wears an object that can cover the face). According to the first detection result, perform identity recognition processing on the first area to obtain the identity information of the target object, including one of the following: when the first detection result is that the preset target does not exist In the case of performing the first identity recognition processing on the face area in the first area to obtain the identity information of the target object; in the case where the first detection result is that the preset target exists, the The face area in the first area is subjected to a second identity recognition process to obtain the identity information of the target object, wherein the weight of the feature of the unoccluded area of the face in the second identity recognition process is greater than that in the first identity recognition process The weight of the feature of the corresponding area.

In a possible implementation manner, a neural network can be used for identity recognition processing, where the neural network used for the first identity recognition processing is different from the neural network used for the second identity recognition processing. In the example, when the face area of the target object is partially occluded, only the unoccluded area can be used for the face area for identity recognition. In the process of neural network training, the features of the unoccluded area have a larger weight For example, the attention mechanism of the neural network is focused on the features of the unoccluded area, and it is conducive to the feature of the unoccluded area for identity recognition.

In a possible implementation manner, in the case where the facial region of the target object is not occluded, all features of the facial region can be used for identity recognition processing, and the neural network can perform identity recognition based on all the features of the facial region.

For example, when the target object wears a mask, the neural network can only obtain the features of the eyes and eyebrows (the nose and mouth features are missing), the neural network can increase the weights of the eye and eyebrow features in the training to Eye and eyebrow features for identification. In the case that the target object is not wearing a mask and the neural network can obtain all the features of the facial area, the neural network can perform identity recognition based on all the features of the facial area.

For example, the neural network can determine the similarity between the eye and eyebrow features and the reference features in the database (the reference features of the eye and eyebrows), and the similarity is greater than or equal to the similarity threshold corresponding to the reference features The identity information is determined as the identity information of the target object. Similarly, the neural network can determine the similarity between facial features and reference features in the database (face reference features), and determine the identity information corresponding to the reference features whose similarity is greater than or equal to the similarity threshold as the identity of the target object information.

For example, if there is no reference feature in the database whose feature similarity with the target object is greater than or equal to the similarity threshold (that is, the target object does not match the existing identity information in the database), the feature of the target object can be changed It is stored in the database, and an identity (for example, identity code, etc.) is added to the target object. The embodiment of the present disclosure does not limit the method of identification.

In a possible implementation, the gender of the target object can also be identified. In a possible implementation, there is a certain difference between the average body temperature of men and women. Therefore, the preset temperature threshold can be set differently according to gender. For example, the average body temperature of women is 0.3°C higher than that of men. Gender is used to set the preset temperature threshold, such as 37 degrees for men and 37.3 degrees for women. The method further includes: acquiring gender information of the target object; and determining the preset temperature threshold according to the gender information.

In a possible implementation manner, the gender information of the target object can be determined based on the facial features of the target object. For example, when the target object is not wearing a mask, the gender of the target object can be determined based on all the features of the target object’s face. When the target object wears a mask, the gender of the target object can be determined according to the characteristics of the eyes and eyebrows of the target object. Alternatively, in the case where the facial features or eye and eyebrow features of the target object match the reference features in the database, the gender of the target object can be determined according to the gender recorded in the identity information corresponding to the reference feature in the database. The embodiment of the present disclosure does not limit the method for determining gender.

In a possible implementation manner, the preset temperature threshold may be determined according to gender information. For example, if the average body temperature of women is higher than the average body temperature of men, the preset temperature threshold for women may be higher than the preset temperature threshold for men. The embodiment of the present disclosure does not limit the method for determining the preset temperature threshold.

In a possible implementation manner, in step S13, the fourth area (corresponding to the second area where the preset position in the visible light image is located) in the infrared image (for example, the forehead) is also temperature The detection process determines the temperature information (for example, body temperature) of the target object. For example, the pixel value of the pixel in the infrared image may represent the color temperature value, and the color temperature value may be used to determine the temperature of the forehead area of the target object. The embodiment of the present disclosure does not limit the manner of determining the temperature information.

In a possible implementation manner, in step S14, when the temperature information is greater than or equal to a preset temperature threshold, first warning information is generated according to the identity information of the target object. For example, during an epidemic, fever is a suspected symptom, and you can focus on monitoring the target object with symptoms. After detection, when the body temperature of the target object is greater than or equal to the preset temperature threshold, it is considered that the target object has fever symptoms, and the first warning information can be generated according to the identity information of the target object. For example, the target object’s information can be displayed on the display. Identity information and body temperature, so as to focus on monitoring the target object. For example, the first warning message can be issued through audio and other equipment to remind the epidemic prevention personnel to focus on the target object, or to isolate the target object. The embodiment of the present disclosure does not limit the output mode of the first early warning information.

In a possible implementation manner, the visible light image and/or infrared image can be displayed through the display, and the identity information and/or temperature information of the target object can be displayed at the location of the target object in the visible light image and/or infrared image. For example, the identity information and/or temperature information of the target object may be superimposed on the first area where the target object in the visible light image is located, that is, the identity information and/or temperature information of the target object may be displayed in the first area. Similarly, the identity information and/or temperature information of the target object can be displayed in the third area of the infrared image. The embodiment of the present disclosure does not limit the display mode.

According to the monitoring method of the embodiment of the present disclosure, the living body detection strategy can be determined by the size of the first area, and the living body detection strategy can be performed based on the living body detection strategy. Isothermal analysis can be used to improve the living body detection accuracy in the far-infrared image, and can be based on Whether there is a preset goal to select the identification method to improve the accuracy of identification. Further, temperature detection can be performed on preset parts, which can quickly detect the body temperature of the target object in the image, improve the efficiency of body temperature detection, and is suitable for places with a large flow of people. And the identification information of the target object can be identified through the visible light image, which helps to determine the identity information of the suspected case and improve the efficiency of epidemic prevention.

4A and 4B show a schematic diagram of the application of the monitoring method according to an embodiment of the present disclosure. As shown in FIG. 4A, the monitoring method can be used in a system for monitoring the monitoring area. An infrared camera 1 for obtaining infrared images by light, a visible light camera 2 for obtaining visible light images, and a processor 3 (for example, a system-on-chip SoC). The system may also include a supplementary light unit 4 used in an environment with poor lighting conditions, a display 5 for displaying visible light images, infrared images and/or warning information, and a communication unit for transmitting warning information, temperature information and other information 6. The warning unit 7 for outputting early warning information, and the interactive unit 8 for inputting instructions, etc.

In a possible implementation, as shown in FIG. 4B, the infrared camera 1 and the visible light camera 2 can capture video frames of the monitored area, for example, visible light image AF (AF1-AF25) and infrared image BF (BF1-BF25). The position correction of the visible light image AF and the infrared image BF can be performed, that is, the position of the third area in the infrared image BF can be determined according to the position of the first area in the visible light image AF.

In a possible implementation manner, the processor 3 can perform image quality detection on the visible light image AF, for example, the detection can detect the texture of the light image and the sharpness of the boundary, and delete the visible light image with poor image quality and the corresponding Infrared image. Further, when there is no target object in the visible light image, the visible light image and the corresponding infrared image are deleted. For example, if there is no person in the monitoring area, the visible light image and the corresponding infrared image can also be deleted.

In a possible implementation manner, the processor 3 may perform position detection processing on the visible light image AF to obtain the first area where the target object is located, and may also obtain the second area where the forehead of the target object is located. According to the positions of the first area and the second area in the visible light image, the position of the third area where the target object is located in the infrared image and the position of the fourth area where the forehead of the target object is located in the infrared image can be determined.

In a possible implementation manner, the processor 3 may determine the distance between the target object and the camera according to the size information of the first area, and determine the living body detection strategy based on the distance.

In a possible implementation manner, in a case where the distance of the target object is greater than or equal to the first distance threshold, the living body detection strategy is determined as the living body detection strategy based on the human body shape. The shape detection can be performed on the third area in the infrared image to obtain the shape detection score. And perform isothermal analysis on the facial area in the infrared image to obtain isothermal analysis scores. Further, according to a living body detection strategy based on human body shape, a larger first weight can be assigned to the shape detection score, and a smaller second weight can be assigned to the isothermal analysis score, and the living body detection result can be obtained after the weighted summation.

In a possible implementation manner, when the distance of the target object is less than the first distance and greater than or equal to the second distance threshold, the living body detection strategy is determined as the living body detection strategy based on the head and neck shape. The shape detection can be performed on the third area in the infrared image to obtain the shape detection score. And perform isothermal analysis on the facial area in the infrared image to obtain isothermal analysis scores. Further, according to the living body detection strategy based on the head and neck morphology, similar weights can be assigned to the morphological detection score and the isothermal analysis score, and after the weighted summation, the living body detection result can be obtained.

In a possible implementation manner, in a case where the distance of the target object is less than the second distance threshold, the living body detection strategy is determined as the living body detection strategy based on the facial morphology. The shape detection can be performed on the third area in the infrared image to obtain the shape detection score. And perform isothermal analysis on the facial area in the infrared image to obtain isothermal analysis scores. Further, according to the living body detection strategy based on facial morphology, a smaller first weight can be assigned to the morphological detection score, and a larger second weight can be assigned to the isothermal analysis score. After the weighted summation, the living body detection result can be obtained.

In a possible implementation manner, the processor 3 can detect whether the target object is wearing a mask in the first area of the visible light image, and when the target object is not wearing a mask, it can generate second warning information and display it on the display 5. The second warning information, or the second warning information is played through the warning unit 7.

In a possible implementation, the processor 3 can determine the method of identity recognition according to whether the target object is wearing a mask. For example, if the target object is not wearing a mask, it can be based on all the facial features of the target object and the data in the database. Compare with reference characteristics to determine the identity information of the target object. When the target object wears a mask, the target object's eye and eyebrow characteristics can be compared with reference characteristics in the database to determine the target object's identity information. Further, the second early warning information may also include the identity information of the target object, for example, it may be displayed on the display 5 that XXX (name) is not wearing a mask, etc.

In a possible implementation manner, the processor 3 may recognize the gender of the target object, for example, may recognize the gender of the target object according to the facial features of the target object, or obtain the gender of the target object according to the identity information of the target object. Further, the preset temperature threshold of the male target and the preset temperature threshold of the female target can be determined respectively according to the gender of the target object. Alternatively, the preset temperature threshold may also be input through the interaction unit 8. The embodiment of the present disclosure does not limit the setting method of the preset temperature threshold.

In a possible implementation manner, the temperature of the fourth area (that is, the area where the forehead in the infrared image is located) may be monitored to obtain the body temperature of the target object. In the case where the body temperature exceeds the preset temperature threshold, the processor 3 may generate the first warning information. And the first warning information is displayed through the display 5, or the first warning information is played through the warning unit 7. Further, the identity information, body temperature, and position information of the camera (ie, the location where the feverish target object appears) can be transmitted to the background database or server through the communication unit 6, so as to facilitate the tracking of suspected cases and the detection of suspected cases. Determination of the trajectory of the case.

In a possible implementation manner, the identity information and body temperature of the target object can be superimposed on the first area in the visible light image or the third area in the infrared image, and displayed on the display 5 to visually observe and monitor The temperature information and identity information of each target object in the area, and the superimposed image can be stored.

In a possible implementation manner, the monitoring method can use visible light images and infrared images to perform temperature monitoring, which can reduce the hardware cost of the camera and can also reduce the pressure of data processing. It can also monitor whether each target object wears a mask. It can be used in fields such as monitoring of health and medical places or monitoring of epidemic prevention. The embodiments of the present disclosure do not limit the application field of the monitoring method.

Fig. 5 shows a block diagram of a monitoring system according to an embodiment of the present disclosure. As shown in Fig. 5, the system includes: an infrared image acquisition part 11, a visible light image acquisition part 12, and a processing part 13,

The processing part 13 is configured to:

Perform target area recognition on the visible light image of the monitoring area acquired by the visible light image acquisition part 12, and obtain the first area where the target object is located in the visible light image and the second area where the preset part of the target object is located in the visible light image;

Performing live detection on the target object according to the size information of the first area and the third area corresponding to the first area in the infrared image obtained by the infrared image obtaining section 11 to obtain a live detection result;

In the case that the result of the living body detection is a living body, perform identity recognition processing on the first area to obtain the identity information of the target object, and perform temperature control on the fourth area corresponding to the second area in the infrared image Detection processing to obtain temperature information of the target object;

In the case that the temperature information is greater than or equal to a preset temperature threshold, first warning information is generated according to the identity information of the target object.

In a possible implementation, the processing part 13 may include a processor (for example, the processor 3 in FIG. 4A), and the infrared image acquisition part 11 may include an infrared camera (for example, the infrared camera 1 in FIG. 4A) The visible light image acquisition part 12 may include a visible light camera (for example, the visible light camera 2 in FIG. 4A).

In a possible implementation manner, the processing part is further configured to:

Determine the distance between the target object and the image acquisition device that acquires the visible light image according to the size information of the first region;

Determine a living body detection strategy according to the distance;

Determining the position of the third area in the infrared image according to the position of the first area in the visible light image;

According to the living body detection strategy, the living body detection processing is performed on the third area in the infrared image to obtain the living body detection result.

In the case that the distance is greater than or equal to the first distance threshold, determining the living body detection strategy as a living body detection strategy based on human body shape; or

In a case where the distance is greater than or equal to the second distance threshold and less than the first distance threshold, determining the living body detection strategy as a living body detection strategy based on the head and neck shape; or

In a case where the distance is less than the second distance threshold, the living body detection strategy is determined as a living body detection strategy based on facial morphology.

Performing morphological detection processing on the third region according to the living body detection strategy to obtain a morphological detection result;

In the case that the shape detection result is a living body shape, perform isothermal analysis processing on the face area in the third area to obtain an isothermal analysis result;

Determining the first weight of the morphological detection result and the second weight of the isothermal analysis result according to the living body detection strategy;

According to the first weight, the second weight, the morphological detection result, and the isothermal analysis result, the living body detection result is determined.

In the case that the living body detection result is a living body, detecting whether a preset target is included in the first area, and obtaining the first detection result, wherein the preset target includes an object that occludes a partial area of the face,

The processing part is further configured to:

In the case that the living body detection result is a living body, according to the first detection result, an identity recognition process is performed on the first area to obtain the identity information of the target object.

Performing detection processing on the facial region in the first region, and determining a feature missing result of the facial region;

In the case that the feature missing result is a preset feature missing, detecting whether a preset target is included in the face area, and obtaining the first detection result.

In the case where the first detection result is that the preset target does not exist, perform first identity recognition processing on the face area in the first area to obtain the identity information of the target object; or

In the case where the first detection result is that the preset target exists, a second identity recognition process is performed on the face area in the first area to obtain the identity information of the target object, wherein the second The weight of the feature of the unoccluded area of the face in the identity recognition processing is greater than the weight of the feature of the corresponding area in the first identity recognition processing.

The position information of the first area or the third area, the temperature information of the target object, and the identity information of the target object are superimposed with the visible light image and/or infrared image to obtain a detection image.

Acquiring gender information of the target object;

Determine the preset temperature threshold according to the gender information.

In a case where the first detection result is that the preset target does not exist, second early warning information is generated.

It can be understood that the various method embodiments mentioned in the embodiments of the present disclosure can be combined with each other to form a combined embodiment without violating the principle and logic. Due to space limitations, the embodiments of the present disclosure will not be repeated.

In addition, the embodiments of the present disclosure also provide monitoring systems, electronic devices, computer-readable storage media, and programs. All of the above can be used to implement any monitoring method provided by the embodiments of the present disclosure. For the corresponding technical solutions and descriptions, refer to the method section. The corresponding records will not be repeated here.

Those skilled in the art can understand that in the above-mentioned method of implementation, the writing order of the steps does not mean a strict execution order but constitutes any limitation on the implementation process. The execution order of each step should be based on its function and possible internal logic. Sure.

In some embodiments, the functions or parts included in the apparatus provided in the embodiments of the present disclosure may be configured to execute the methods described in the above method embodiments. For implementation, refer to the description of the above method embodiments. For brevity, here No longer

The embodiment of the present disclosure also proposes a computer-readable storage medium on which computer program instructions are stored, and the above method is implemented when the computer program instructions are executed by a processor. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

An embodiment of the present disclosure also provides an electronic device, including: a processor; a memory configured to store executable instructions of the processor; wherein the processor is configured as the aforementioned method.

The embodiment of the present disclosure also provides a computer program product, including computer readable code, when the computer readable code runs on the device, the processor in the device executes instructions for implementing the monitoring method provided by any of the above embodiments .

The embodiments of the present disclosure also provide another computer program product for storing computer-readable instructions, which when executed, cause the computer to perform the operation of the monitoring method provided in any of the foregoing embodiments.

The electronic device can be provided as a terminal, server or other form of device.

Fig. 6 is a block diagram showing an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and other terminals.

6, the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, and an input/output (Input/Output, I/O) interface 812 , The sensor component 814, and the communication component 816.

The processing component 802 generally controls the overall operations of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the foregoing method. In addition, the processing component 802 may include one or more sub-parts to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia sub-part to facilitate the interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations in the electronic device 800. Examples of these data include instructions for any application or method to operate on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable and Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic Disk or Optical Disk.

The power supply component 806 provides power for various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with the generation, management, and distribution of power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (Liquid Crystal Display, LCD) and a touch panel (TP). In the case where the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, sliding, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure related to the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC). When the electronic device 800 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive external audio signals. The received audio signal may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker configured to output audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface part. The above-mentioned peripheral interface part may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: home button, volume button, start button, and lock button.

The sensor component 814 includes one or more sensors for providing the electronic device 800 with various aspects of state evaluation. For example, the sensor component 814 can detect the on/off status of the electronic device 800 and the relative positioning of the components. For example, the component is the display and the keypad of the electronic device 800. The sensor component 814 can also detect the electronic device 800 or the electronic device 800. The position of the component changes, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration/deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor component 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 may also include a light sensor, such as a (Complementary Metal Oxide Semiconductor) or CCD (Charge-coupled Device) image sensor, which is configured to be used in imaging applications. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access wireless networks based on communication standards, such as Wi-Fi, 2G (2-Generation wireless telephone technology, second-generation mobile communication technology) or 3G (3-Generation wireless telephone technology, third-generation mobile communication technology) ), or a combination of them. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) part to facilitate short-range communication. For example, the NFC part can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and others. Technology to achieve.

In an exemplary embodiment, the electronic device 800 may be used by one or more application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processor (Digital Signal Processing, DSP), and digital signal processing device (Digital Signal Processing Device). , DSPD), programmable logic device (programmable logic device, PLD), field programmable gate array (Field Programmable Gate Array, FPGA), controller, microcontroller, microprocessor or other electronic components to implement the above method.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as the memory 804 including computer program instructions, which can be executed by the processor 820 of the electronic device 800 to complete the foregoing method.

Fig. 7 is a block diagram showing an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Referring to FIG. 7, the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as application programs. The application program stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above-described methods.

The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to the network, and an input output (I/O) interface 1958 . The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as the memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to complete the foregoing method.

The embodiments of the present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for enabling a processor to implement various aspects of the embodiments of the present disclosure.

The computer-readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. Examples of computer-readable storage media (non-exhaustive list) include: portable computer disks, hard disks, random access memory (Random Access Memory, RAM), read-only memory (Read Only Memory, ROM), erasable programmable Read-only memory (Electrical Programmable Read Only Memory, EPROM or flash memory), static random access memory (Static Random Access Memory, SRAM), portable compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), digital multiple Function disks (Digital Video Disc, DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards on which instructions are stored or raised structures in the grooves, and any suitable combination of the foregoing. The computer-readable storage medium used here is not interpreted as the instantaneous signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses through fiber optic cables), or through wires Transmission of electrical signals.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing/processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network, and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device .

The computer program instructions used to perform the operations of the embodiments of the present disclosure may be assembly instructions, instruction set architecture (Industry Standard Architecture, ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or a combination of Or source code or object code written in any combination of multiple programming languages, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as "C" language or similar programming Language. Computer-readable program instructions can be executed entirely on the user's computer, partly on the user's computer, executed as a stand-alone software package, partly on the user's computer and partly executed on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network-including local area network (LAN) or wide area network (WAN)-or it can be connected to an external computer (such as Use an Internet service provider to connect via the Internet). In some embodiments, the electronic circuit is customized by using the state information of the computer-readable program instructions, such as programmable logic circuit, field programmable gate array (FPGA) or programmable logic array (PLA), The electronic circuit can execute computer-readable program instructions to implement various aspects of the embodiments of the present disclosure.

Here, various aspects of the embodiments of the present disclosure are described with reference to the flowcharts and/or block diagrams of the methods, apparatuses (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each block of the flowcharts and/or block diagrams, and combinations of blocks in the flowcharts and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine that makes these instructions when executed by the processor of the computer or other programmable data processing device , A device that implements the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams is produced. It is also possible to store these computer-readable program instructions in a computer-readable storage medium. These instructions make computers, programmable data processing apparatuses, and/or other devices work in a specific manner. Thus, the computer-readable medium storing the instructions includes An article of manufacture, which includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

It is also possible to load computer-readable program instructions on a computer, other programmable data processing device, or other equipment, so that a series of operation steps are executed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , So that the instructions executed on the computer, other programmable data processing apparatus, or other equipment realize the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions, and operations of the system, method, and computer program product according to multiple embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of an instruction, and the module, program segment, or part of an instruction contains one or more components for realizing the specified logical function. Executable instructions. In some alternative implementations, the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions Or it can be realized by a combination of dedicated hardware and computer instructions.

The computer program product can be implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and so on.

The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Without departing from the scope and spirit of the illustrated embodiments, many modifications and changes are obvious to those of ordinary skill in the art. The choice of terms used herein is intended to best explain the principles, practical applications, or technical improvements in the market of the various embodiments, or to enable other ordinary skilled in the art to understand the various embodiments disclosed herein.

Claims

A monitoring method including:

Performing target area recognition on the visible light image of the monitoring area, and obtaining the first area where the target object is located in the visible light image and the second area where the preset part of the target object is located in the visible light image;

Performing live detection on the target object according to the size information of the first area and the third area corresponding to the first area in the infrared image of the monitoring area to obtain a live detection result;

In the case that the living body detection result is a living body, perform identity recognition processing on the first area to obtain the identity information of the target object, and compare the fourth area corresponding to the second area in the infrared image Performing temperature detection processing to obtain temperature information of the target object;

In the case that the temperature information is greater than or equal to a preset temperature threshold, first warning information is generated according to the identity information of the target object.
The method according to claim 1, wherein said performing live detection on said target object according to size information of said first area and a third area corresponding to said first area in said infrared image, Obtain live test results, including:

Determine the distance between the target object and the image acquisition device that acquires the visible light image according to the size information of the first region;

Determine a living body detection strategy according to the distance;

Determining the position of the third area in the infrared image according to the position of the first area in the visible light image;

According to the living body detection strategy, the living body detection processing is performed on the third area in the infrared image to obtain the living body detection result.
The method according to claim 2, wherein the determining a living body detection strategy according to the distance includes one of the following:

In a case where the distance is greater than or equal to the first distance threshold, determining the living body detection strategy as a living body detection strategy based on human body shape;

In a case where the distance is greater than or equal to a second distance threshold and less than the first distance threshold, determining the living body detection strategy as a living body detection strategy based on head and neck morphology;

In a case where the distance is less than the second distance threshold, the living body detection strategy is determined as a living body detection strategy based on facial morphology.
The method according to claim 2, wherein said performing a living body detection process on said third area according to said living body detection strategy to obtain said living body detection result comprises:

Performing morphological detection processing on the third region according to the living body detection strategy to obtain a morphological detection result;

In the case that the shape detection result is a living body shape, perform isothermal analysis processing on the face area in the third area to obtain an isothermal analysis result;

Determining the first weight of the morphological detection result and the second weight of the isothermal analysis result according to the living body detection strategy;

According to the first weight, the second weight, the morphological detection result, and the isothermal analysis result, the living body detection result is determined.
The method according to claim 1, wherein the method further comprises:

In the case that the living body detection result is a living body, it is detected whether a preset target is included in the first area, and the first detection result is obtained, wherein the preset target includes an object that occludes a partial area of the face ,

The performing identity recognition processing on the first area to obtain the identity information of the target object includes:

According to the first detection result, perform identity recognition processing on the first area to obtain the identity information of the target object.
The method according to claim 5, wherein said detecting whether a preset target is included in said first area and obtaining said first detection result comprises:

Performing detection processing on the facial region in the first region, and determining a feature missing result of the facial region;

In the case that the feature missing result is a preset feature missing, detecting whether the preset target is included in the face area, and obtaining the first detection result.
The method according to claim 5, wherein said performing identity recognition processing on said first area according to said first detection result to obtain identity information of said target object comprises one of the following:

In the case where the first detection result is that the preset target does not exist, perform first identity recognition processing on the face area in the first area to obtain the identity information of the target object;

In the case where the first detection result is that the preset target exists, a second identity recognition process is performed on the face area in the first area to obtain the identity information of the target object, wherein the second The weight of the feature of the unoccluded area of the face in the identity recognition processing is greater than the weight of the feature of the corresponding area in the first identity recognition processing.
The method according to claim 5, wherein the method further comprises:

In a case where the first detection result is that the preset target does not exist, second early warning information is generated.
The method according to any one of claims 1 to 8, wherein the method further comprises:

The position information of the first area or the third area, the temperature information of the target object, and the identity information of the target object are superimposed with the visible light image and/or the infrared image to obtain detection image.
A monitoring system includes: a visible light image acquisition part, an infrared image acquisition part, and a processing part,

The processing part is configured as:

Performing target area recognition on the visible light image of the monitoring area acquired by the visible light image acquisition part, and obtain the first area where the target object is located in the visible light image and the second area where the preset part of the target object is located in the visible light image;

Performing live detection on the target object according to the size information of the first area and the third area corresponding to the first area in the infrared image obtained by the infrared image obtaining part to obtain a live detection result;

In the case that the living body detection result is a living body, perform identity recognition processing on the first area to obtain the identity information of the target object, and compare the fourth area corresponding to the second area in the infrared image Performing temperature detection processing to obtain temperature information of the target object;

In the case that the temperature information is greater than or equal to a preset temperature threshold, first warning information is generated according to the identity information of the target object.
The system according to claim 10, wherein the processing part is further configured to:

Determine the distance between the target object and the image acquisition device that acquires the visible light image according to the size information of the first region;

Determine a living body detection strategy according to the distance;

Determining the position of the third area in the infrared image according to the position of the first area in the visible light image;

According to the living body detection strategy, the living body detection processing is performed on the third area in the infrared image to obtain the living body detection result.
The system according to claim 11, wherein the processing part is further configured to:

In the case that the distance is greater than or equal to the first distance threshold, determining the living body detection strategy as a living body detection strategy based on human body shape; or

In a case where the distance is greater than or equal to the second distance threshold and less than the first distance threshold, determining the living body detection strategy as a living body detection strategy based on the head and neck shape; or

In a case where the distance is less than the second distance threshold, the living body detection strategy is determined as a living body detection strategy based on facial morphology.
The system according to claim 11, wherein the processing part is further configured to:

Performing morphological detection processing on the third region according to the living body detection strategy to obtain a morphological detection result;

In the case that the shape detection result is a living body shape, perform isothermal analysis processing on the face area in the third area to obtain an isothermal analysis result;

Determining the first weight of the morphological detection result and the second weight of the isothermal analysis result according to the living body detection strategy;

According to the first weight, the second weight, the morphological detection result, and the isothermal analysis result, the living body detection result is determined.
The system according to claim 10, wherein the processing part is further configured to:

In the case that the living body detection result is a living body, it is detected whether a preset target is included in the first area, and the first detection result is obtained, wherein the preset target includes an object that occludes a partial area of the face ,

The processing part is configured as:

According to the first detection result, perform identity recognition processing on the first area to obtain the identity information of the target object.
The system according to claim 14, wherein the processing part is further configured to:

Performing detection processing on the facial region in the first region, and determining a feature missing result of the facial region;

In the case that the feature missing result is a preset feature missing, detecting whether the preset target is included in the face area, and obtaining the first detection result.
The system according to claim 14, wherein the processing part is further configured to:

In the case where the first detection result is that the preset target does not exist, perform first identity recognition processing on the face area in the first area to obtain the identity information of the target object; or

In the case where the first detection result is that the preset target exists, a second identity recognition process is performed on the face area in the first area to obtain the identity information of the target object, wherein the second The weight of the feature of the unoccluded area of the face in the identity recognition processing is greater than the weight of the feature of the corresponding area in the first identity recognition processing.
The system according to any one of claims 10 to 16, wherein the processing part is further configured to:

The position information of the first area or the third area, the temperature information of the target object, and the identity information of the target object are superimposed with the visible light image and/or the infrared image to obtain detection image.
An electronic device including:

processor;

A memory configured to store executable instructions of the processor;

Wherein, the processor is configured to execute the method according to any one of claims 1-9.
A computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 9 is realized.
A computer program, comprising computer readable code, when the computer readable code runs in an electronic device, a processor in the electronic device executes the method for implementing any one of claims 1 to 9 .