JP2022526207A - Monitoring methods and systems, electronic devices and storage media - Google Patents

Abstract

The embodiments of the present application relate to monitoring methods and systems, electronic devices and storage media. In the above method, the target region is recognized for the visible light image to obtain the first region where the target target is located and the second region where the predetermined portion is located, the dimensions of the first region, and the infrared image. Based on the third region corresponding to the first region, the biological detection result is obtained by performing the biological detection on the target target, and when the biological detection result is a living body, the identification processing is performed on the first region. The identification information is obtained, and the temperature is detected for the fourth region corresponding to the second region in the infrared image to obtain the temperature information. When the temperature information is equal to or higher than a predetermined threshold value, the identification information is obtained. Includes generating first early warning information based on.

Description

(Mutual reference of related applications)
The present application claims priority based on the Chinese patent application with application number 20100177357.0 filed on 13 March 2020, the entire contents of which are incorporated herein by reference.

The present application relates to the field of computer vision technology, and relates to monitoring methods and systems, electronic devices and storage media.

Early detection and quarantine are the most primitive and extremely effective measures in the management and prevention of outbreaks of respiratory viral infections. Early detection is a medical diagnostic instrument that includes early detection of a suspicious person, which may include body temperature detection. Early quarantine is to reduce contact between the virus carrier and others and reduce the risk of cross-infection, which may include measures such as non-contact temperature detection, masking and the like.

In related techniques, body temperature can be detected by methods such as thermometers and thermographic images, but the body temperature detection process is cumbersome, difficult to adapt to places with large traffic flows, and cases where infection with fever is suspected. If detected, it is difficult to verify the identity of suspected cases.

The embodiments of the present application provide monitoring methods and systems, electronic devices and storage media.

According to one embodiment of the present application, a monitoring method is provided. In the method, target area recognition is performed on a visible light image in a monitoring area to obtain a first region in which the target target is located and a second region in which a predetermined portion of the target target is located in the visible light image. Based on the dimensional information of the first region and the third region corresponding to the first region in the infrared image of the monitoring region, the biological detection is performed on the target target and the biological detection result is obtained. When the biological detection result is a living body, the first region is subjected to identification processing to obtain the identification information of the target target, and the fourth region corresponding to the second region in the infrared image is obtained. The temperature detection process is performed to obtain the temperature information of the target target, and when the temperature information is equal to or higher than a predetermined temperature threshold, the first early warning information is generated based on the identification information of the target target. Including that.

According to the monitoring method of the embodiment of the present application, biological detection and temperature detection can be performed based on an infrared image, the body temperature of the target target in the image can be quickly detected, and the body temperature detection efficiency is improved. , Applicable to places with large traffic flow. In addition, by recognizing the identification information of the target target by the visible light image, it is possible to contribute to the determination of the identification information of the case suspected of being infected and improve the epidemic prevention efficiency.

In a possible embodiment, biometric detection is performed on the target target based on the dimensional information of the first region and the third region corresponding to the first region in the infrared image, and the biological detection result is obtained. To obtain, determine the distance between the target object and the image acquisition device that acquires the visible light image based on the dimensional information of the first region, and based on the distance, obtain a biological detection strategy. Determining, determining the position of the third region in the infrared image based on the position of the first region in the visible light image, and determining the position in the infrared image based on the biodetection strategy. The third region includes performing a biological detection process and obtaining the biological detection result.

With such a method, a biological detection strategy can be determined based on the dimensions of the first region, and biological detection can be performed based on the biological detection strategy. Isothermal analysis can be used to detect biological detection in a far-infrared image. The accuracy can be improved.

In a possible implementation, determining the biodetection strategy based on the distance means determining the biodetection strategy as a biodetection strategy based on the human body morphology when the distance is greater than or equal to the first distance threshold. When the distance is equal to or greater than the second distance threshold and is less than the first distance threshold, the biological detection strategy is determined as the biological detection strategy based on the head and neck morphology, and the distance is the distance. When it is less than the second distance threshold, the biodetection strategy includes determining the biodetection strategy as a biodetection strategy based on the face morphology.

In a possible implementation, the biodetection process is performed on the third region based on the biodetection strategy, and the biodetection result is obtained on the third region based on the biodetection strategy. The morphological detection process is performed to obtain a morphological detection result, and when the morphological detection result is a biological morphology, an isothermal analysis process is performed on the face region in the third region to obtain an isothermal analysis result. Based on the biological detection strategy, the first weight of the morphology detection result and the second weight of the isothermal analysis result are determined, and based on the first weight, the second weight, the morphology detection result, and the isothermal analysis result. The present invention includes determining the biological detection result.

In a possible embodiment, the method is to detect whether a predetermined target is included in the first region and obtain the first detection result when the biological detection result is a living body. The predetermined target further includes an article that shields a part of the face, and when the biological detection result is a living body, the first region is subjected to the identification processing, and the identification information of the target target is performed. Obtaining includes the case where the living body detection result is a living body, the identification processing is performed on the first region based on the first detection result, and the identification information of the target target is obtained.

By such a method, the identification method can be selected based on whether or not a predetermined target exists, and the accuracy of identification can be improved.

In a possible embodiment, detecting whether or not a predetermined target is included in the first region and obtaining the first detection result is performed by performing a detection process on the face region in the first region. To determine the feature defect result of the face region, and if the feature defect result is a predetermined feature defect, detect whether or not a predetermined target is included in the face region, and obtain the first detection result. And, including.

In a possible implementation, the first detection result is the predetermined target when the identification processing is performed on the first region based on the first detection result and the identification information of the target target is obtained. When is not present, the first identification process is performed on the face region in the first region to obtain the identification information of the target target, and the first detection result is that the predetermined target exists. In the case of doing so, the second identification processing is performed on the face region in the first region to obtain the identification information of the target target, and the face is not shielded in the second identification processing. The weight of the feature of the region is larger than the weight of the feature of the corresponding region in the first identification process, and includes one of the following.

In a possible embodiment, the method further comprises generating a second early warning information if the first detection result is the absence of the predetermined target.

In a possible embodiment, the method further comprises acquiring the gender information of the target object and determining the predetermined temperature threshold based on the gender information.

In a possible embodiment, the method superimposes the position information of the first region or the third region, the temperature information of the target target, and the identification information of the target target on the visible light image and / or the infrared image. And further include obtaining a detected image.

According to one embodiment of the present application, a monitoring system is provided. The monitoring system includes a visible light image acquisition portion, an infrared image acquisition portion, and a processing portion, and the processing portion targets the visible light image in the monitoring region acquired by the visible light image acquisition portion. Region recognition is performed to obtain a first region in which the target target is located in the visible light image and a second region in which a predetermined portion of the target target is located, and the second region is acquired by the dimensional information of the first region and the infrared image acquisition portion. Based on the third region corresponding to the first region in the obtained infrared image, biological detection is performed on the target target to obtain a biological detection result, and when the biological detection result is a living body, the first The region is subjected to identification processing to obtain the identification information of the target target, and the fourth region corresponding to the second region in the infrared image is subjected to temperature detection processing to obtain the temperature of the target target. When information is obtained and the temperature information is equal to or higher than a predetermined temperature threshold, the first early warning information is generated based on the identification information of the target target.

In a possible embodiment, the processing portion further determines the distance between the target object and the image acquisition device that acquires the visible light image based on the dimensional information of the first region, and is based on the distance. The biometric detection strategy is determined, the position of the third region in the infrared image is determined based on the position of the first region in the visible light image, and the infrared image is determined based on the biometric detection strategy. The third region in the above is subjected to a biological detection process, and the biological detection result is obtained.

In a possible embodiment, the processing portion further determines the biodetection strategy as a biodetection strategy based on human morphology when the distance is greater than or equal to the first distance threshold, or the distance is the second distance. If it is greater than or equal to the threshold and less than the first distance threshold, the biodetection strategy is determined as a biodetection strategy based on head and neck morphology, or if the distance is less than the second distance threshold, the biodetection strategy is determined. The biodetection strategy is configured to be determined as a biodetection strategy based on face morphology.

In a possible implementation, the processed portion further performs morphological detection processing on the third region based on the biological detection strategy to obtain a morphological detection result, and when the morphological detection result is a biological morphology, the above. An isothermal analysis process is performed on the face region in the third region to obtain an isothermal analysis result, and the first weight of the morphological detection result and the second weight of the isothermal analysis result are determined based on the biological detection strategy. It is configured to determine the biological detection result based on the first weight, the second weight, the morphological detection result and the isothermal analysis result.

In a possible embodiment, the processing portion is further configured to detect if a predetermined target is included in the first region and obtain the first detection result when the biological detection result is a living body. The predetermined target includes an article that shields a partial region of the face, and the processed portion further relates to the first region based on the first detection result when the biological detection result is a living body. It is configured to perform identification processing and obtain identification information of the target target.

In a possible implementation, the processed portion further performs detection processing on the face region in the first region to determine a feature defect result of the face region, and the feature defect result is a predetermined feature defect. , It is configured to detect whether or not a predetermined target is included in the face region and obtain the first detection result.

In a possible implementation, the processing portion further performs a first identification process on the face region in the first region when the first detection result is that the predetermined target does not exist. When the identification information of the target target is obtained or the first detection result is that the predetermined target exists, the face region in the first region is subjected to the second identification processing, and the target target is subjected to the second identification processing. The weight of the feature of the unobstructed region of the face in the second identity recognition process is larger than the weight of the feature of the corresponding region in the first identity recognition process.

In a possible embodiment, the processing portion further combines the position information of the first or third region, the temperature information of the target target, and the identification information of the target target with the visible light image and / or the infrared image. It is configured to superimpose and obtain a detected image.

In a possible implementation, the processing portion is further configured to acquire the gender information of the target and determine the predetermined temperature threshold based on the gender information.

In a possible implementation, the processing portion is further configured to generate a second early warning information if the first detection result is the absence of the predetermined target.

According to one embodiment of the present application, an electronic device is provided. The electronic device comprises a processor and a memory configured to store instructions that can be executed by the processor, the processor being configured to perform the monitoring method.

According to one embodiment of the present application, a computer-readable storage medium is provided. A computer program instruction is stored in the computer-readable storage medium, and the computer program instruction causes the processor to realize the monitoring method when executed by the processor.

According to one aspect of the present application, a computer program is provided. The computer program includes a computer-readable code, and when the computer-readable code is executed in the electronic device, the processor in the electronic device is made to execute the monitoring method.

It should be understood that the general description above and the detailed description described below are for illustration and illustration purposes only and are not intended to limit the embodiments of the present application.

Other features and embodiments of the present application will become apparent with reference to the detailed description of exemplary embodiments based on the drawings below.

It is a flowchart which shows the image processing method by the Example of this application. It is a schematic diagram which shows the biological detection by the Example of this application. It is a schematic diagram which shows the identification process by an Example of this application. It is a schematic diagram which shows the application of the monitoring method by the Example of this application. It is a schematic diagram which shows the application of the monitoring method by the Example of this application. It is a block diagram which shows the monitoring system by the Example of this application. It is a block diagram which shows the electronic device by an Example of this application. It is a block diagram which shows the electronic device by an Example of this application.

The drawings attached herein constitute a part of the present specification by drawing into the specification, show an embodiment conforming to the embodiment of the present application, and interpret the technical solution of the embodiment of the present application together with the specification. Used for.

Hereinafter, various exemplary embodiments, features and embodiments of the present application will be described in detail with reference to the drawings. The same reference numerals in the drawings indicate elements having the same or similar functions. The drawings show various embodiments of the embodiments, but the drawings are not necessarily in proportion to each other unless otherwise specified.

The term "exemplary" as used herein means "as an example, as an example, or for illustration purposes". Here, any embodiment described as "exemplary" should not necessarily be construed as more suitable or advantageous than other embodiments.

As used herein, the term "and / or" is used to describe the relationship of a related object and indicates that there are three types of relationships. For example, A and / or B represent three cases: that only A exists, that A and B exist at the same time, and that only B exists. Also, as used herein, the term "at least one" refers to any one of a plurality or any combination of at least two of the plurality. For example, including at least one of A, B, and C means containing any one or more elements selected from the set consisting of A, B, and C.

In addition, in order to better explain the present application, many specific details have been described in the following specific embodiments. Those skilled in the art should understand that this disclosure is similarly feasible, regardless of these specific details. In order to clarify the gist of the present invention, in some examples, methods, means, elements and circuits familiar to those skilled in the art will not be described in detail.

FIG. 1 is a flowchart showing an image processing method according to an embodiment of the present application. As shown in FIG. 1, the method includes:

In step S11, the target region is recognized for the visible light image of the monitoring region, and a first region in which the target target is located in the visible light image and a second region in which the predetermined portion of the target target is located are obtained.

In step S12, biological detection is performed on the target target based on the dimensional information of the first region and the third region corresponding to the first region in the infrared image of the monitoring region, and the biological detection result is obtained. To get.

In step S13, when the biological detection result is a living body, the first region is subjected to identification processing, the identification information of the target target is obtained, and the fourth region corresponding to the second region in the infrared image is obtained. A temperature detection process is performed on the region to obtain temperature information of the target target.

In step S14, when the temperature information is equal to or higher than a predetermined temperature threshold value, the first early warning information is generated based on the identification information of the target target.

According to the monitoring method of the embodiment of the present application, biological detection and temperature detection can be performed based on an infrared image, the body temperature of the target target in the image can be quickly detected, and the body temperature detection efficiency is improved. , Applicable to places with large traffic flow. In addition, by recognizing the identification information of the target target by the visible light image, it is possible to contribute to the determination of the identification information of the case suspected of being infected and improve the epidemic prevention efficiency.

In a possible implementation, the monitoring method may be performed by a terminal device and other processing devices. Here, the terminal device includes a user device (User Equipment: UE), a mobile device, a user terminal, a terminal, a cellular telephone, a cordless telephone, a personal digital assistant (PDA), a handheld device, a computing device, and an in-vehicle device. , It may be a wearable device or the like. The other processing device may be a server or a cloud server. In a possible embodiment, the monitoring method can be implemented by calling a computer-readable instruction stored in memory by a processor.

In a possible implementation, the monitoring method may be applied to front monitoring equipment. For example, the front monitoring device may be a device that integrates components such as a processor and a camera. The processor can control components such as cameras, acquire images in the surveillance area, and the processor can execute the surveillance method. In a possible embodiment, the monitoring method is in a server (eg, the processor performing the monitoring method is located on the server, the processor and components such as the camera are not packaged together, and are in the monitoring area and backend processor. It is also applicable to (distribute and configure a monitoring system). An image taken by a server or a camera in a monitoring area can be received, and the monitoring method can be executed on the image.

In a possible implementation, the processor may be a System on a Chip (SoC). The camera may include an infrared camera for acquiring an infrared image and a camera for acquiring a visible light image. In a possible embodiment, a special sensor such as a vanadium oxide uncooled infrared focal plane detector can capture the spatial optical signal. The sensor can be applied to a far-infrared camera to acquire life rays and monitor the temperature.

In a possible embodiment, the camera can capture video in the surveillance area. The video frame of the video is a visible light image and an infrared image. The infrared camera and the visible light camera may be provided in close proximity to each other. For example, the infrared camera and the visible light camera may be provided adjacent to each other or may be provided in parallel. Alternatively, the two cameras may be integrated. The embodiments of the present application do not limit the installation method. Therefore, in the images taken by the two cameras at the same time, the positions of the target objects are similar. That is, the positional deviation of the same target object in the visible light image and the corresponding infrared image is small, and the deviation may be corrected based on the positional relationship between the two cameras. For example, based on the position of the first region (that is, the region where the target target is located) in the visible light image, the position of the third region where the target target is located can be determined in the infrared image.

In a possible embodiment, the image taken by the camera may have poor image quality. For example, there are situations such as image blur, focal length error, and camera contamination or occlusion. First, the image quality can be detected for the image. For example, image quality detection can be performed on a visible light image. For example, the texture of a visible light image and the sharpness of a boundary can be detected. When the sharpness is equal to or higher than the sharpness threshold value, it is recognized that the image quality is high, and processing such as the next position detection can be performed. When the sharpness is less than the sharpness threshold, it is recognized that the image quality is low, and the visible light image and the infrared image corresponding to the visible light image can be deleted at the same time.

In a possible implementation, in step S11, the target region can be recognized with respect to the visible light image, and the first region in which the target target is located can be obtained. The first region may be a face region of the target target and / or a human body region of the target target. By the process, the position coordinates of the first region where the target target is located or the position coordinates of the detection frame including the first region can be obtained.

In a possible implementation, the temperature of a region such as the forehead can be measured during the temperature detection process. For example, the temperature of the forehead region can be determined from the pixel value of the infrared image, and the body temperature of the target can be obtained. Therefore, it is possible to detect a second region in which a predetermined portion (for example, a region such as a forehead) is located in a visible light image, and further determine a fourth region in which a predetermined portion is located in an infrared image. The correspondence between the pixel value of the infrared image and the temperature can be established in advance, and the body temperature of the target target can be determined based on the pixel value in the fourth region.

In a possible embodiment, the organism is a biological modality that is biologically active and is represented by the human body with signs of life. A prosthesis is a model similar to a living body (for example, a photograph, a mask, headgear, etc.), which is made by imitating the biological characteristics of a living body. Generally, in the biological detection process, biological detection is performed using a near-infrared image of the face. It is based primarily on the differences between the imaging features of the organism and the imaging features of the prosthesis, primarily under near-infrared light, such as feature differences such as optical flow, texture, and color. And the prosthesis. However, in the temperature monitoring process, the infrared image taken by the far-infrared camera is a far-infrared image (that is, an image based on a life ray), not a commonly used near-infrared image. Based on the dimensions of the first region where the target target is located, the biodetection strategy can be determined and the accuracy of biodetection can be improved.

In a possible embodiment, step S12 determines the distance between the target object and the image acquisition device that acquires the visible light image based on the dimensional information of the first region, and is based on the distance. Based on the determination of the biological detection strategy, the determination of the position of the third region in the infrared image based on the position of the first region in the visible light image, and the determination of the biological detection strategy. , The third region in the infrared image may be subjected to a biological detection process to obtain the biological detection result.

In a possible implementation, the first region is the region in the visible light image where the target object is located. For example, it may be a face area to be targeted. The dimensions of the region may negatively correlate with the distance between the target object and the image acquisition device (visible light camera) that acquires the visible light image. That is, the larger the size of the region, the smaller the distance. The smaller the size of the region, the larger the distance. Further, the living body is represented by an infrared image (for example, a far-infrared image based on a life ray) as a peculiar sign of life. The blood vessels of the human body extend to the body (for example, parts of the human body, face, neck and shoulders, etc.), but in response to the close position of the target object in the visible light image, only a part of the body is shown in the visible light image. Present. For example, only the face, head and neck, etc. are presented, but the facial features are clearer. The farther the distance, the smaller the facial features and the larger the human body features. Based on different characteristics, different biodetection strategies can be combined to improve the accuracy of biodetection and make it suitable for use at different distances.

It is possible to pre-establish the correspondence between the reference dimension and the reference distance in the image, and determine the distance between the target object and the camera based on the proportional relationship between the dimension of the area where the target target is located and the reference dimension and the reference distance. can.

FIG. 2 is a schematic diagram showing biological detection according to the embodiment of the present application. As shown in FIG. 2, the distance between the target target and the camera can be determined based on the dimensional information of the first region, and the biological detection strategy can be selected based on the distance. Based on the first region in the visible light image, the corresponding third region in the infrared image can be determined. For example, if the first region is the face region of the target target A in the visible light image (for example, the target target A is close to the camera and only the face region of the target target A can be photographed), the third region. Is the face region of the target target A in the infrared image. When the first region is the head and neck region of the target target B in the visible light image (for example, the face and shoulder neck region or the upper body region of the target target B can be photographed), the third region is the infrared region. It is the head and neck region of the target object B in the image. When the first region is the face region of the target target C in the visible light image (for example, the target target C is far from the camera and the human body region of the target target C can be photographed), the third region is infrared. It is a human body region of the target target C in the image.

In a possible implementation, determining a biodetection strategy based on the distance of the target object means that if the distance is greater than or equal to the first distance threshold, the biodetection strategy is based on the human morphology. When the distance is equal to or greater than the second distance threshold and is less than the first distance threshold, the biological detection strategy is determined as a biological detection strategy based on the head and neck morphology, and the distance is determined. Is less than the second distance threshold, the biodetection strategy includes determining the biodetection strategy as a face morphology-based biodetection strategy.

In a possible implementation, the first distance threshold and the second distance threshold may be determined by parameters such as the range of the camera detection area and the focal length. For example, when the distance between the target object and the camera is equal to or greater than the first distance threshold value, the screen captured by the camera may include the whole body or most of the body of the target object. Therefore, it is possible to select a biological detection strategy based on the human body morphology. When the distance between the target and the camera is less than the first distance threshold and greater than or equal to the second distance threshold, it is impossible for the screen captured by the camera to include the whole body of the target, but the target is targeted. Face and shoulder neck areas are included, and thus biodetection strategies based on head and neck morphology can be selected. When the distance between the target target and the camera is less than the second threshold value, the screen captured by the camera includes only the face area of the target target. Therefore, it is possible to select a biological detection strategy based on the face morphology.

In a possible embodiment, the position of the third region in the infrared image is determined based on the position of the first region in the visible light image. In the example, the visible light image may include one or more target objects. That is, it includes one or more first regions. The position of the third region in the infrared image can be determined based on the positional relationship between the infrared camera and the visible light camera and the position of the first region.

In a possible embodiment, the infrared camera is not used to obtain a near-infrared image for detecting image features, but is used to obtain a far-infrared image for detecting life rays. In a far-infrared image, the accuracy rate for directly detecting the third region where the target target is located may be low. In addition, when there are a plurality of target targets, if the third region in which the target target is located is directly detected in the infrared image, the correspondence between each third region and each first region in the visible light image is clarified. It is difficult to do. Therefore, the position of the third region corresponding to the first region can be acquired based on the position of the first region in the visible light image and the positional relationship between the two cameras. Similarly, the position of the second region in the visible light image can be used to obtain the position of the corresponding fourth region in the infrared image.

In a possible implementation, after determining a biodetection strategy, the selected biodetection strategy can be used to perform biodetection processing on a third region in an infrared image. To obtain the biological detection result by performing the biological detection process on the third region based on the biological detection strategy, the morphological detection process is performed on the third region based on the biological detection strategy. Obtaining a morphological detection result, and when the morphological detection result is a biological morphology, an isothermal analysis process is performed on the face region in the third region to obtain an isothermal analysis result, and based on the biological detection strategy. , The biological detection result based on the determination of the first weight of the morphological detection result and the second weight of the isothermal analysis result, and the first weight, the second weight, the morphological detection result and the isothermal analysis result. To determine and include.

In a possible implementation, morphology detection can be performed on the third region based on the biological detection strategy.

In a possible implementation, the third region may include a target face region and a human body region in a biological detection strategy based on human morphology. There are many features of the human body. It is possible to perform feature extraction processing by a method such as a neural network, acquire a human body feature amount, perform morphological detection processing, and determine whether or not the human body morphology of the target target is a biological morphology (for example, the posture of the target target). , Motion, shape, and other characteristics can be used to determine whether the target morphology is a biological morphology).

In a possible implementation, in a biodetection strategy based on head and neck morphology, the third region may include a target target face and shoulder neck region, or upper body region. It is possible to perform feature extraction processing by a method such as a neural network, acquire facial and shoulder-neck region feature quantities, perform morphological detection processing, and determine whether the head and neck morphology of the target target is a biological morphology (). For example, it can be determined whether or not the morphology of the target object is a biological morphology based on features such as the posture, movement, and shape of the target object).

In a possible implementation, the third region may include a target face region in a biodetection strategy based on face morphology. The amount of human body features is small, and the amount of facial features is large. It is possible to perform feature extraction processing by a method such as a neural network, acquire facial area feature quantities and perform morphological detection processing, and determine whether or not the facial morphology of the target target is a biological morphology (for example, of the target target). Whether or not the morphology of the target target is a biological morphology can be determined based on features such as facial expressions and textures).

In a possible implementation, the form detection result of the form detection process may be a score format result. For example, when the morphology detection score is equal to or higher than the first score threshold value, the morphology detection result is recognized as a biological morphology. The embodiments of the present application do not limit the format of the morphological detection result.

In a possible implementation, when the morphology detection result is a biological morphology, an isothermal analysis process is performed on the face region in the third region to obtain an isothermal analysis result.

In a possible implementation, isothermal analysis can determine whether the facial region is a living body. For example, the true face of the target has a uniform vascular distribution. When the target is wearing a prosthesis such as a mask or headgear, the isotherm is different from the isotherm of the true face because there is no blood vessel distribution in the prosthesis. Can be decided.

In a possible implementation, the isothermal analysis result may be a score format result. For example, when the score of the isothermal analysis is equal to or higher than the second score threshold value, the result of the isothermal analysis is recognized as a living body. The embodiments of the present application do not limit the format of the isothermal analysis result.

In a possible implementation, the features of the face region occupy different proportions in different biodetection strategies (eg, in a biodetection strategy based on human morphology, the third region includes the face region and the human body region, and the face region. In the biometric detection strategy based on the face morphology, only the face region is included in the third region, and the proportion occupied by the face region is large). Based on the biological detection strategy, the first weight of the morphological detection result and the second weight of the isothermal analysis result can be determined.

For example, in the biological detection strategy based on the human body morphology, since the proportion occupied by the face region is small, the second weight of the isothermal analysis result based on the face region may be small, and the first weight of the morphology detection result may be large. good. For example, in a biodetection strategy based on face morphology, the proportion occupied by the face region is large, so that the second weight of the isothermal analysis result based on the face region may be large, and the first weight of the morphology detection result is small. May be good. For example, in a biodetection strategy based on head and neck morphology, the second weight and the first weight are close or equal. The embodiments of the present application do not limit the values of the first weight and the second weight.

In a possible implementation, the morphological detection result and the isothermal analysis result can be weighted and added based on the first weight and the second weight, respectively, to obtain a biological detection result. In a possible embodiment, the biodetection result may be a score format result. If the biological detection result is equal to or higher than the biological score threshold, it is recognized that the target target is a biological body. If the biological detection result is less than the biological score threshold, it is recognized that the target is a prosthesis or a non-human body, and no further processing is required.

By such a method, a biological detection strategy can be determined based on the dimensions of the first region, and biological detection can be performed based on the biological detection strategy. Isothermal analysis can be used to determine the accuracy of biological detection in a far-infrared image. Can be improved.

In a possible implementation, in step S13, when the target target is a living body, an identification process is performed on the target target. In possible embodiments, it is generally necessary to wear protective equipment (eg, for masks, goggles, etc.) in medical facilities (eg, hospitals, operating rooms, etc.) or public places during outbreaks. However, after wearing, it shields the face area to some extent. Therefore, it is necessary to monitor whether or not the target target is wearing a predetermined item such as a mask, and determine the identification method based on the detection result.

FIG. 3 is a schematic diagram showing an identification process according to an embodiment of the present application. The method is to detect whether or not a predetermined target is included in the first region when the biological detection result is a living body, and obtain the first detection result. The predetermined target is a face. It further includes, including articles that shield a part of the area of. The first detection result is whether or not the target target is wearing an article such as a mask that can shield the face area. Further, the identity recognition can be performed based on the first detection result. Step S13 includes performing an identification process on the first region based on the first detection result when the biological detection result is a living body, and obtaining the identification information of the target target.

In a possible implementation, it is possible to first determine if a predetermined target (eg, an article such as a mask) is present on the face of the target target. To detect whether or not a predetermined target is included in the first region and obtain the first detection result is to perform a detection process on the face region in the first region and obtain a feature defect result of the face region. The present invention includes determining whether or not a predetermined target is included in the face region when the feature defect result is a predetermined feature defect, and obtaining the first detection result.

In a possible implementation, the neural network can perform detection processing on the face region in the first region. For example, a convolutional neural network can be used to acquire a feature map of the face region. When the target target is wearing an article (predetermined target) that can shield the face area such as a mask, there is a feature defect in the feature map of the face area. For example, if the target target is wearing a mask, the characteristics of the mouth and nose of the target target cannot be detected. That is, the mouth and nose features are lost. If the target is wearing sunglasses, eye features cannot be detected. That is, the eye features are lost.

In a possible implementation, if a given feature (eg, a snout feature or an eye feature, etc.) is missing, the target subject may be wearing an article such as a mask, sunglasses, etc. for further detection. Can also be done. For example, it is possible to monitor whether or not a predetermined target such as a mask is present on the face of the target target by a method such as a convolutional neural network. For example, it is possible to detect features such as the shape and texture of a mask, recognize an article such as a mask, and prevent the target object from shielding the face by hand or being shielded by another person.

In a possible embodiment, if a given feature (eg, a nose feature or an eye feature, etc.) is not missing (ie, complete with facial features), then there is no given target, such as a mask. For example, if the surveillance area is a medical facility such as a hospital or a public place during an outbreak, not wearing a mask can be fraudulent. The method further comprises generating a second early warning information if the first detection result is the absence of the predetermined target. Further, the alarm device or the display can output the second alarm information and remind the target to wear the mask. For example, the alarm device may include devices such as audio and alarm lights. This can output the second early warning information by a method such as voice or light. Alternatively, the display can display information that the target target is not wearing a mask. The embodiment of the present application does not limit the output method of the second early warning information.

In a possible implementation, the identification method can also be determined based on the first detection result (ie, whether the target target is wearing an article capable of shielding the face). The identification processing is performed on the first region based on the first detection result to obtain the identification information of the target target, that is, the first detection result does not have the predetermined target. In this case, when the first identification processing is performed on the face region in the first region to obtain the identification information of the target target, and the first detection result is that the predetermined target exists. The second identification processing is performed on the face region in the first region to obtain the identification information of the target target, and the weight of the feature of the unshielded region of the face in the second identification processing is , Is greater than the weight of the feature of the corresponding region in the first identity recognition process, and includes one of the following.

In a possible implementation, the identification process can be performed using a neural network. Here, the neural network used for the first identity recognition process is different from the neural network used for the second identity recognition process. In the example, when a part of the face area of the target target is shielded and only the unshielded area is used for the identification, the weight occupied by the unshielded area is large in the neural network training process. For example, the attention mechanism of the neural network is concentrated on the characteristics of the unshielded region, and the identification is recognized by using the characteristics of the unshielded region.

In a possible implementation, the neural network will perform identity recognition based on all features of the face region if any features of the face region are applicable to the identity recognition process if the target face region is not occluded. It can be performed.

For example, if the target is wearing a mask and the neural network can only acquire eye and eyebrow features (missing mouth and nose features), the neural network will in training the eye and eyebrows. The weight of the part feature can be added, and the identification can be performed based on the eye part and eyebrow part features. If the target target is not wearing a mask and the neural network can acquire all the features of the face region, the neural network can perform identity recognition based on all the features of the face region.

For example, the neural network determines the similarity between the eye and eyebrow features and the reference features in the database (eye and eyebrow reference features), and the status corresponding to the reference features whose similarity is greater than or equal to the similarity threshold. The information can be determined as the identity information of the target target. Similarly, the neural network determines the degree of similarity between the face feature and the reference feature (face reference feature) in the database, and sets the identification information corresponding to the reference feature whose similarity is equal to or higher than the similarity threshold as the target identification information. Can be decided.

For example, if the database does not have a reference feature whose similarity to the target is greater than or equal to the similarity threshold (that is, the target does not match any of the existing identification information in the database), then the feature of the target is databased. It is possible to add an identification identifier (for example, an identification number) to the target target. The embodiments of the present application do not limit the method of identification.

By such a method, it is possible to select an identification method based on the presence or absence of a predetermined target and improve the accuracy of identification recognition.

It is also possible to recognize the gender of the target in a possible implementation. In a possible implementation, there is a certain difference between the average body temperature of men and the average body temperature of women. Therefore, different predetermined temperature thresholds can be set depending on the gender. For example, the average body temperature of women is 0.3 ° C higher than that of men. In temperature monitoring, a predetermined temperature threshold can be set based on gender. For example, males have a temperature of 37 ° C and females have a temperature of 37.3 ° C. The method further includes acquiring the gender information of the target target and determining the predetermined temperature threshold value based on the gender information.

In a possible implementation, the gender information of the target target can be determined based on the facial features of the target target. For example, if the target is not wearing a mask, the gender of the target can be determined based on all the features of the target's face. When the target target wears a mask, the gender of the target target can be determined based on the eye and eyebrow characteristics of the target target. Alternatively, if the facial features or eye and eyebrow features of the target match the reference features in the database, the gender of the target is determined based on the gender recorded in the identity information corresponding to the reference features in the database. be able to. The embodiments of the present application do not limit the method of determining gender.

In a possible implementation, a predetermined temperature threshold can be determined based on gender information. For example, the average body temperature of women is higher than the average body temperature of men. Therefore, the predetermined temperature threshold for women may be higher than the predetermined temperature threshold for men. The embodiments of the present application do not limit the method for determining a predetermined temperature threshold.

In a possible embodiment, in step S13, the temperature is relative to a fourth region (corresponding to a second region in the visible light image where a predetermined portion is located) where a predetermined portion (eg, forehead) is located in the infrared image. It is also possible to perform a detection process and determine the temperature information (for example, body temperature) of the target target. For example, the pixel value of a pixel point in an infrared image can represent a color temperature value. The color temperature value can be used to determine the temperature of the target forehead area. The embodiments of the present application do not limit the determination method of temperature information.

In a possible implementation, in step S14, when the temperature information is equal to or higher than a predetermined temperature threshold value, the first early warning information is generated based on the identification information of the target target. For example, during an outbreak period, fever belongs to a suspected case of infection and the targeted subject with symptoms can be monitored intensively. According to the detection, when the body temperature of the target target is equal to or higher than a predetermined temperature threshold value, it is recognized that the target target has a fever symptom. The first early warning information can be generated based on the identification information of the target target. For example, in order to intensively monitor the target target, the identification information and the body temperature of the target target can be displayed on the display. Alternatively, the first early warning information is issued by a device such as audio, and the epidemic prevention person is presented to pay attention to the target target or to perform processing such as isolation for the target target. The embodiment of the present application does not limit the output method of the first early warning information.

In a possible embodiment, the method superimposes the position information of the first region or the third region, the temperature information of the target target, and the identification information of the target target on the visible light image and / or the infrared image. And further include obtaining a detected image.

In a possible embodiment, the display displays a visible light image and / or an infrared image and displays the identity information and / or temperature information of the target object at the position of the target object in the visible light image and / or the infrared image. can do. For example, the identification information and / or the temperature information of the target object can be superimposed on the first region where the target object is located in the visible light image. That is, the identification information and / or the temperature information of the target target is displayed in the first area. Similarly, the identification information and / or the temperature information of the target target can be displayed in the third region of the infrared image. The embodiments of the present application do not limit the display method.

According to the monitoring method of the embodiment of the present application, the biological detection strategy can be determined based on the dimensions of the first region, and the biological detection can be performed based on the biological detection strategy. Isothermal analysis can improve the accuracy of biological detection in far-infrared images, select an identity recognition method based on the presence or absence of a predetermined target, and improve the accuracy of identity recognition. Further, it is possible to detect the temperature of a predetermined part and quickly detect the body temperature of the target target in the detected image, improve the body temperature detection efficiency, and can be applied to a place where the traffic flow is large. In addition, by recognizing the identification information of the target target by the visible light image, it is possible to contribute to the determination of the identification information of the case suspected of being infected and improve the epidemic prevention efficiency.

4A and 4B are schematic views showing the application of the monitoring method according to the embodiment of the present application. As shown in FIG. 4A, the monitoring method is applicable to a system for monitoring a monitoring area. The monitoring system includes an infrared camera 1 that collects life rays and acquires an infrared image, a visible light camera 2 for acquiring a visible light image, and a processor 3 (for example, a system-on-chip SoC). You may prepare. The system transmits information such as a supplementary light unit 4 used in an environment with poor light irradiation conditions, a display 5 for displaying a visible light image, an infrared image and / or early warning information, early warning information, and temperature information. A communication unit 6 for outputting an early alarm information, an alarm unit 7 for outputting early alarm information, an interaction unit 8 for inputting an instruction, and the like may be further provided.

In a possible embodiment, as shown in FIG. 4B, the infrared camera 1 and the visible light camera 2 can capture video frames in the surveillance area. For example, a visible light image AF (AF1-AF25) and an infrared image BF (BF1-BF25) can be taken. In addition, position correction can be performed on the visible light image AF and the infrared image. That is, the position of the third region in the infrared image BF can be determined based on the position of the first region in the visible light image AF.

In a possible embodiment, the processor 3 can perform image quality detection on the visible light image AF. For example, it detects the texture of the visible light image, the sharpness of the boundary, and deletes the visible light image with low image quality and the corresponding infrared image. Further, if there is no target target in the visible light image, the visible light image and the corresponding infrared image are deleted. For example, if there is no human in the surveillance area, the visible light image and the corresponding infrared image can be deleted.

In a possible implementation, the processor 3 can perform position detection processing on the visible light image AF to obtain a first region in which the target target is located, and obtain a second region in which the forehead of the target target is located. You can also. Based on the positions of the first and second regions in the visible light image, the position of the third region in which the target target is located in the infrared image and the position of the fourth region in which the target target is located in the infrared image are determined. can do.

In a possible implementation, the processor 3 can determine the distance between the target and the camera based on the dimensions of the first region and determine the biodetection strategy based on the distance.

In a possible implementation, if the distance of the target target is greater than or equal to the first distance threshold, the biodetection strategy is determined as the biodetection strategy based on the human body morphology. The morphology detection can be performed on the third region in the infrared image, and the morphology detection score can be obtained. An isothermal analysis score can be obtained by performing an isothermal analysis on the face region in the infrared image. Further, based on the biological detection strategy based on the human body morphology, a large first weight is assigned to the morphological detection score, a small second weight is assigned to the isothermal analysis score, weighted addition is performed, and then the biological detection result is obtained. be able to.

In a possible implementation, if the distance of the target target is less than the first distance threshold and greater than or equal to the second distance threshold, the biodetection strategy is determined as the biodetection strategy based on the head and neck morphology. The morphology detection can be performed on the third region in the infrared image, and the morphology detection score can be obtained. An isothermal analysis score can be obtained by performing an isothermal analysis on the face region in the infrared image. Further, based on the biometric detection strategy based on the head and neck morphology, close weights are assigned to the morphological detection score and the isothermal analysis score, weighted addition is performed, and then the biometric detection result can be obtained.

In a possible implementation, if the distance of the target target is less than the second distance threshold, the biodetection strategy is determined as the biodetection strategy based on the face morphology. The morphology detection can be performed on the third region in the infrared image, and the morphology detection score can be obtained. An isothermal analysis score can be obtained by performing an isothermal analysis on the face region in the infrared image. Further, based on the biometric detection strategy based on the face morphology, a small first weight is assigned to the morphological detection score, a large second weight is assigned to the isothermal analysis score, weighted addition is performed, and then the biometric detection result is obtained. be able to.

In a possible embodiment, the processor 3 detects in the first region of the visible light image whether the target target is wearing a mask and, if the target target is not wearing a mask, provides second early warning information. The second early warning information can be displayed by the generation and the display 5, or the second early warning information can be reproduced by the warning unit 7.

In a possible implementation, the processor 3 can determine the identity recognition method based on whether the target object is wearing a mask. For example, when the target target does not wear a mask, the identification information of the target target can be determined by comparing all the facial features of the target target with the reference features in the database. When the target target wears a mask, the identification information of the target target can be determined by comparing the eye and eyebrow features of the target target with the reference features in the database. Further, the second early warning information may further include the identification information of the target target. For example, it is possible to display on the display 5 that XXX (first and last name) is not wearing a mask.

In a possible implementation, the processor 3 can recognize the gender of the target target. For example, the gender of the target target can be recognized based on the facial features of the target target, or the gender of the target target can be acquired based on the identification information of the target target. Further, a predetermined temperature threshold of the male target and a predetermined temperature threshold of the female target can be determined based on the gender of the target target. Alternatively, a predetermined temperature threshold value can be input by the interaction unit 8. The embodiment of the present application does not limit the setting method of the predetermined temperature threshold value.

In a possible implementation, the temperature of the fourth region (that is, the region where the forehead is located in the infrared image) can be monitored to obtain the body temperature of the target target. If the body temperature exceeds a predetermined temperature threshold, the processor 3 can generate the first early warning information. In addition, the first early warning information is displayed on the display 5. Alternatively, the alarm unit 7 reproduces the first early alarm information. Further, the identification information, the body temperature, and the position information of the camera of the target target having the fever symptom (that is, the appearance position of the target target having the fever symptom) are transmitted to the back-end database or the server by the communication unit 6. This facilitates follow-up for suspected cases and determination of activity trajectories for suspected cases.

In a possible embodiment, the identity information and body temperature of the target object can be superimposed on the first region in the visible light image or the third region in the infrared image and displayed on the display 5. This makes it possible to intuitively observe the temperature information and identity information of each target in the monitoring area and store the superimposed image.

In a possible embodiment, the monitoring method can monitor the temperature by using a visible light image and an infrared image, can reduce the hardware cost of the camera, and can also reduce the data processing load. It is also possible to monitor whether each target is wearing a mask. It can be applied to fields such as medical facility monitoring or epidemic prevention monitoring. The embodiments of the present application do not limit the application fields of the monitoring method.

FIG. 5 is a block diagram showing a monitoring system according to an embodiment of the present application. As shown in FIG. 5, the system includes an infrared image acquisition portion 11, a visible light image acquisition portion 12, and a processing portion 13.
The processing portion 13 is
Target area recognition is performed on the visible light image of the monitoring area acquired by the visible light image acquisition portion 12, and the first region in which the target target is located in the visible light image and the second region in which the predetermined portion of the target target is located. Get the area,
Based on the dimensional information of the first region and the third region corresponding to the first region in the infrared image acquired by the infrared image acquisition portion 11, the living body is detected for the target target. Get the detection result
When the biological detection result is a living body, the first region is subjected to identification processing to obtain the identification information of the target target, and the fourth region corresponding to the second region in the infrared image is obtained. , Temperature detection processing is performed to obtain temperature information of the target target.
When the temperature information is equal to or higher than a predetermined temperature threshold value, the first early warning information is generated based on the identification information of the target target.

In a possible implementation, the processing portion 13 may include a processor (eg, processor 3 in FIG. 4A) and the infrared image acquisition portion 11 may include an infrared camera (eg, infrared camera 1 in FIG. 4A). The visible light image acquisition portion 12 may include a visible light camera (for example, the visible light camera 2 in FIG. 4A).

In a possible implementation, the processing portion further
Based on the dimensional information of the first region, the distance between the target object and the image acquisition device that acquires the visible light image is determined.
Based on the distance, determine a biodetection strategy and
Based on the position of the first region in the visible light image, the position of the third region in the infrared image is determined.
Based on the biometric detection strategy, the biometric detection process is performed on the third region in the infrared image, and the biometric detection result is obtained.

In a possible implementation, the processing portion further
When the distance is equal to or greater than the first distance threshold, the biological detection strategy is determined as a biological detection strategy based on the human body morphology, or
If the distance is greater than or equal to the second distance threshold and less than the first distance threshold, the biodetection strategy is determined as a biodetection strategy based on head and neck morphology, or
When the distance is less than the second distance threshold, the biodetection strategy is configured to be determined as a biodetection strategy based on face morphology.

In a possible implementation, the processing portion further
A morphological detection process was performed on the third region based on the biological detection strategy, and a morphological detection result was obtained.
When the morphological detection result is a biological morphology, an isothermal analysis process is performed on the face region in the third region to obtain an isothermal analysis result.
Based on the biological detection strategy, the first weight of the morphological detection result and the second weight of the isothermal analysis result are determined.
It is configured to determine the biological detection result based on the first weight, the second weight, the morphological detection result, and the isothermal analysis result.

In a possible implementation, the processing portion further
When the biological detection result is a living body, it is configured to detect whether or not a predetermined target is included in the first region and obtain the first detection result, and the predetermined target is a partial region of the face. Including articles that shield
The processed portion is further
When the living body detection result is a living body, it is configured to perform the identification processing on the first region based on the first detection result and obtain the identification information of the target target.

In a possible implementation, the processing portion further
The face region in the first region is subjected to detection processing, and the feature defect result of the face region is determined.
When the feature defect result is a predetermined feature defect, it is configured to detect whether or not a predetermined target is included in the face region and obtain the first detection result.

In a possible implementation, the processing portion further
When the first detection result is that the predetermined target does not exist, the first identification process is performed on the face region in the first region to obtain the identification information of the target target, or
When the first detection result is that the predetermined target exists, the face region in the first region is configured to perform the second identification processing to obtain the identification information of the target target. The weight of the feature of the unshielded region of the face in the second identity recognition process is larger than the weight of the feature of the corresponding region in the first identity recognition process.

In a possible implementation, the processing portion further
It is configured to superimpose the position information of the first region or the third region, the temperature information of the target target, and the identification information of the target target on the visible light image and / or the infrared image to obtain a detected image. To.

In a possible implementation, the processing portion further
Acquire the gender information of the target target and
It is configured to determine the predetermined temperature threshold value based on the gender information.

In a possible implementation, the processing portion further
If the first detection result is that the predetermined target does not exist, it is configured to generate a second early warning information.

The embodiments of the above methods provided in the embodiments of the present application can be combined with each other as long as they do not deviate from the principle or logic, and the number of papers is limited. It should be understood that the examples do not explain one by one.

The embodiments of the present application further provide a monitoring system, an electronic device, a computer-readable storage medium, and a program. All of the above are for realizing any one of the monitoring methods provided in the present application. See the description related to the method for the corresponding technical solutions and explanations. Here, a detailed description will be omitted.

In the above method of the embodiment, the description order of each step does not mean that the specific execution order means a strict execution order, and the specific execution order of each step is its function and. Those skilled in the art should understand that it is determined by possible intrinsic logic.

In some embodiments, the functions and modules in the apparatus provided in the embodiments of the present application are used to carry out the method described in the embodiment of the above method, and the specific embodiment is the embodiment of the above method. Please refer to the explanation of. For the sake of brevity, detailed description is omitted here.

The embodiments of the present application further provide a computer-readable storage medium. A computer program instruction is stored in the computer-readable storage medium, and when the computer program instruction is executed by the processor, the processor realizes the above method. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

The embodiments of the present application further provide electronic devices. The electronic device comprises a processor and a memory configured to store instructions that can be executed by the processor, the processor being configured to perform the method.

The embodiments of the present application further provide computer program products. The computer program product includes a computer-readable code, and when the computer-readable code is executed in the device, the processor in the device executes an instruction for realizing the monitoring method provided in any one of the above embodiments. ..

The embodiments of the present application further provide another computer program product. The computer program product is configured to store a computer-readable instruction, and when the instruction is executed, causes the computer to perform an operation of the monitoring method provided in any one of the above embodiments.

The electronic device may be provided as a terminal, a server or other form of device.

FIG. 6 is a block diagram showing an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a terminal such as a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, or a personal digital assistant.

Referring to FIG. 6, the electronic device 800 includes processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input / output (I / O) interface 812, sensor component 814 and It may include one or more of the communication components 816.

The processing component 802 generally controls the overall operation of the electronic device 800. For example, it controls operations related to display, call call, data communication, camera operation and recording operation. The processing component 802 may include one or more processors 820 for executing instructions. Thereby, all or part of the steps of the above method are performed. Note that the processing component 802 may include one or more modules for interaction with other units. For example, the processing component 802 includes a multimedia module, which contributes to the interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to support operations in the electronic device 800 by storing various types of data. Examples of these data include instructions, contact data, phonebook data, messages, images, videos, etc. of any application or method that can be operated on the electronic device 800. Memory 804 is realized by any type of volatile or non-volatile storage device, or a combination thereof. For example, static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), electrically erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM). ), Magnetic memory, flash memory, magnetic or optical disk.

The power component 806 powers various units of the electronic device 800. The power component 806 may include a power management system, one or more power sources, and other units involved in power generation, management, and distribution for the electronic device 800.

The multimedia component 808 includes a screen for providing an output interface between the electronic device 800 and the user. In some embodiments, the screen includes a liquid crystal display (LCD) and a touch panel (Touch Panel: TP). When the screen includes a touch panel, it is realized as a touch panel and receives an input signal from the user. The touch panel comprises one or more touch sensors that sense touches, slides and gestures on the panel. The touch sensor can not only detect the boundary of the touch or slide motion, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 comprises a front camera and / or a rear camera. If the electronic device 800 is in an operating mode such as a shooting mode or a video mode, the front camera and / or the rear camera can receive multimedia data from the outside. Each front and rear camera may have a fixed optical lens system or focal and optical zoom capabilities.

The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 includes a microphone (MIC). If the electronic device 800 is in an operating mode such as a call mode, a recording mode and a voice recognition mode, the microphone is configured to receive an audio signal from the outside. The received audio signal can be further stored in memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further comprises a speaker configured to output an audio signal.

The I / O interface 812 provides an interface between the processing component 802 and the peripheral interface module. The peripheral interface module may be a keyboard, a click wheel, a button, or the like. These buttons include, but are not limited to, a home button, a volume button, a start button and a lock button.

The sensor component 814 comprises one or more sensors and is configured to perform various state assessments for the electronic device 800. For example, the sensor component 814 can detect the on / off state of the electronic device 800 and the relative positioning of the unit. For example, the unit is a display and a keypad of an electronic device 800. The sensor component 814 detects a change in the position of one unit in the electronic device 800 or the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature fluctuation of the electronic device 800. You can also do it. The sensor component 814 may include a proximity sensor and is configured to detect the presence of surrounding objects in the absence of any physical contact. The sensor component 814 may include an optical sensor such as a Complementary Metal Oxide Semiconductor or CCD (Charge Coupled Device) image sensor and is configured to be applied for imaging. Will be done. In some embodiments, the sensor component 814 may include an accelerometer, gyro sensor, magnetic sensor, pressure sensor or temperature sensor.

The communication component 816 is configured to contribute to wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 is a communication standard such as WiFi, 2G (2-Generation wireless telephone technology: 2nd generation mobile communication technology) or 3G (3-Generation wireless telephone technology: 3rd generation mobile communication technology), or a combination thereof. You can access the wireless network based on. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast-related information from an external broadcast channel management system via the broadcast channel. In an exemplary embodiment, the communication component 816 further comprises a Near Field Communication (NFC) module to facilitate short-range communication. For example, NFC modules include Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth® (registered trademark) (Blue). Trademark): Realized based on BT) technology and other technologies.

In an exemplary embodiment, the electronic device 800 is one or more application specific integrated circuits (ASICs), a digital signal processor (DSP), a digital signal processing device (Digital Signal Process). Implemented by a Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor or other electronic element to perform the above method. It may be configured as follows.

In an exemplary embodiment, further provided is a non-volatile computer readable storage medium, such as memory 804, which stores computer program instructions. The computer program instruction is executed by the processor 820 of the electronic device 800 to complete the method.

FIG. 7 is a block diagram showing an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Referring to FIG. 7, the electronic device 1900 comprises a processing component 1922. It further comprises one or more processors and a memory resource represented by memory 1932. The memory lease is for storing instructions executed by the processing component 1922, such as an application program. The application program stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. The processing component 1922 is configured to execute the instruction and execute the above method.

The electronic device 1900 comprises a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and input / output (I / O). O) Interface 1958 may be further provided. The electronic device 1900 may run an operating system stored in memory 1932, such as Windows® ServerTM, Mac OS XTM, Unix®, Linux®, FreeBSDTM or the like. can.

In an exemplary embodiment, further provided is a non-volatile computer readable storage medium such as memory 1932 containing computer program instructions. The computer program instruction is executed by the processing component 1922 of the electronic device 1900 to complete the method.

The present application may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium, in which the computer-readable program instructions for realizing each aspect of the present application are stored in the processor.

The computer-readable storage medium may be a tangible device capable of holding or storing instructions used in the instruction execution device. The computer-readable storage medium may be, for example, an electric storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination of the above, but is not limited thereto. More specific examples (non-exhaustive lists) of computer-readable storage media are portable computer disks, optical discs, random access memory (RAM), read-only memory (Read Only Memory: ROM), and erasable. Programmable Read-Only Memory (Electrical Program Read Only Memory: EPROM or Flash), Static Random Access Memory (SRAM), Portable Compact Disk Read-Only Memory (Compact Disk-Read Only Memory) Includes a multipurpose disc (Digital Video Disc: DVD), a memory stick, a flexible disc, a punch card in which instructions are stored, or a mechanical coding device such as a protrusion structure in a groove, and any suitable combination described above. The computer-readable storage medium used herein is a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (eg, an optical pulse passing through an optical fiber cable), or through an electric wire. It should not be construed as a temporary signal itself, such as an electrical signal being transmitted.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or networks such as the Internet, local area networks, wide area networks and / or wireless networks. It can be downloaded to an external computer or an external storage device via. The network may include copper cables for transmission, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. The network interface card or network interface in each computing / processing device receives computer-readable program instructions from the network, transfers the computer-readable program instructions, and stores them in a computer-readable storage medium in each computing / processing device.

The computer program instructions configured to perform the operations of the embodiments of the present application are assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcodes, firmware instructions, state setting data. , Or source code or target code written in one or more programming languages. The programming language includes object-oriented programming languages such as Smalltalk, C ++, and conventional procedural programming languages such as "C" programming languages or similar programming languages. Computer-readable program instructions may be executed entirely on the user computer, partially on the user computer, as a separate software package, or partially on the user computer. It may be executed partially on the remote computer, or it may be executed completely on the remote computer or the server. In the case of a remote computer, the remote computer connects to or connects to the user's computer through any type of network, including a local area network (LAN) or a wide area network (Wide Area Network: WAN). , Can connect to an external computer (eg, connect through the internet using an internet service provider). In some embodiments, the state information of computer-readable program instructions is used to customize electronic circuits such as programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (PLAs). The electronic circuit can realize each aspect of the embodiment of the present application by executing a computer-readable program instruction.

Here, each aspect of the present application will be described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products of the embodiments of the present application. Each block of the flowchart and / or the block diagram and each combination of the blocks in the flowchart and / or the block diagram can be realized by a computer-readable program instruction.

These computer-readable program instructions can be provided to the processor of a general purpose computer, dedicated computer or other programmable data processing device, thereby creating a device, and when these instructions are executed by the processor of the computer or other programmable data processing device, the flowchart. And / or created a device that realizes the functions / operations specified in one or more blocks in the block diagram. These computer-readable program instructions may be stored in a computer-readable storage medium. According to these instructions, computers, programmable data processing devices and / or other devices operate in a particular manner. Accordingly, a computer-readable storage medium in which instructions are stored comprises a product comprising instructions of each aspect that realizes a function / operation defined in one or more blocks in a flow chart and / or a block diagram.

Computer-readable program instructions may be loaded into a computer, other programmable data processing device, or other device. This causes a computer, other programmable data processing device, or other device to perform a series of steps of operation to create a process performed on the computer. Therefore, an instruction executed by a computer, another programmable data processing device, or another device realizes a function / operation specified in one or more blocks in a flowchart and / or a block diagram.

The flowcharts and book diagrams in the drawings exemplify the feasible architectures, functions and operations of the systems, methods and computer program products according to the plurality of embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent part of a module, program segment or instruction. Some of the modules, program segments or instructions include executable instructions to implement one or more predetermined logical functions. During implementation as some replacement, the functions shown in the blocks can occur in a different order than shown in the drawing. For example, two consecutive blocks can actually be executed essentially in parallel, and in some cases in reverse order, which is determined by the functions involved. Each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, is realized by a dedicated hardware-based system for performing a predetermined function or operation, or a dedicated hardware and a computer instruction. It can be realized by the combination of.

The computer program product may be realized by hardware, software or a combination thereof. In one embodiment of the embodiments of the present application, the computer program product may be represented as a computer storage medium. In another embodiment, the computer program product may be expressed as a software product such as, for example, a software development kit (SDK).

Although each embodiment of the present invention has been described above, the above description is exemplary, not exhaustive, and is not limited to the disclosed examples. Many modifications and changes are readily conceivable to those skilled in the art, provided that they do not deviate from the scope and intent of each of the embodiments described. The choice of terminology used herein is to best interpret the principles, practical applications, or technical improvements in the market of each example, or each of the examples disclosed herein by those of ordinary skill in the art. The purpose is to be able to understand.

It should be understood that the general description above and the detailed description described below are for illustration and illustration purposes only and are not intended to limit the embodiments of the present application.
For example, the present application provides the following items.
(Item 1)
It ’s a monitoring method.
Target area recognition is performed on the visible light image of the monitoring area to obtain a first area in which the target target is located and a second area in which the predetermined portion of the target target is located in the visible light image.
Based on the dimensional information of the first region and the third region corresponding to the first region in the infrared image of the monitoring region, the biological detection is performed on the target target and the biological detection result is obtained. ,
When the biological detection result is a living body, the first region is subjected to identification processing to obtain the identification information of the target target, and the fourth region corresponding to the second region in the infrared image is obtained. , Performing temperature detection processing to obtain temperature information of the target,
A monitoring method comprising generating first early warning information based on the identity information of the target target when the temperature information is equal to or higher than a predetermined temperature threshold value.
(Item 2)
Based on the dimensional information of the first region and the third region corresponding to the first region in the infrared image, it is possible to perform biological detection on the target target and obtain a biological detection result.
To determine the distance between the target target and the image acquisition device that acquires the visible light image based on the dimensional information of the first region.
Determining a biodetection strategy based on the distance,
Determining the position of the third region in the infrared image based on the position of the first region in the visible light image.
Based on the biometric detection strategy, the biometric detection process is performed on the third region of the infrared image to obtain the biometric detection result.
The method according to item 1.
(Item 3)
Determining the biodetection strategy based on the distance
When the distance is equal to or greater than the first distance threshold value, the biological detection strategy is determined as a biological detection strategy based on the human body morphology.
When the distance is equal to or greater than the second distance threshold value and less than the first distance threshold value, the biological detection strategy is determined as a biological detection strategy based on the head and neck morphology.
When the distance is less than the second distance threshold, the biodetection strategy is determined as a biodetection strategy based on the face morphology, and is characterized by including one of the following.
The method described in item 2.
(Item 4)
Based on the biometric detection strategy, the biometric detection process may be performed on the third region to obtain the biometric detection result.
Based on the biological detection strategy, the morphological detection process is performed on the third region to obtain the morphological detection result.
When the morphological detection result is a biological morphology, an isothermal analysis process is performed on the face region in the third region to obtain an isothermal analysis result.
Determining the first weight of the morphological detection result and the second weight of the isothermal analysis result based on the biological detection strategy.
It is characterized by including determining the biological detection result based on the first weight, the second weight, the morphological detection result, and the isothermal analysis result.
The method described in item 2.
(Item 5)
The method is
When the biological detection result is a living body, it is to detect whether or not a predetermined target is included in the first region and obtain the first detection result, and the predetermined target is a part of the face. Including, further including, including articles that shield the area,
Performing the identification process on the first area and obtaining the identification information of the target target is not possible.
Based on the first detection result, the first region is subjected to identification processing, and the identification information of the target target is obtained.
The method according to item 1.
(Item 6)
It is possible to detect whether or not a predetermined target is included in the first region and obtain the first detection result.
The detection process is performed on the face region in the first region to determine the feature defect result of the face region.
When the feature defect result is a predetermined feature defect, it is characterized by detecting whether or not a predetermined target is included in the face region and obtaining the first detection result.
The method according to item 5.
(Item 7)
Based on the first detection result, it is possible to perform the identification processing on the first region and obtain the identification information of the target target.
When the first detection result is that the predetermined target does not exist, the first identification process is performed on the face region in the first region to obtain the identification information of the target target.
When the first detection result is that the predetermined target exists, the second identification process is performed on the face region in the first region to obtain the identification information of the target target. The feature of the unobstructed region of the face in the second identity recognition process is larger than the weight of the feature of the corresponding region in the first identity recognition process, and one of them is included. do
The method according to item 5.
(Item 8)
The method is
The first detection result further comprises generating a second early warning information when the predetermined target does not exist.
The method according to item 5.
(Item 9)
The method is
Further including superimposing the position information of the first region or the third region, the temperature information of the target target, and the identification information of the target target on the visible light image and / or the infrared image to obtain a detected image. Characterized by
The method according to any one of items 1 to 8.
(Item 10)
It is a monitoring system, and the monitoring system includes a visible light image acquisition portion, an infrared image acquisition portion, and a processing portion.
The processed part is
Target area recognition is performed on the visible light image of the monitoring area acquired by the visible light image acquisition portion, and the first area in which the target target is located and the second area in which the predetermined portion of the target target is located in the visible light image. And
Based on the dimensional information of the first region and the third region corresponding to the first region in the infrared image acquired by the infrared image acquisition portion, the biological detection is performed on the target target and the biological detection is performed. Get results,
When the biological detection result is a living body, the first region is subjected to identification processing to obtain the identification information of the target target, and the fourth region corresponding to the second region in the infrared image is obtained. Then, the temperature detection process is performed to obtain the temperature information of the target target.
A monitoring system configured to generate first early warning information based on the identity information of the target target when the temperature information is equal to or higher than a predetermined temperature threshold.
(Item 11)
The processed portion is further
Based on the dimensional information of the first region, the distance between the target object and the image acquisition device that acquires the visible light image is determined.
Based on the distance, the biological detection strategy is determined, and the position of the third region in the infrared image is determined based on the position of the first region in the visible light image.
Based on the biometric detection strategy, the third region in the infrared image is subjected to biometric detection processing, and the biometric detection result is obtained.
The system according to item 10.
(Item 12)
The processed portion is further
When the distance is equal to or greater than the first distance threshold, the biological detection strategy is determined as a biological detection strategy based on the human body morphology, or
If the distance is greater than or equal to the second distance threshold and less than the first distance threshold, the biodetection strategy is determined as a biodetection strategy based on head and neck morphology, or
When the distance is less than the second distance threshold, the biodetection strategy is configured to be determined as a biodetection strategy based on the face morphology.
The system according to item 11.
(Item 13)
The processed portion is further
A morphological detection process was performed on the third region based on the biological detection strategy, and a morphological detection result was obtained.
When the morphological detection result is a biological morphology, an isothermal analysis process is performed on the face region in the third region to obtain an isothermal analysis result.
Based on the biological detection strategy, the first weight of the morphological detection result and the second weight of the isothermal analysis result are determined.
It is characterized in that the biological detection result is determined based on the first weight, the second weight, the morphological detection result, and the isothermal analysis result.
The system according to item 11.
(Item 14)
The processed portion is further
When the biological detection result is a living body, it is configured to detect whether or not a predetermined target is included in the first region and obtain the first detection result, and the predetermined target is a part of the face. Including articles that shield the area
The processed portion is further
Based on the first detection result, the first region is subjected to an identification process to obtain the identification information of the target target.
The system according to item 10.
(Item 15)
The processed portion is further
The face region in the first region is subjected to detection processing, and the feature defect result of the face region is determined.
When the feature defect result is a predetermined feature defect, it is characterized in that it is configured to detect whether or not a predetermined target is included in the face region and obtain the first detection result.
Item 14. The system according to item 14.
(Item 16)
The processed portion is further
When the first detection result is that the predetermined target does not exist, the first identification process is performed on the face region in the first region to obtain the identification information of the target target, or
When the first detection result is that the predetermined target exists, the face region in the first region is configured to perform the second identification processing to obtain the identification information of the target target. The weight of the feature of the unshielded region of the face in the second identity recognition process is larger than the weight of the feature of the corresponding region in the first identity recognition process.
Item 14. The system according to item 14.
(Item 17)
The processed portion is further
It is configured to superimpose the position information of the first region or the third region, the temperature information of the target target, and the identification information of the target target on the visible light image and / or the infrared image to obtain a detected image. Characterized by
The system according to any one of items 10 to 16.
(Item 18)
It ’s an electronic device,
With the processor
With memory configured to store instructions that can be executed by the processor,
The processor is an electronic device configured to perform the method according to any one of items 1-9.
(Item 19)
A computer-readable storage medium that stores computer program instructions for causing the processor to execute the method according to any one of items 1 to 9, when executed by a processor.
(Item 20)
A computer program comprising a computer-readable code for causing a processor in the electronic device to perform the method according to any one of items 1 to 9, when executed in the electronic device.

Claims (20)

It ’s a monitoring method.
Target area recognition is performed on the visible light image of the monitoring area to obtain a first area in which the target target is located and a second area in which the predetermined portion of the target target is located in the visible light image.
Based on the dimensional information of the first region and the third region corresponding to the first region in the infrared image of the monitoring region, the biological detection is performed on the target target and the biological detection result is obtained. ,
When the biological detection result is a living body, the first region is subjected to identification processing to obtain the identification information of the target target, and the fourth region corresponding to the second region in the infrared image is obtained. , Performing temperature detection processing to obtain temperature information of the target,
A monitoring method comprising generating first early warning information based on the identity information of the target target when the temperature information is equal to or higher than a predetermined temperature threshold value. Based on the dimensional information of the first region and the third region corresponding to the first region in the infrared image, it is possible to perform biological detection on the target target and obtain a biological detection result.
To determine the distance between the target target and the image acquisition device that acquires the visible light image based on the dimensional information of the first region.
Determining a biodetection strategy based on the distance,
Determining the position of the third region in the infrared image based on the position of the first region in the visible light image.
The method according to claim 1, further comprising performing a biological detection process on the third region of an infrared image based on the biological detection strategy to obtain the biological detection result. Determining the biodetection strategy based on the distance
When the distance is equal to or greater than the first distance threshold value, the biological detection strategy is determined as a biological detection strategy based on the human body morphology.
When the distance is equal to or greater than the second distance threshold value and less than the first distance threshold value, the biological detection strategy is determined as a biological detection strategy based on the head and neck morphology.
The second aspect of claim 2, wherein when the distance is less than the second distance threshold value, the biometric detection strategy is determined as a biometric detection strategy based on the face morphology, and one of the following is included. the method of. Based on the biometric detection strategy, the biometric detection process may be performed on the third region to obtain the biometric detection result.
Based on the biological detection strategy, the morphological detection process is performed on the third region to obtain the morphological detection result.
When the morphological detection result is a biological morphology, an isothermal analysis process is performed on the face region in the third region to obtain an isothermal analysis result.
Determining the first weight of the morphological detection result and the second weight of the isothermal analysis result based on the biological detection strategy.
The method according to claim 2, wherein the biological detection result is determined based on the first weight, the second weight, the morphological detection result, and the isothermal analysis result. The method is
When the biological detection result is a living body, it is to detect whether or not a predetermined target is included in the first region and obtain the first detection result, and the predetermined target is a part of the face. Including, further including, including articles that shield the area,
Performing the identification process on the first area and obtaining the identification information of the target target is not possible.
The method according to claim 1, wherein the first region is subjected to an identification process based on the first detection result, and the identification information of the target target is obtained. It is possible to detect whether or not a predetermined target is included in the first region and obtain the first detection result.
The detection process is performed on the face region in the first region to determine the feature defect result of the face region.
5. The feature claim 5 is characterized in that when the feature deficiency result is a predetermined feature deficiency, it is detected whether or not a predetermined target is included in the face region and the first detection result is obtained. The method described in. Based on the first detection result, it is possible to perform the identification processing on the first region and obtain the identification information of the target target.
When the first detection result is that the predetermined target does not exist, the first identification process is performed on the face region in the first region to obtain the identification information of the target target.
When the first detection result is that the predetermined target exists, the second identification process is performed on the face region in the first region to obtain the identification information of the target target. The feature of the unobstructed region of the face in the second identity recognition process is larger than the weight of the feature of the corresponding region in the first identity recognition process, and one of them is included. The method according to claim 5. The method is
The method according to claim 5, further comprising generating a second early warning information when the first detection result is the absence of the predetermined target. The method is
Further including superimposing the position information of the first region or the third region, the temperature information of the target target, and the identification information of the target target on the visible light image and / or the infrared image to obtain a detected image. The method according to any one of claims 1 to 8, wherein the method is characterized by the above. It is a monitoring system, and the monitoring system includes a visible light image acquisition portion, an infrared image acquisition portion, and a processing portion.
The processed part is
Target area recognition is performed on the visible light image of the monitoring area acquired by the visible light image acquisition portion, and the first area in which the target target is located and the second area in which the predetermined portion of the target target is located in the visible light image. And
Based on the dimensional information of the first region and the third region corresponding to the first region in the infrared image acquired by the infrared image acquisition portion, the biological detection is performed on the target target and the biological detection is performed. Get results,
When the biological detection result is a living body, the first region is subjected to identification processing to obtain the identification information of the target target, and the fourth region corresponding to the second region in the infrared image is obtained. Then, the temperature detection process is performed to obtain the temperature information of the target target.
A monitoring system configured to generate first early warning information based on the identity information of the target target when the temperature information is equal to or higher than a predetermined temperature threshold. The processed portion is further
Based on the dimensional information of the first region, the distance between the target object and the image acquisition device that acquires the visible light image is determined.
Based on the distance, the biological detection strategy is determined, and the position of the third region in the infrared image is determined based on the position of the first region in the visible light image.
The system according to claim 10, wherein a biometric detection process is performed on the third region of an infrared image based on the biometric detection strategy, and the biometric detection result is obtained. The processed portion is further
When the distance is equal to or greater than the first distance threshold, the biological detection strategy is determined as a biological detection strategy based on the human body morphology, or
If the distance is greater than or equal to the second distance threshold and less than the first distance threshold, the biodetection strategy is determined as a biodetection strategy based on head and neck morphology, or
The system according to claim 11, wherein when the distance is less than the second distance threshold value, the biometric detection strategy is configured to be determined as a biometric detection strategy based on the face morphology. The processed portion is further
A morphological detection process was performed on the third region based on the biological detection strategy, and a morphological detection result was obtained.
When the morphological detection result is a biological morphology, an isothermal analysis process is performed on the face region in the third region to obtain an isothermal analysis result.
Based on the biological detection strategy, the first weight of the morphological detection result and the second weight of the isothermal analysis result are determined.
11. The system according to claim 11, wherein the system is configured to determine the biological detection result based on the first weight, the second weight, the morphological detection result, and the isothermal analysis result. The processed portion is further
When the biological detection result is a living body, it is configured to detect whether or not a predetermined target is included in the first region and obtain the first detection result, and the predetermined target is a part of the face. Including articles that shield the area
The processed portion is further
The system according to claim 10, wherein an identification process is performed on the first region based on the first detection result, and the identification information of the target target is obtained. The processed portion is further
The face region in the first region is subjected to detection processing, and the feature defect result of the face region is determined.
14. Claim 14 is characterized in that when the feature defect result is a predetermined feature defect, it is configured to detect whether or not a predetermined target is included in the face region and obtain the first detection result. The system described in. The processed portion is further
When the first detection result is that the predetermined target does not exist, the first identification process is performed on the face region in the first region to obtain the identification information of the target target, or
When the first detection result is that the predetermined target exists, the face region in the first region is configured to perform the second identification processing to obtain the identification information of the target target. The system according to claim 14, wherein the weight of the feature of the unshielded region of the face in the second identity recognition process is larger than the weight of the feature of the corresponding region in the first identity recognition process. The processed portion is further
It is configured to superimpose the position information of the first region or the third region, the temperature information of the target target, and the identification information of the target target on the visible light image and / or the infrared image to obtain a detected image. The system according to any one of claims 10 to 16, wherein the system is characterized by the above. It ’s an electronic device,
With the processor
With memory configured to store instructions that can be executed by the processor,
The processor is an electronic device configured to perform the method according to any one of claims 1-9. A computer-readable storage medium that, when executed by a processor, stores a computer program instruction for causing the processor to perform the method according to any one of claims 1 to 9. A computer program comprising a computer-readable code for causing a processor in the electronic device to perform the method according to any one of claims 1 to 9, when executed in the electronic device.

Images