TWI777689B - Method of object identification and temperature measurement - Google Patents

Abstract

The present invention is a method of object identification and temperature measurement, comprising the following steps: receiving the original image transmitted by the imaging device, converting the original image into a grayscale image, performing blur processing on the grayscale image into blurred image, extracting the object from the blurred image into an object image, performing binarization processing on the object image to form a binarized image, performing expansion and erosion processing on the binarized image to form a recognition image, using a machine learning and training recognition model extracts the characteristic area to be selected in the recognition image, capturing the contour of the characteristic area, setting a contour frame around the contour of the characteristic area corresponding to the original image, detecting the object by a temperature sensor, making the sensing result of the temperature sensor around the contour frame of the original image to form a temperature sensing image.

Description

Object identification and body temperature measurement method

The present invention relates to a method for object identification, in particular to a method for object identification and body temperature measurement.

Affected by the new crown pneumonia epidemic recently, people need to measure their body temperature when entering and leaving public places such as MRT stations, department stores, office buildings, supermarkets, and convenience stores. In order to ensure or at least reduce the access of people who may be infected to public places.

Generally speaking, the method of measuring the body temperature of people entering and exiting in public places is usually to plan a designated entry and exit line, and measure the body temperature of people in the following ways at the entrance. Assign a special person to hold a forehead temperature measuring tool at the entrance to measure the body temperature of the people entering and exiting one by one. The second is also a place where there are few people entering and leaving. A temperature measuring machine is set up at the entrance and exit, and those who enter the public place go to the public place autonomously. Before the vertical temperature measuring machine is placed, place one of the human body surface, such as the forehead, palm or back of the hand, before the infrared sensor of the vertical temperature measuring machine to complete the temperature measurement. In public places where many people come and go, it is also placed at the entrance. The equipment used is usually an infrared thermal imaging face recognition device. It will detect the face position of all people passing through the detection area, and will automatically measure the temperature, and will mark the measured temperature around the face, which is currently the best and more efficient. A method that can measure the temperature of multiple people at the same time.

However, the price of infrared thermal imaging face recognition instruments is high. Some infrared thermal imaging face recognition instruments are only suitable for recognizing faces and cannot be used to identify other objects, and most infrared thermal imaging face recognition instruments cannot be used with other devices, and can only be used in a fixed position. However, these limit the application field and convenience of temperature detection.

In addition, some manufacturers have developed some different temperature sensing solutions, such as: Chinese Invention Patent Publication No. CN108760053A (name of invention: a forehead temperature detection method and device), the abstract mentions that "when a face is recognized, Obtain the infrared energy data radiated from the forehead area in the face; then, determine the user's forehead temperature based on the infrared energy data. The present application integrates the infrared temperature measurement function into the mobile terminal, so that the user can perform human Face recognition unlocking, or, in other scenes where faces can be captured, the infrared energy data radiated from the forehead area is collected, and then the user's forehead temperature is calculated. The solution that requires additional infrared temperature measurement equipment to measure temperature in the prior art Compared with this, it has the advantages of convenience and speed.” However, the previous patent case can only be applied to personal use of mobile terminals (such as smart mobile phones), and is not applicable to the situation where temperature monitoring for multiple people is required at the same time in public places.

Another example is Invention Patent Publication No. WO2020024553A1 (name of invention: human health detection system), the abstract of which mentions "a human health detection system, including: a face detection module (1), used to detect the detected area in the to-be-detected area. The temperature sensing module (2) is used to detect the face temperature of the detected person; the feature extraction module (3) is used to extract the facial features contained in the face image; the identity The confirmation module (4) is used to determine the identity information of the tested person; the judgment module (5) is used to determine whether the face temperature exceeds the normal body temperature range, and when the face temperature exceeds the normal body temperature range, the central processing is triggered The module (6); the central processing module (6) is used to send feedback data to the abnormal information feedback terminal corresponding to the identity information of the tested person. Detectors can be treated in time. Face recognition and temperature sensing are both existing technologies, so it can be inferred that the technical improvement of the previous patent case is that when abnormal body temperature is detected, it can be notified immediately, but it is not specified. How to identify the specific technical features of face and temperature sensing.

Therefore, if there is a face recognition device that does not require infrared thermal imaging, and can provide a specific and simple way to quickly recognize objects, instead of only recognizing faces, and it can be applied to static (place the detection device) Identifying and measuring body temperature in a fixed position) or dynamic (detecting instruments can identify and measure body temperature in a moving state) use environment will be an urgent problem to be solved.

In view of the problems of the prior art, an object of the present invention is to provide a method for identifying objects, further identifying the contour of the object, and when the contour is identified, the surface of the object can be detected to determine the temperature of the object, It can even track objects to achieve object recognition and body temperature measurement, and this method can be easily transplanted to various computing-capable devices.

According to an object of the present invention, it is to provide a method for object recognition and body temperature measurement, which includes the following steps: receiving an original image transmitted by an image capturing device, converting the original image into a grayscale image, and converting the grayscale image into a grayscale image. Convert the blurred image into a blurred image, extract the object from the blurred image to become an object image, perform binarization processing on the object image to form a binarized image, and then perform dilation and erosion processing on the binarized image to form an identification Image, use the recognition model trained by machine learning to extract the feature area to be selected for identifying the image, then capture the contour of the feature area, and set a contour frame around the contour of the feature area corresponding to the original image, and the temperature sensor will detect the object. Sensing, marking the sensing result of the temperature sensor around the outline frame of the original image to generate a temperature sensing image.

Wherein, the plurality of temperature sensing images cyclically generated according to the foregoing steps are converted into network transmission images, and transmitted to the remote web device for synchronous display by using the network transmission protocol.

Wherein, the image capturing device can be a camera module set by a network camera or a smart mobile device, and the original image is a color image. And the fuzzy processing is Gaussian fuzzy processing.

Wherein, the object can be a person, and the feature area can be a human face.

The step of extracting objects from the blurred image includes subtracting the pixel array of the current blurred image and the pixel array of the previous blurred image to obtain a pixel change array difference, and comparing whether the pixel change array difference is greater than the movement confidence value , when the difference value of the pixel change array is greater than the movement confidence value, it means that the changed part of the pixel array of the current blurred image is an object, and the changed part of the pixel array of the current blurred image can be extracted as the object map image, otherwise use the current blurred image as the object image.

The step of performing binarization processing on the object image to form a binary painting image includes comparing whether each pixel in the object image is greater than the grayscale standard value, and each pixel in the object image is greater than the grayscale standard value. The pixels are converted into white points, and the pixels in the object image that are smaller than the grayscale standard value are converted into black points, that is, a binarized image is formed.

Wherein, the binarized image is subjected to erosion and expansion processing to form the step of identifying the image, including the expansion processing of filling the broken and discontinuous pixels in the object image with surrounding pixels, so that the contours of the object image are connected to form expansion Image, dilated image and re-erosion processing eliminates over-filled pixels to form a recognition image, so that the recognition image can be as close to the real outline of the object as possible.

Among them, the identification model trained by machine learning is used to extract the feature area to be selected for the identification image,

Wherein, the step of capturing the contour of the feature area is to obtain the contour of the feature area by using the edge detection method, and further, to obtain the edge point by the zero-cross point of the second derivative of the luminance of the feature area, preferably in Marr (Marr)-Hildreth edge detection method.

According to the above, the present invention simplifies the information of the original image into a recognition image, and then uses the recognition image to obtain the position of the object in the original image and mark the outline frame, which greatly reduces the amount of calculation, so that this method can be used in Electronic devices with low computing power allow the invention to be easily extended to various places.

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Please refer to FIG. 1 , the present invention is an object identification and body temperature measurement method, which uses an electronic device with computing capability to perform the following steps: (S101) Receive the original image transmitted by the image capturing device (as shown in FIG. 2, the part of the outline frame and the temperature mark is removed); (S102) converting the original image into a grayscale image; (S103) blurring the grayscale image and converting it into a blurred image; (S104) extracting an object from the blurred image to become an object image; (S105) binarizing the object image to form a binarizing image; (S106) performing a closed operation process on the binarized image to form a recognition image; (S107) utilize the identification model of machine learning training to extract the feature area to be selected by the identification image; (S108) capturing the outline of the feature area; (S109) setting a contour frame around the contour of the feature region corresponding to the original image; (S110) The temperature sensor senses the object; ( S111 ) Mark the sensing result of the temperature sensor around the outline frame of the original image to generate a temperature sensing image (as shown in FIG. 2 ).

In the present invention, the plurality of temperature sensing images cyclically generated according to the foregoing steps are converted into network transmission images, and transmitted to the remote web device for synchronous display by using the network transmission protocol. Further, the network transmission image may be an image in Joint Photographic Experts Group (JPEG) format, the network transmission protocol may be Hyper Text Markup Language (English: Hyper Text Markup Language, HTML for short), and It can be edited with web editing software, such as the beaker kit (Flask), to upload each network transmission image to the designated storage space in the network, and can output the network transmission image synchronously, so, use The user can watch the monitoring and record the monitoring results at the remote end.

In the present invention, the image capturing device can be a camera module set in a network camera or a smart mobile device, and the original image is a color image or a grayscale image. If it is a grayscale image, the present invention can Starting directly from step S103, since the present invention uses color images or grayscale images, high-cost infrared thermal imaging cameras are not required, as long as various devices that can capture images can complete the identification, or even Provides tracking effects.

In the present invention, the electronic device with computing capability may be an embedded system with a single-chip microcomputer or a microprocessor unit, or even a higher-end desktop type with a central processing unit. Computers, servers...etc. Blur processing is Gaussian blur processing, which is used to simplify the complexity of image processing and eliminate noise. Furthermore, it is double-sided filtering blur processing, which is used to preserve the contour information in the grayscale image without blurring the smoothness of the contour. Fuzzy method. In addition, in the present invention, the object can be a person, and the characteristic area can be a human face.

In the present invention, referring to FIG. 3, the step of extracting objects from the blurred image includes the following steps: (S201) The pixel array of the current blurred image is subtracted from the pixel array of the previous blurred image to obtain a pixel change array difference; (S202) Compare whether the pixel change array difference is greater than the movement confidence value, and if so, perform the following steps (S203), otherwise, perform the step (S204); (S203) When the pixel change array difference value is greater than the movement confidence value, it means that the changed part in the pixel array of the current blurred image is an object, and the changed part in the pixel array of the current blurred image can be determined Extracting an image as an object, the purpose of this step is to reduce the amount of information processed in the subsequent steps, so that the present invention can use an electronic device with lower computing power to complete the subsequent steps; (S204) When the pixel change array difference is not greater than the movement confidence value, the current blurred image is used as the object image, and the subsequent steps are continued.

In the present invention, the step of performing a binarization process on the object image to form a binary painting image includes comparing whether each pixel in the object image is greater than a grayscale standard value (for example, the grayscale is 128), Each pixel in the object image that is greater than the grayscale standard value is converted into a white point (grayscale is 255), and each pixel in the object image that is smaller than the grayscale standard value is converted into a black point (grayscale is 0), That is, a binarized image is formed. For the selection of the grayscale standard value, all grayscales of the object image can be averaged, or the position of the largest peak or trough from the histogram distribution of all grayscales is also a suitable grayscale standard value. , but the present invention is not limited to this in actual implementation.

In the present invention, please refer to FIG. 4 , the steps of performing a closed operation on the binarized image to form a recognition image include: (S301) Dilation processing is performed on the broken and discontinuous pixels in the object image, in other words, the broken and discontinuous pixels in the object image are filled with the surrounding pixels, so that the contours of the object image are connected to form an expansion image; (S302) The dilated image is then subjected to erosion processing to eliminate over-filled pixels to form a recognition image, so that the recognition image can be as close to the real outline of the object image as possible. The reason for this step is that expanding the image is equivalent to expanding the object image. In order to eliminate the noise after the expansion, the identification image is similar in size to the object image, but the edge of the identification image is more similar. Easier to be identified by subsequent processing.

In the present invention, the identification model trained by machine learning is used to extract the characteristic area to be selected for the identification image, the object of the identification image is the human body, and the target to be selected is the face position of the human body, and the identification model is based on the identification image. Find the key positions of the face, such as the facial features of the face, and then use the key positions to convert the recognition image into the orientation aligned with the standard face as much as possible, and finally pass the face feature vector (such as a 128-dimensional vector), and finally Then, the location of the face of the object is identified by methods such as clustering, similarity detection, and classification. It is not recognized whether the face is a specific person, so the calculation amount of the face position of the present invention can be significantly reduced compared with the real face recognition which can distinguish which person is. However, when the present invention is actually implemented, it can still further identify whether the face is a specific person, and perform face tracking.

In the present invention, the step of capturing the contour of the feature area is to obtain the contour of the feature area by using the edge detection method, and further, to obtain the edge point by the zero-cross point of the second derivative of the luminance of the feature area, preferably It is based on the Marr-Hildreth edge detection method. However, when the present invention is actually implemented, it is not limited to this.

Furthermore, the present invention can be applied to a fixed position for identification, the image capturing device can be a network camera, the temperature sensor can be an infrared sensor, and the network camera and the infrared sensor are placed in a common location. At the entrance of the venue, through wired or wireless transmission, the original image and infrared sensing signal are transmitted to the remote electronic equipment for processing, or the image shooting equipment and temperature sensor can be installed on the drone, And it can track, move or point to obtain the original image and infrared sensing signal. Of course, if the image capture equipment and temperature sensor are installed on the drone, wireless transmission equipment must be used to convert the original image and infrared sensing signal. sent to a remote electronic device for processing.

Based on the above, when the present invention is applied to a fixed position for identification, the aforementioned steps ( S202 ) to ( S204 ) can be applied to identify a moving object. In the shooting area, the pixel array of the blurred image before and after the time will definitely change, and the changed part should be an object, so the changed part can be extracted as an image, and if the face is close to the image shooting device, usually People don't move much, so the face in front of the image capture device hardly moves, so the entire blurred image needs to be taken as the object image.

However, when the present invention is applied to the identification of the drone in a moving state, the aforementioned steps (S202) to (S204) need to be changed. The reason is that when the human body moves, the drone is also moving. The human body and the shooting area are constantly changing, or the human body is different, but the drone is moving. At this time, the human body does not change, but the shooting area is constantly changing. Therefore, in this case, step (S203) needs to be changed to, when the pixel If the difference value of the change array is greater than the movement confidence value, it means that the changed part of the pixel array of the current blurred image is the background, and the unchanged part of the pixel array of the current blurred image can be extracted as the object image , and if the pixel change array difference value is smaller than the movement confidence value, it means that both the human body and the shooting area are moving, the current blurred image is used as the object image, and the subsequent steps are continued.

To sum up, the present invention can use a common image capturing device plus a low-cost temperature sensor to provide the original image and infrared sensing value to the remote electronic device for calculation, and can transmit it to the network synchronously. The observer can watch directly, and the electronic equipment can be used with a low-cost single-chip microcomputer or embedded system of a microprocessor unit, which does not require a specific high unit price. The device can complete the purpose of object recognition and body temperature measurement.

The above detailed descriptions are for specific descriptions of feasible embodiments of the present invention, but the foregoing embodiments are not intended to limit the scope of the patent of the present invention. Any equivalent implementation or modification that does not depart from the technical spirit of the present invention shall be included in the within the scope of the patent in this case.

S101~S111: Step flow S201~S204: Step flow S301~S302: Step flow

FIG. 1 is a schematic flow chart of the present invention. FIG. 2 is a schematic diagram of a temperature sensing image of the present invention. FIG. 3 is a schematic flowchart of the blurred image extraction object of the present invention. FIG. 4 is a schematic flow chart of the present invention, which is a process flow diagram of performing a closed operation process on a binarized image to form a recognition image.

S101~S111: Step flow

Claims (7)

A method for object identification and body temperature measurement, using an electronic device with computing capability to perform the following steps: receiving an original image from an image capturing device; when the original image is a grayscale image, directly performing the following steps: The blurring process is converted into a blurred image; the object is extracted from the blurred image to become an object image; the object image is binarized to form a binarized image; and the binarized image is expanded and Erosion processing to form a recognition image; using a recognition model trained by machine learning to extract a feature area to be selected from the recognition image; capturing the contour of the feature area again; and surrounding the contour of the feature area corresponding to the original image setting an outline frame; using a temperature sensor to sense the object; marking the sensing result of the temperature sensor around the outline frame of the original image to generate a temperature sensing image; wherein when the If the original image is a color image, the color image is first converted into the grayscale image, and then the blurring process is performed; the plurality of temperature sensing images generated in a loop according to the above steps are further converted into network transmission images The image is transmitted to a remote web device for synchronous display using a network transmission protocol; wherein the step of extracting objects from the blurred image includes: subtracting the pixel array of the current blurred image from the pixel array of the previous blurred image Obtaining a pixel change array difference; comparing whether the pixel change array difference is greater than a movement confidence value; When the difference value of the pixel change array is greater than the movement confidence value, it means that the changed part in the pixel array of the current blurred image is an object, and the change in the pixel array of the current blurred image can be determined. The part of the image is extracted as the object image; when the pixel change array difference value is less than the movement confidence value, the current blurred image is used as the object image. The object recognition and body temperature measurement method according to claim 1, wherein the blurring processing is Gaussian blurring processing. The object identification and body temperature measurement method according to claim 1, wherein the object can be a human body, and the characteristic area can be a human face. The object identification and body temperature measurement method according to claim 1, wherein the step of performing a binarization process on the object image to form the binary image includes: comparing whether each pixel in the object image is larger than A grayscale standard value, each pixel in the object image greater than the grayscale standard value is converted into a white point, and each pixel in the object image smaller than the grayscale standard value is converted into a black point, that is, the two Valued image. The object identification and body temperature measurement method according to claim 1, wherein the step of performing a closed operation process on the binarized image to form an identification image includes: performing expansion processing on broken and discontinuous pixels in the object image, The broken and discontinuous pixels in the object image are filled with surrounding pixels, so that the contours of the object image are connected to form an expanded image; the expanded image is then eroded to eliminate the excessively filled pixels to form the identification image. The object identification and body temperature measurement method according to claim 1, wherein the identification model trained by machine learning is used to extract the characteristic region to be selected in the identification image, and further the identification image The object of the image is a human body, and the target to be selected is the face position of the human body. The recognition model finds the key position of the face from the identification image, and the key position is the facial features of the face. The recognition image is converted into a position aligned with a standard face, and finally the face position of the object is recognized by the recognition model through processes such as face feature vector, clustering, similarity detection, and classification. The object identification and body temperature measurement method according to claim 1, wherein the step of capturing the contour of the characteristic area is to obtain the contour of the characteristic area by an edge detection method.

Images