JP7492211B2 - Non-contact body temperature measuring device, method and program - Google Patents

Description

The present invention relates to a non-contact body temperature measurement device, method, and program, and in particular to a non-contact body temperature measurement device, method, and program for measuring the surface temperature of a measurement target without contact.

Conventionally, radiation thermometers for non-contact measurement of the surface temperature of a measurement target are known. One example of a radiation thermometer is a handheld non-contact thermometer that is held by the person measuring the temperature by placing it over the forehead of the person being measured. Such handheld non-contact thermometers can measure body temperature in a shorter time than contact thermometers that measure body temperature under the arm or in the mouth, and are therefore widely used to measure body temperature when entering a building, etc. However, with such handheld non-contact thermometers, one person is needed to measure each thermometer, and each person is measured one by one. In addition, manpower is required for managing the measured temperature data, etc.

Another example of a radiation thermometer is an infrared thermography camera, which analyzes the infrared rays emitted from the object to be measured and measures and visualizes the heat distribution in and around the object. Such a thermography camera is installed on a freestanding stand together with a monitor, and outputs to the monitor a thermal image of anyone who enters the measurement range. This makes it possible to monitor the body temperature of people when they enter or leave a building.

As a related technology using an infrared thermography camera (hereinafter also referred to as a thermal imaging camera), Patent Document 1 discloses a body temperature measurement system that measures the body temperature of a target animal without contact by measuring the temperature at a specific location on the body surface (the eye margin of a calf) that reflects the body temperature of the target animal.

JP 2017-062125 A

In the body temperature measurement system disclosed in Patent Document 1, a reference heat source surface is placed within the field of view of the thermographic camera to take into account the fact that the measurement accuracy of the thermographic camera decreases due to the influence of the environmental temperature, and the thermal image is corrected using the temperature of the reference heat source surface. However, there are the following problems.

In other words, in the above-mentioned body temperature measurement system, in order to accurately measure the temperature of the measurement target (the eye margin of the calf), the thermal image is taken while the distance from the thermographic camera to the measurement target is kept constant. However, in places where many people repeatedly enter and exit, such as building entrances, the position of the measurement target in the thermal image is not constant, which causes a decrease in the measurement accuracy of the thermographic camera.

In addition, the above-mentioned body temperature measurement system requires that a reference heat source surface be placed within the field of view of the thermographic camera, which limits the freedom of placement and is time-consuming, making it inefficient.

The present invention has been made in consideration of these circumstances, and aims to provide a non-contact body temperature measurement device, method, and program that can measure the surface temperature of a measurement target with high accuracy and efficiency.

In order to solve the above problems, the non-contact body temperature measuring device according to the first aspect of the present invention comprises a visible image camera that receives light in the wavelength range of visible light from the measurement target and captures a visible image, a thermal image camera that captures a thermal image that visualizes the amount of infrared light emitted from the measurement target, a facial temperature detection unit that detects a facial area from the visible image and detects the facial temperature from the facial area in the thermal image, and a facial temperature correction unit that identifies a background area outside the contour of the measurement target from the visible image, determines the temperature of the area in the thermal image corresponding to the background area as the environmental temperature of the space in which the measurement target exists, and corrects the facial temperature based on the environmental temperature.

In the non-contact body temperature measuring device according to the second aspect of the present invention, in the first aspect, the face temperature correction unit determines a representative value or median value of temperatures at multiple measurement points in the background area of the thermal image as the environmental temperature.

In the non-contact body temperature measuring device according to the third aspect of the present invention, in the second aspect, the face temperature correction unit calculates a representative value by excluding, from among the multiple measurement points, measurement points whose absolute difference from the average temperature of the measurement points is equal to or greater than a threshold value.

In the non-contact body temperature measuring device according to the fourth aspect of the present invention, in any of the first to third aspects, the face temperature correction unit identifies a background region from each of the multiple frames of visible images, and determines, as the environmental temperature, a temporal representative value of the temperature of the region corresponding to the background region in the multiple frames of thermal images that correspond to the multiple frames of visible images.

In the non-contact body temperature measuring device according to the fifth aspect of the present invention, in any of the first to fourth aspects, the face temperature correction unit calculates an estimate of the measured distance of the face area using a table showing the correspondence between the measured distance between the visible image camera and the face of the measurement subject and the size of the face area, and adds a correction value to the face temperature if the estimated value is equal to or greater than a threshold value.

In the non-contact body temperature measuring device according to the sixth aspect of the present invention, in any one of the first to fifth aspects, the facial temperature correction unit acquires information on at least one of the environmental temperature, humidity, and air pressure of the space in which the measurement subject is present, and corrects the facial temperature based on the information.

The non-contact body temperature measurement method according to the seventh aspect of the present invention includes the steps of receiving light in the visible light wavelength range from the measurement target using a visible image camera to capture a visible image, capturing a thermal image that visualizes the amount of infrared radiation emitted from the measurement target using a thermal image camera, detecting a face area from the visible image and detecting the face temperature from the face area in the thermal image, identifying a background area outside the contour of the measurement target from the visible image, determining the temperature of the area in the thermal image corresponding to the background area as the environmental temperature of the space in which the measurement target exists, and correcting the face temperature based on the environmental temperature.

The non-contact body temperature measurement program according to the eighth aspect of the present invention causes a computer to realize the following functions: receiving light in the visible light wavelength range from the measurement target and capturing a visible image using a visible image camera; capturing a thermal image that visualizes the amount of infrared light emitted from the measurement target using a thermal image camera; detecting a face area from the visible image and detecting the face temperature from the face area in the thermal image; identifying a background area outside the contour of the measurement target from the visible image, determining the temperature of the area in the thermal image corresponding to the background area as the environmental temperature of the space in which the measurement target exists, and correcting the face temperature based on the environmental temperature.

According to the present invention, the surface temperature of the measurement object can be measured with high accuracy and efficiency by performing temperature correction based on the environmental temperature obtained from the background area.

FIG. 1 is a block diagram showing a non-contact body temperature measuring device according to a first embodiment of the present invention. FIG. 2 is a block diagram showing an example in which the non-contact body temperature measuring device according to the first embodiment of the present invention is realized by a general-purpose operation terminal. FIG. 3 is a data flow showing the process flow of the non-contact body temperature measuring method according to the first embodiment of the present invention. FIG. 4 is a diagram showing an example of a visible image. FIG. 5 is a diagram showing an example of a thermal image. FIG. 6 is a diagram showing an example of an output to an input/output device. FIG. 7 is a block diagram showing a non-contact body temperature measuring device according to the second embodiment of the present invention. FIG. 8 is a flowchart showing a process flow when the non-contact body temperature measuring device according to the second embodiment of the present invention is applied to an entrance/exit management system. FIG. 9 is a flow chart showing the person detection process of FIG.

Below, an embodiment of the non-contact body temperature measuring device, method, and program according to the present invention will be described with reference to the attached drawings.

[First embodiment]
(Example of the configuration of a non-contact body temperature measuring device)
FIG. 1 is a block diagram showing a non-contact body temperature measuring device according to a first embodiment of the present invention.

As shown in FIG. 1, the non-contact body temperature measuring device 10 according to this embodiment includes a computing device 12, a power supply 14, a visible image camera 16, and a thermal image camera 18. The non-contact body temperature measuring device 10 is a device for non-contactly measuring the body temperature of a person to be measured from images captured using the visible image camera 16 and the thermal image camera 18.

The arithmetic device 12 includes a CPU (Central Processing Unit) that controls the operation of each part of the non-contact body temperature measuring device 10. The arithmetic device 12 is capable of sending and receiving control signals and data to and from each part of the non-contact body temperature measuring device 10. The arithmetic device 12 accepts instruction input from the user via the input/output device 20, and transmits control signals corresponding to this instruction input to each part of the non-contact body temperature measuring device 10 to control the operation of each part.

The calculation device 12 also includes an EEPROM (Electronically Erasable and Programmable Read Only Memory), a RAM (Random Access Memory) used as a working area for various calculations, and a VRAM (Video Random Access Memory) used as an area for temporarily storing image data output to the input/output device 20.

Furthermore, the computing device 12 includes a storage device including, for example, a magnetic disk such as a hard disk drive (HDD), or a flash memory such as a solid state drive (SSD) or an embedded multi media card (eMMC). This storage device stores data including control programs for various calculations and a non-contact body temperature measurement program.

The input/output device 20 includes a display device for displaying images and an input device for receiving instruction input from a user. Specifically, the input/output device 20 includes a liquid crystal display (LCD) monitor as the display device, and a touch panel provided on the surface of the LCD monitor as the input device. Note that the input device may include a keyboard and a pointing device (e.g., a mouse or a trackball) for character input, etc., instead of a touch panel.

The power supply 14 is a device that supplies power to each part of the non-contact body temperature measuring device 10. The power supply 14 can be, for example, an AC (Alternating Current) adapter that converts alternating current input from a commercial power source and outputs a predetermined direct current, or a battery (e.g., rechargeable) that can be attached to the non-contact body temperature measuring device 10.

The visible image camera 16 is a device that receives light in the visible light wavelength range (for example, wavelengths of about 360 nm to 800 nm) from an object in the imaging area and captures an image (e.g., a video). The visible image camera 16 is, for example, a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera.

The thermal imaging camera 18 is a device for visualizing (e.g., creating a video) the amount of infrared radiation emitted from an object in the imaging area. The thermal imaging camera 18 includes an infrared imaging element (quantum type or uncooled type) that converts the infrared radiation emitted from the object in the imaging area into an electrical signal to measure the amount of infrared radiation.

The visible image camera 16 and the thermal image camera 18 are positioned so that their imaging ranges (angle of view) are approximately the same. Here, the imaging ranges of the visible image camera 16 and the thermal image camera 18 do not need to be strictly the same. Even if the imaging ranges of the visible image camera 16 and the thermal image camera 18 are not strictly the same, they can be made to match by image processing (e.g., pattern matching, etc.).

The visible image camera 16 and the thermal image camera 18 are capable of capturing images of approximately the same angle of view in a substantially synchronized manner. Here, the image capturing timing of the visible image camera 16 and the thermal image camera 18 does not need to be perfectly synchronized, and a degree of deviation that does not affect the measurement accuracy is acceptable. In addition, frame images whose image capturing timing is synchronized with each other may be generated by interpolating frames of the visible image IMG1 or the thermal image IMG2. In the following explanation, for the sake of simplicity, it is assumed that the image capturing timing of the visible image camera 16 and the thermal image camera 18 is synchronized (see Figures 4 to 6).

The visible image camera 16 is capable of capturing images at a predetermined frame rate with a resolution sufficient to identify the person being measured (detect the person's face or eyes). The resolution of the visible image camera 16 is, for example, approximately 800 pixels x 600 pixels.

The thermal imaging camera 18 is capable of capturing images at a predetermined frame rate with a resolution that allows the measurement target person to be separated from background areas other than the measurement target (e.g., walls, etc.) based on the detection results of the measurement target from the visible image captured by the visible image camera 16. The resolution of the thermal imaging camera 18 is, for example, about one-tenth the resolution of the visible image camera 16, about 80 pixels x 60 pixels. In this embodiment, a thermal imaging camera 18 with a lower resolution than the visible image camera 16 is used, thereby making it possible to achieve low-cost, highly accurate measurements.

The resolution of the thermal imaging camera 18 is not limited to this, and may be the same as that of the visible image camera 16. In this case, further improvement in measurement accuracy can be achieved.

In this embodiment, a visible image and a thermal image of a person within the imaging range are captured using a visible image camera 16 and a thermal image camera 18, respectively. The computing device 12 processes the visible image to detect the face or pupils of the person within the imaging range. Then, based on the detection result of the person's face or pupils from the visible image, the computing device 12 separates the area of the person being measured contained in the thermal image from the background area, and calculates an estimate of the body temperature of the person being measured.

The sensor 22 is a device for measuring the air temperature (ambient temperature), humidity, or pressure (atmospheric pressure) in the location where the person being measured is present. The sensor 22 can be installed at any location within the imaging range of the visible image camera 16 and the thermal image camera 18. When the distance between the visible image camera 16 and the thermal image camera 18 and the measurement target is short, the sensor 22 may be provided in a module consisting of the visible image camera 16 and the thermal image camera 18.

The computing device 12 can correct the estimated body temperature of the person being measured based on the measurement results of the environmental temperature, humidity, or air pressure by the sensor 22.

The sensor 22 may be included in the non-contact body temperature measuring device 10, or may be configured to obtain the measurement results from a device other than the non-contact body temperature measuring device 10. For example, the computing device 12 may obtain information such as local weather data published by the Japan Meteorological Agency and published on the Internet, or weather data (temperature, humidity, air pressure, etc.) from an IoT (Internet of Things) weather sensor. In this case, the computing device 12 may obtain location information regarding the measurement range and use this location information to search for weather data on the Internet. The location information of the measurement range may be manually input to the computing device 12, or may be automatically obtained from information regarding the computing device's IP (Internet Protocol) address or wireless LAN (Local Area Network), etc. In addition, the sensor 22 is not essential for the non-contact body temperature measuring device 10 and may be omitted.

In addition, for example, when one or more non-contact body temperature measuring devices 10 are remotely operated by a management device (see the second embodiment), the measurement results by the sensor 22 corresponding to each non-contact body temperature measuring device 10 may be transmitted to the management device.

The calculation device 12 can be realized by, for example, a general-purpose operation terminal such as a personal computer, a workstation, or a tablet terminal. In this case, the input/output device 20 may be included in the non-contact body temperature measuring device 10.

(An example of a non-contact body temperature measuring device using a general-purpose operating terminal)
FIG. 2 is a block diagram showing an example in which the non-contact body temperature measuring device according to the first embodiment of the present invention is realized by a general-purpose operation terminal.

In the example shown in FIG. 2, the non-contact body temperature measuring device 10A includes an operation terminal 30 and a camera module 32.

The operation terminal 30 is a general-purpose operation terminal such as a personal computer, a workstation, or a tablet terminal, and functions as the calculation device 12, the power supply 14, and the input/output device 20 shown in FIG. 1. The operation terminal 30 has software installed therein for image processing, temperature (body temperature, facial temperature) detection, facial authentication, and result output, which will be described later.

The camera module 32 includes the visible image camera 16 and the thermal image camera 18 shown in FIG. 1. The camera module 32 is connected to the operation terminal 30 via an interface such as a USB (Universal Serial Bus), and is capable of receiving power from the operation terminal 30 and transmitting control signals and data to and from the operation terminal 30.

Processing such as image processing, temperature (body temperature, facial temperature) detection, facial recognition, and result output may be performed by transmitting data such as the visible image IMG1 and thermal image IMG2 from the camera module 32 to the operation terminal 30, and the temperature of the measurement target may be estimated by the operation terminal 30. Also, various processes such as temperature estimation may be performed in the camera module 32, and the results may be transmitted to the operation terminal 30 to display the results. For example, when the bandwidth of the transmission path between the operation terminal 30 and the camera module 32 is narrow, the amount of data transmitted between the operation terminal 30 and the camera module 32 can be reduced by performing various processes such as temperature estimation in the camera module 32.

(Non-contact body temperature measurement method)
Next, a non-contact body temperature measuring method according to a first embodiment of the present invention will be described with reference to Fig. 3 to Fig. 5. Fig. 3 is a data flow showing the process flow of the non-contact body temperature measuring method according to the first embodiment of the present invention. In this embodiment, the visible image camera 16 and the thermal image camera 18 synchronously acquire images (moving images) of the imaging range, and the calculation device 12 performs the process shown in Fig. 3 for each frame of the moving images.

First, the calculation device 12 acquires a visible image IMG1 and a thermal image IMG2 from the visible image camera 16 and the thermal image camera 18, respectively. Then, the calculation device 12 performs pre-processing on the visible image IMG1 and the thermal image IMG2, respectively (steps S10 and S20). Note that the pre-processing S20 on the thermal image IMG2 may be performed after steps S10 to S16, or may be performed in parallel.

In pre-processing of the visible image IMG1 (step S10), the calculation device 12 recognizes the general shape (outline) of the person being measured within the measurement range based on color information, etc. Then, the calculation device 12 adjusts the position of the person while comparing the visible image IMG1 with the thermal image IMG2. Specifically, the calculation device 12 performs processes such as enlarging/reducing and translating the person being measured. Here, the enlargement/reduction rate and the amount of movement may be set automatically by the calculation device 12 based on the results of image recognition, or may be set manually.

The calculation device 12 also performs image adjustments to facilitate detection of a human face (face recognition) from the visible image IMG1. Specifically, the calculation device 12 performs white balance adjustment, gamma correction, contrast adjustment, etc.

Furthermore, the calculation device 12 may also perform processing such as cutting out parts of the visible image IMG1 that are outside the imaging range of the thermal image IMG2.

On the other hand, in pre-processing of the thermal image IMG2 (step S20), a process is performed to match the size and position (angle of view) of the measurement target in the thermal image IMG2 with the measurement target in the visible image IMG1. Specifically, similar to the pre-processing of the visible image IMG1, the calculation device 12 performs processes such as enlargement/reduction processing and translation. Here, the enlargement/reduction rate and the amount of movement may be set automatically by the calculation device 12 based on the results of image recognition, or may be set manually.

When adjusting the image positions automatically in steps S10 and S20, the calculation device 12 may extract contour lines from each of the visible image IMG1 and the thermal image IMG2, and superimpose the two images based on the contour lines extracted from both images. Specifically, the calculation device 12 may superimpose the two images so that the correlation between the contours is maximized. In addition, a person's eyes or glasses may be detected from each of the visible image IMG1 and the thermal image IMG2, and the two images may be superimposed based on the detected eyes or glasses. Alternatively, the calculation device 12 may detect characteristic singular points, such as lighting devices, from each image, and superimpose the images based on the singular points.

Figure 4 shows an example of a visible image IMG1, and Figure 5 shows an example of a thermal image IMG2. In the thermal image IMG2 shown in Figure 5, the temperatures detected by the thermal imaging camera 18 are visualized by color coding, with the colors red (R), orange (O), yellow (Y), green (G), and blue (B) representing the highest temperatures. In Figure 5, red (R) is represented by vertical stripes, orange (O) by a diagonal grid, yellow (Y) by a grid, green (G) by diagonal stripes, and blue (B) by horizontal stripes. Note that the symbol CS in Figure 5 is a color scale that indicates the correspondence between colors and detected temperatures.

The visible image IMG1 shown in FIG. 4 shows one person P1 to be measured. The calculation device 12 detects the outline of the person P1 by processing the difference in color (temperature) between the person P1 and the background region BG1, or by pattern matching with the human body type.

On the other hand, the thermal image IMG2 shown in Figure 5 shows an area P2 corresponding to person P1. Within area P2, the face and ears (i.e., exposed skin) of area H2 corresponding to head H1 are hot (red), while the hair part of area H2 corresponding to head H1 and area B2 corresponding to torso B1 are cold (blue). Additionally, area G2 corresponding to glasses is colder (green) than the face.

In addition, in the hair and torso area B2, the areas closer to the face, which is the hotter area, are detected as hotter. That is, in the actual thermal image IMG2, the color of the hair and torso area B2 changes from blue to blue-green, green, and yellow-green as it gets closer to the face. In Figure 5, due to the limitations of the illustration, the areas of the hair and torso area B2 that are closer to the face are shown as green.

The computing device 12 detects the contour of the area P2 corresponding to the person P1 by performing processing such as the difference in color (temperature) between the area P2 in the thermal image IMG2 and the background area BG2, or pattern matching with the human body shape.

Next, the calculation device 12 detects the face of the person being measured from the visible image IMG1 as shown in FIG. 3 (step S12). In step S12, the calculation device 12 detects the pupil area G1 including the pupils or glasses of the face of the person being measured from the visible image IMG1 to determine the presence or absence of a face. Then, if a human face is not detected from the visible image IMG1 (No in step S14), the processing of the visible image IMG1 and the corresponding thermal image IMG2 is stopped, and steps S10 and S12 are repeated for the next acquired visible image IMG1.

On the other hand, if the face of person P1 is detected from the visible image IMG1 (Yes in step S14), the calculation device 12 extracts the face area F1 based on the detection result of the pupil area G1, and extracts the forehead area A1 used to measure the face temperature (body temperature) from the face area F1 in the visible image IMG1 (step S16).

Specifically, the arithmetic unit 12 detects the pupil region G1 (e.g., a rectangular region including both eyes or glasses) of the person P1 to be measured based on the color information in the visible image IMG1 or the result of pattern matching (step S12). Here, the pupil region can be detected, for example, by an object detection method using a Haar feature-based Cascade-type classifier (see, for example, Echizen Isao, "Privacy protection technology utilizing the difference in sensitivity between humans and devices," [online], December 12, 2012, National Institute of Informatics, Inter-University Research Institute Corporation, [searched September 29, 2020], Internet <URL: https://www.nii.ac.jp/userimg/press_20121212.pdf>).

Next, the calculation device 12 extracts an area in the visible image IMG1 obtained by enlarging the pupil area G1 vertically and horizontally at a predetermined magnification as the face area F1 in the visible image IMG1 (step S16).

4, if the vertical and horizontal sizes of the pupil region G1 in the visible image IMG1 are V _G and H _G , and the vertical and horizontal sizes of the face region F1 are V _F and H _F , respectively, the relationship between the pupil region G1 and the face region F1 is expressed by the following formula: In the formula, m and n are the vertical and horizontal magnifications used when obtaining the face region F1 from the pupil region G1, and the magnifications m and n can be determined experimentally or statistically.

_VF = m × _VG
HF = n x _HG
The pupil region G1 is generally a horizontally long rectangular region. Therefore, in order to approximate the shape of the face region F1 (for example, a square or rectangle), the relationship between the magnifications m and n is generally m>n. In one example, the vertical magnification m is 8 times, and the horizontal magnification n is 1.5 times.

As described above, in this embodiment, the face area F1 is extracted based on the pupil area G1, so it is possible to detect the human face even if the person P1 being measured is wearing a mask.

Next, the calculation device 12 extracts the forehead area A1 used for measuring the body temperature from the face area F1 in the visible image IMG1 (step S16). For example, the calculation device 12 estimates the area (A) above the upper side of the pupil area G1 and (B) a specific ratio below the top end of the face outline (the top end of the head H1) as the forehead area A1. Conditions for estimating the forehead area A1 may include (C) a skin-colored area, or (D) an area excluding hair based on color information. In addition, the forehead area A1 may be detected as a rectangular area satisfying the above conditions (A) and (B), or the rectangular area satisfying the above conditions (A) and (B) may be a deformed area (e.g., an elliptical area) according to the conditions (C) or (D). In addition, in the calculation of the face area described later, if pixels whose absolute difference from the average value is equal to or greater than a predetermined value are excluded, it is not necessary to strictly detect the forehead area A1, so the conditions (C) and (D) are not essential.

Next, the calculation device 12 identifies an area F2 corresponding to the face area F1 in the thermal image IMG2 based on the extraction result of the face area F1 in the visible image IMG1, and identifies an area A2 corresponding to the forehead area A1. Then, the calculation device 12 detects the temperature of the area A2 (hereinafter referred to as the face temperature) based on the temperature information contained in the thermal image IMG2 (step S22). Here, the calculation device 12 functions as a face temperature detection unit. In step S22, the face temperature is detected from the area A2 in the thermal image IMG2, which has the same position and size as the forehead area A1 in the visible image IMG1. Here, as the face temperature, the face temperature of the pixel with the highest temperature in the area A2 may be adopted, or a representative value (for example, the average value or the median value) of the temperatures at multiple measurement points in the area A2 may be adopted. In addition, when calculating the representative value of the face temperature, pixels in the area A2 whose absolute difference from the average value is equal to or greater than a predetermined value may be excluded.

In the above example, a representative value (spatial representative value) of multiple measurement points in thermal image IMG2 was described, but a temporal representative value of measurement points within area A2 detected in multiple frames of thermal image IMG2 may also be used.

Next, the calculation device 12 performs temperature correction of the face temperature detected in step S22 (step S24). Here, the calculation device 12 functions as a face temperature correction unit.

<A> Correction of face temperature based on correlation between forehead area A2 and armpit temperature In step S24, for example, the face temperature detected in step S22 is corrected based on the correlation between the forehead area A2 and armpit temperature. For correction based on the correlation between the forehead area A2 and armpit temperature, for example, the method described in Yoshiko Endo and 3 others, "Comparative study of armpit temperature and deep forehead temperature using electronic thermometer", [online], January 2009, Miyagi University, [searched on September 29, 2020], Internet <URL: https://myu.repo.nii.ac.jp/?action=repository_action_common_download&item_id=225&item_no=1&attribute_id=19&file_no=1> can be applied.

<B> Correction of face temperature based on environmental temperature (ambient temperature) Furthermore, the face temperature measured from the thermal image IMG2 may contain errors due to the environmental temperature around the measurement target and the measurement distance (the distance between the person P1 and the thermal image camera 18). For this reason, in step S24, it is possible to correct the errors due to the environmental temperature and the measurement distance.

Generally, the environmental temperature is estimated using a temperature sensor placed inside the housing of the device, but the temperature sensor placed inside the housing is affected by heat generated by electronic circuits or devices such as monitors placed inside the housing. Therefore, in this embodiment, the environmental temperature is estimated from the detected temperature of the background region BG2 of the thermal image IMG2.

Specifically, the calculation device 12 identifies the background area BG2 in the thermal image IMG2 based on the background area BG1 outside the contour of the person P1 to be measured, which is estimated from the visible image IMG1. The calculation device 12 then measures the temperature of the background area BG2 (e.g., a wall, etc.) at multiple measurement points, and adopts the representative value (e.g., the average value or the median value, etc.) as the environmental temperature of the space in which the person P1 is present. Here, when calculating the environmental temperature, the median value may be adopted as the representative value in order to eliminate the influence of windows, lighting fixtures, etc. reflected in the background area BG2. In addition, the average value or the median value after excluding measurement points whose temperatures are significantly different from other measurement points (e.g., the absolute difference value from the average value is equal to or greater than a threshold value) through statistical processing may be adopted as the representative value.

In the above example, the representative value (spatial representative value) of multiple measurement points in the thermal image IMG2 was described, but a temporal representative value at a measurement point in the background region BG2 detected in multiple frames of the thermal image IMG2 may also be used. When a temporal representative value at a measurement point in the background region BG2 is used, the visible image IMG1 and thermal image IMG2 may also be used to calculate the temporal representative value when no human face is detected in the visible image IMG1 (No in step S14).

For example, the method described in Tamura, Teruko, "Estimation of whole-body and sectional average skin temperature using thermography," [online], 1980, Home Economics Magazine, [searched September 29, 2020], Internet (URL: https://www.jstage.jst.go.jp/article/jhej1951/31/6/31_6_461/_pdf) can be applied to correct the facial temperature based on the environmental temperature.

<C> Correction of face temperature based on measurement distance Furthermore, regarding the distance between the thermal imaging camera 18 and the person being measured, some devices encourage measurement at a certain distance using a structure (e.g., an indicator sign, etc.) attached to the device, while others encourage the person being measured to stand at a specified position in front of the thermal imaging camera 18 and measure at a certain distance. Some devices also use a laser distance measuring device to measure the distance and perform correction. However, these devices are not suitable for measuring the temperature of a moving subject.

In this embodiment, the angle of view of the visible image camera 16 and the contour of the captured face can be used to estimate the distance of the measurement object and make corrections. Specifically, it has been experimentally shown that when measurements are taken with a camera of the same angle of view, (a) if the area of the face region is large, the distance will be closer, and (b) if the area of the measurement object is small (the distance is farther), the measured temperature value of the measurement object will be lower. With regard to (b), the error in the measured temperature value of the measurement object can be 1.0°C or more.

Therefore, the calculation device 12 determines that the distance is close if the area of the face area F1 is relatively large, and determines that the distance is far if the area of the face area F1 is relatively small, and calculates the facial temperature detection result according to a predetermined correction formula.

Specifically, a table showing the correspondence between the measured distance between the visible image camera 16 and the face of the person being measured and the size of the face area F1 (e.g., area, vertical or horizontal length, etc.) is stored in advance in the storage device of the calculation device 12. Then, the calculation device 12 obtains the size of the face area F1 from the visible image IMG1 and calculates an estimate of the measured distance using this table.

Next, when the estimated value of the measurement distance is equal to or greater than the threshold value, the calculation device 12 adds a correction value to the face temperature obtained from the thermal image IMG2. Here, multiple threshold values for the measurement distance may be set so that the longer the measurement distance, the larger the correction value becomes.

Also, if the measurement distance is equal to or greater than a predetermined threshold, the face area F1 may be excluded from the measurement of the face temperature. In this case, the measurement distance may be estimated in step S14, and if the measurement distance is equal to or greater than a predetermined threshold, it may be determined that there is no face.

The distance between the visible image camera 16 and the face of the person being measured may be determined using the autofocus function of the visible image camera 16, or may be measured directly using a distance sensor.

<D> Correction of facial temperature based on the measurement results by sensor 22 In addition, in step S24, temperature correction may be performed based on the measurement results by sensor 22 (e.g., the air temperature (ambient temperature), humidity or pressure (atmospheric pressure) at the location where the person being measured is present, etc.).

The facial temperature data corrected as described above is stored as facial temperature data D1 in the storage device of the computing device 12 together with the visible image IMG1 and the thermal image IMG2.

In addition, when the calculation device 12 outputs the measurement results to the input/output device 20, it may create a composite image IMG3 (step S30). In the composite image IMG3, for example, the visible image IMG1 and the thermal image IMG2 may be displayed side by side or may be superimposed. When creating the composite image IMG3, for example, image processing is performed so that the resolution or number of pixels of the visible image IMG1 and the thermal image IMG2 are the same. If the resolution of the thermal image IMG2 is lower than that of the visible image IMG1, an interpolation process may be performed on the thermal image IMG2. For example, if the resolution of the visible image IMG1 is 800 pixels x 600 pixels and the resolution of the thermal image IMG2 is 80 pixels x 60 pixels, the resolution of the thermal image IMG2 may be increased by 10 times by the interpolation process. Furthermore, if the resolution of the output display device is low, the resolution may be reduced by performing a thinning process on the visible image IMG1, or both the thinning process on the visible image IMG1 and the interpolation process on the thermal image IMG2 may be performed.

Next, the calculation device 12 outputs the composite image IMG3 and the measurement results to the input/output device 20 and displays them on the display device (step S32). In step S32, information such as the visible image IMG1, the thermal image IMG2, the facial temperature, and the environmental temperature is output to the display device.

According to this embodiment, the body temperature of the person being measured can be measured with high accuracy and efficiency by performing temperature correction based on the environmental temperature obtained from the background area BG2.

The non-contact body temperature measuring device 10 according to this embodiment may be installed, for example, at the entrance and exit gates of a building, and may unlock the door or activate an alarm device in response to a signal from the computing device 12.

Figure 6 is a diagram showing an example of the output (composite image IMG3) to the input/output device 20 when the non-contact body temperature measuring device 10 according to this embodiment is applied to an entrance/exit gate or the like. Note that the color representation in Figure 6 is the same as in Figure 5. In Figure 6, the regions corresponding to the person P1, head H1, torso B1, pupil region G1 and face region F1 in Figure 4 are respectively labeled with the symbols P3, H3, B3, G3 and F3 to show the correspondence between the regions.

In the example shown in FIG. 6, a visible image IMG1 and a thermal image IMG2 are displayed superimposed, with a color scale CS displayed to the right of the superimposed images. In addition, the facial temperature measurement result and a warning message M1 based on the facial temperature measurement result are displayed. This makes it possible to notify an operator or the like of an abnormality in the body temperature of the person being measured.

Second Embodiment
7 is a block diagram showing a non-contact body temperature measuring device according to a second embodiment of the present invention. The non-contact body temperature measuring devices 10-1 and 10-2 of this embodiment have the same configuration as the first embodiment, and are assigned sub-numbers to distinguish between the multiple non-contact body temperature measuring devices. Note that the sub-numbers will be omitted for explanations common to the non-contact body temperature measuring devices 10-1 and 10-2.

As shown in FIG. 7, in this embodiment, multiple (two in the example of FIG. 7) non-contact body temperature measuring devices 10-1, 10-2 are installed, and the measurement results by the non-contact body temperature measuring devices 10-1, 10-2 can be collected and stored by the management device 50. Here, the management device 50 and the non-contact body temperature measuring devices 10-1, 10-2 are connected via a wired interface such as a Universal Serial Bus (USB) or a Local Area Network (LAN), or a wireless interface such as a wireless LAN, WiFi, or Bluetooth (registered trademark).

The management device 50 is a general-purpose operation terminal such as a personal computer, a workstation, or a tablet terminal. The management device 50 includes a CPU, various memories such as an EEPROM, a RAM, and a VRAM, a storage device, and an input/output device. The management device 50 may have a configuration similar to that of the calculation device 12.

In addition, data such as the measurement results from the non-contact body temperature measuring devices 10-1 and 10-2 can also be processed, for example, on a cloud server via an internet line.

In this embodiment, by arranging multiple non-contact body temperature measuring devices 10, it becomes easy to apply the device to facilities that have multiple entrances and exits and where many people enter and exit, such as schools, stations, and entertainment facilities. For example, by arranging non-contact body temperature measuring devices 10 at all entrances and exits and entrance and exit gates, it becomes possible to manage the health status and entry and exit status of all people entering and exiting the facility.

It is also possible to transmit data such as the visible image IMG1 and thermal image IMG2 from the non-contact body temperature measuring device 10 to a management device 50 (external computer), and perform the processing described in the first embodiment in the management device 50. This allows all collected data to be estimated using the same criteria. For example, if the face temperature of each measurement target is repeatedly determined using each non-contact body temperature measuring device 10 by identifying the measurement target using face recognition in addition to temperature estimation, there is a possibility that the criteria for detecting and correcting the face temperature may vary depending on the machine learning situation. However, by repeating estimation using the same criteria from each image, the frequency of machine learning can be improved, and the estimation accuracy can be improved. In this case, it is also easy to store the collected image data.

(Example of application to an entrance/exit management system)
The following describes the case where this embodiment is applied to an entrance/exit management system for a facility. In a typical entrance/exit management system, a person is determined to be a registered person using identification information (ID) and image information such as a face registered in advance, and the entrance/exit gate is unlocked. However, such a system requires prior registration, and is therefore not suitable for managing the entrance/exit of an unspecified number of people.

In this embodiment, therefore, a unique identification information (ID) is automatically assigned to information presumed to be from the same person from data collected by the visible image camera 16, etc., and the ID is used to determine whether the person is currently staying in the facility or has already left. According to this embodiment, by recording the entry and exit times, it is possible to know the length of stay of each person. In addition, the composition of people who visited in groups can be inferred from the simultaneous (close) entry times and locations and exit times and locations.

In addition, by storing data collected by the visible image camera 16 etc. for a fixed period of time, such as one month, if a problem such as an infectious disease occurs later, it is possible to determine the person's behavioral record from the image stored in the management device 50 and extract images of people who may have been nearby at the time. At this time, if the image stored in the management device 50 is linked to the person's personal information such as contact details, it becomes possible to easily contact the person or request an examination as necessary.

As described above, in this embodiment, there is no need to register personal information in advance, and data such as the visible image IMG1 of the face can be linked to personal information after the fact as necessary. For example, if a problem occurs at a facility where such equipment is installed, it is possible to announce the date on which such problem occurred at the facility and, if necessary, have the person involved report it voluntarily, thereby determining the person's stay time and location of activity, and efficiently narrowing down contact information.

In addition, if the system according to this embodiment detects that someone has an abnormality in their health condition, it can be used to notify staff via wireless device or take other actions such as stopping the entrance and exit gates.

In addition, in cases where a particular person, such as a care recipient, hospitalized patient, or facility staff member, enters and exits a care facility or hospital multiple times in a short period of time, the body temperature of the photographed person is measured each time, making it possible to review the history of changes in physical condition and quickly respond to any abnormalities that may occur. For example, by placing multiple non-contact body temperature measuring devices 10 along the routes that such people frequently pass, it becomes possible to more accurately detect their behavior and the time when the abnormality occurred.

According to this embodiment, there is no need to register identification information in advance, so if an abnormality occurs, for example, in an entertainment facility such as an amusement park or a commercial facility such as a shopping mall, it is possible to later estimate, based on the facial image, the behavioral record of the individual, or people who are likely to have come into contact with the person with the abnormality. If necessary, it is also possible to later link personal information such as name and contact information to the facial image.

Figure 8 is a flowchart showing the process flow when the non-contact body temperature measuring device according to the second embodiment of the present invention is applied to an entrance/exit management system, and Figure 9 is a flowchart showing the person detection process of Figure 8. The processes shown in Figures 8 and 9 will be explained as being performed in the management device 50 based on data (visible image IMG1 and thermal image IMG2, etc.) collected using each non-contact body temperature measuring device 10.

First, the management device 50 sends an instruction to each non-contact body temperature measuring device 10 to start monitoring (step S50), and the visible image camera 16 and thermal image camera 18 of each non-contact body temperature measuring device 10 capture images of their respective measurement ranges (step S52).

Next, the management device 50 performs processes such as person detection and recording (steps S54 and S56). Note that steps S54 and S56 are performed for each image sent from each non-contact body temperature measuring device 10.

First, a person is detected from the visible image IMG1 acquired in step S52 (step S54). The person detection process is the same as that in FIG. 3. If a person is not detected from the visible image IMG1 in step S54 (No in step S54), the process returns to step S52 and person detection is performed on the visible image IMG1 of the next frame.

On the other hand, if a person is detected from the visible image IMG1 in step S54 (Yes in step S54), a person detection process is performed (step S56).

The above process is then repeated until the management device 50 sends an instruction to end monitoring to each non-contact body temperature measuring device 10 (Yes in step S58).

Next, the person detection process (step S56) will be described. In step S56, the management device 50 first detects a person from the visible image IMG1 (step S70), and determines whether the data stored in the storage device of the management device 50 contains data of the same person as the person detected in step S70 (step S72).

If there is no data of the same person as the person detected in step S70 (No in step S72), the management device 50 issues a new ID and associates the visible image IMG1 and the corresponding thermal image IMG2 with the new ID and stores them in the storage device (step S74).

On the other hand, if there is data of the same person as the person detected in step S70 (Yes in step S72), the management device 50 associates the visible image IMG1 and the corresponding thermal image IMG2 with the ID of the same person and stores them in the storage device of the management device 50 (step S76).

Next, the management device 50 performs processing similar to that shown in FIG. 3 to extract the face area F1 and the forehead area A1 using the visible image IMG1, and detect and correct the face temperature using the thermal image IMG2. These detection results may be output to an input/output device of the management device 50, or may be output to the input/output device 20 of the non-contact body temperature measuring device 10.

If the facial temperature is below the reference value, the management device 50 determines that there is nothing abnormal with the person (No in step S78) and returns to the main flow in FIG. 8.

On the other hand, if the facial temperature is equal to or higher than the reference value, the management device 50 determines that there is something wrong with the person (Yes in step S78) and performs a notification process on the non-contact body temperature measuring device 10 that transmitted the visible image IMG1 showing the person (step S80). As a result, for example, a composite image IMG3 and a warning message M1 as shown in FIG. 6 are output to the input/output device 20 of the non-contact body temperature measuring device 10 that is the target of the notification process. In addition, a notification may be sent wirelessly or otherwise to a staff member near the non-contact body temperature measuring device 10. This makes it possible to identify people whose facial temperature has been detected to be abnormal, and to stop their entry at the gate.

In the above example, the processes shown in Figures 8 and 9 are performed in the management device 50, but the present invention is not limited to this. For example, each non-contact body temperature measuring device 10 may perform the processes shown in Figures 8 and 9, and the results may be transmitted to and stored in the management device 50.

In addition, in each of the above embodiments, the measurement subject is described as a human being, but the present invention can also be applied to measurement subjects other than humans, such as animals, by estimating the body temperature based on a specific position on the body surface that reflects the body temperature of the target animal.

10, 10A, 10-1, 10-2... Non-contact body temperature measuring device, 12, 12-1, 12-2... Computing device, 14, 14-1, 14-2... Power supply, 16, 16-1, 16-2... Visible image camera, 18, 18-1, 18-2... Thermal image camera, 20, 20-1, 20-2... Input/output device, 22, 22-1, 22-2... Sensor, 30... Operation terminal, 32... Camera module, 50... Management device

Claims (8)

a visible image camera that receives light in a visible light wavelength range from a measurement object and captures a visible image;
A thermal imaging camera for capturing a thermal image that visualizes the amount of infrared radiation emitted from the measurement object;
a face temperature detection unit that detects a face area from the visible image and detects a face temperature from the face area in the thermal image;
a face temperature correction unit that identifies a background area outside the contour of the measurement object from the visible image, calculates a temperature of an area in the thermal image corresponding to the background area as an environmental temperature of the space in which the measurement object exists, and corrects the face temperature based on the environmental temperature;
A non-contact body temperature measuring device comprising: The non-contact body temperature measurement device according to claim 1, wherein the face temperature correction unit determines a representative value or a median value of temperatures at multiple measurement points in the background region of the thermal image as the environmental temperature. The non-contact body temperature measuring device according to claim 2, wherein the face temperature correction unit calculates the representative value by excluding, from among the plurality of measurement points, measurement points whose absolute difference from the average temperature of the measurement points is equal to or greater than a threshold value. The non-contact body temperature measuring device according to any one of claims 1 to 3, wherein the face temperature correction unit identifies a background region from each of multiple frames of visible images, and determines, as the environmental temperature, a temporal representative value of the temperature of the region corresponding to the background region in multiple frames of thermal images that correspond to the multiple frames of visible images. The non-contact body temperature measuring device according to any one of claims 1 to 4, wherein the face temperature correction unit uses a table showing the correspondence between the measured distance between the visible image camera and the face of the measurement target and the size of the face area to calculate an estimate of the measured distance of the face area, and adds a correction value to the face temperature if the estimated value is equal to or greater than a threshold value. The non-contact body temperature measurement device according to any one of claims 1 to 5, wherein the facial temperature correction unit acquires information on at least one of the environmental temperature, humidity, and air pressure of the space in which the measurement target exists, and corrects the facial temperature based on the information. A step of receiving light in a wavelength range of visible light from the measurement object by a visible image camera and capturing a visible image;
Taking a thermal image by a thermal imaging camera, which visualizes the amount of infrared rays emitted from the measurement target;
detecting a face region from the visible image and detecting a face temperature from the face region in the thermal image;
identifying a background area outside the contour of the measurement object from the visible image, determining a temperature of an area in the thermal image corresponding to the background area as an environmental temperature of the space in which the measurement object exists, and correcting the face temperature based on the environmental temperature;
A non-contact body temperature measurement method including: A function to capture a visible image by receiving light in the visible wavelength range from the measurement target using a visible image camera;
A function of capturing a thermal image that visualizes the amount of infrared rays emitted from the measurement object by a thermal imaging camera;
a function of detecting a face area from the visible image and detecting a face temperature from the face area in the thermal image;
a function of identifying a background area outside the contour of the measurement object from the visible image, calculating the temperature of an area in the thermal image corresponding to the background area as an environmental temperature of the space in which the measurement object exists, and correcting the face temperature based on the environmental temperature;
A non-contact body temperature measurement program to realize this on a computer.

Images