KR102581989B1 - Method for auto-recognition of temperature calibrator in thermal camera system - Google Patents

Abstract

According to an embodiment of the present invention, the image processing device includes: receiving a real image of a subject including a subject to be photographed; and recognizing, by the image processing device, a temperature calibrator in the real image of the subject based on identification information of the registered temperature calibrator. A method for automatically recognizing a temperature calibrator is disclosed, comprising:

Description

Method for auto-recognition of temperature calibrator in thermal camera system}

The present invention relates to a method for automatically recognizing a thermal camera and a temperature calibrator, and more specifically, to a device and method for measuring body temperature and monitoring fever using a live camera and a thermal camera.

Recently, respiratory infectious diseases such as SARS (Severe Acute Respiratory Syndrome), new flu, MERS, and coronavirus have been occurring frequently, and equipment that measures body temperature using a thermal camera is being developed to detect and track people with symptoms of these infectious diseases. It is widely used.

A conventional method of measuring body temperature using a thermal camera includes installing a thermal camera in a passageway and measuring the body temperature of passengers passing through the passageway to identify people with high fever. In addition, a real camera and a thermal camera are used simultaneously to photograph passersby and the real image and/or thermal image are displayed on the screen to easily display and identify each passerby and their body temperature.

In the case of these thermal cameras for measuring body temperature, it is important to accurately measure body temperature, so it is necessary to frequently calibrate the temperature measurement results of the thermal camera. However, there is an inconvenience in that a separate temperature calibration means such as a black body must be provided for temperature calibration, and the temperature of the passerby cannot be measured during temperature calibration. Additionally, in the case of conventional thermal cameras, when multiple people are photographed at once and each person's body temperature information is displayed on one screen, the screen becomes too complicated and it is difficult to identify body temperature information, making it often difficult to identify people with abnormal body temperature.

Patent Document 1: Korean Patent Publication No. 10-2018-0123900 (published on November 20, 2018) Patent Document 2: Korean Patent No. 10-1729327 (announced on April 17, 2017)

The present invention is intended to solve the above problems, and its purpose is to provide a technology that allows a thermal camera to automatically recognize a temperature correction means and perform temperature correction.

In addition, the present invention displays each person's body temperature information obtained by shooting with a thermal camera on the screen in a more appropriate manner, so that even if a large number of people are photographed at once, the body temperature information is appropriately displayed on the screen, making it possible to easily identify people with abnormal body temperature. The purpose is to provide technology that can

According to one embodiment, in an image processing device including an image processing unit that processes real images and thermal images respectively taken by a real image camera and a thermal camera shooting the same subject, and a black recognition unit that recognizes an infrared radiator in a thermal image, A method of automatically recognizing a temperature calibrator, comprising: receiving, by the image processing device, a real image of a subject including a subject to be photographed; and recognizing, by the image processing device, a temperature calibrator in the real image of the subject based on identification information of the registered temperature calibrator. A method for automatically recognizing a temperature calibrator is provided, comprising:

According to one embodiment, a temperature calibration method of calibrating the temperature of a thermal camera by an image processing device includes the steps of recognizing, by the image processing device, the temperature calibrator in an image taken of the temperature calibrator; Recognizing, by the image processing device, an infrared radiator in the image of the recognized temperature calibrator; and calibrating, by the image processing device, temperature information of a thermal image output by the thermal camera by using the temperature of the infrared radiator as a reference temperature. .

According to one embodiment of the present invention, at a predetermined period or when a predetermined event occurs, the thermal camera can automatically recognize the temperature correction means and perform a temperature correction procedure, and can also collect the body temperature information of each person obtained by photographing with the thermal camera. By dividing the display into full display mode and selective display mode according to predetermined standards, body temperature information can be displayed appropriately even when photographing a large number of people, and people with abnormal body temperature can be easily identified and tracked.

1 is a diagram for explaining a thermal camera system according to an embodiment of the present invention;
Figure 2 is a block diagram of a thermal camera system according to one embodiment;
3 is a flowchart of a method for displaying a real or thermal image according to one embodiment;
4 is a flowchart of a method for detecting a face in an image according to one embodiment;
5 is a flowchart of an example method for compositing real images and thermal images according to one embodiment;
6 and 7 are diagrams illustrating a method of displaying body temperature on a thermal or real image according to an embodiment;
Figure 8 is an example flowchart for explaining a temperature calibration method according to one embodiment.

The above objects, other objects, features and advantages of the present invention will be easily understood through the following preferred embodiments related to the attached drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the disclosure will be thorough and complete and so that the spirit of the invention can be sufficiently conveyed to those skilled in the art.

In this specification, when an element is referred to as being on another element, it means that it may be formed directly on the other element or that a third element may be interposed between them. Similarly, when this specification refers to a component being connected (or coupled, fastened, attached, etc.) to another component, it is either directly connected (or coupled, fastened, attached, etc.) to the other component, or between them. This means that it can be indirectly connected (or combined, fastened, attached, etc.) through a third component. Additionally, the thickness of components in the drawings of this specification are exaggerated for effective explanation of technical content. Additionally, in the drawings of this specification, the length, width, volume, size, or thickness of components are exaggerated for effective explanation of technical content.

In this specification, when terms such as first, second, etc. are used to describe components, these components should not be limited by these terms. These terms are merely used to distinguish one component from another. Embodiments described and illustrated herein also include complementary embodiments thereof.

As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. The expressions 'including', 'consisting of', and 'consisting of' used in the specification do not exclude the presence or addition of one or more other components in addition to the mentioned components.

In this specification, the term 'software' refers to technology that moves hardware in a computer, and the term 'hardware' refers to the tangible devices or devices that make up a computer (CPU, memory, input device, output device, peripheral device, etc.) , the term 'step' refers to a series of processing or manipulation connected in time series to achieve a predetermined goal, the term 'computer program' or 'program' refers to a set of instructions combined to be processed by a computer, and the term ' ‘Program recording medium’ refers to a computer-readable recording medium that records programs used to install, run, or distribute programs.

In the specification herein, the term 'management' of data or information may be used to include at least one operation of receiving, transmitting, storing, modifying, and deleting data or information. Terms such as '~unit', '~module', '~unit', '~block', and '~board' used in this specification to refer to the components of the invention refer to processing at least one function or operation. It may refer to a physical, functional, or logical unit, which may be implemented as one or more hardware, software, or firmware, or as a combination of one or more hardware, software, and/or firmware.

As used herein, 'processing unit', 'computer', or 'computing device' refers to an operating system such as Windows, Mac, or Linux, a computer processor, memory, applications, storage devices (e.g., HDD, SDD), and It may be a device equipped with a monitor. The computer may be, for example, a desktop computer or a laptop, but these are examples and the present invention is not limited to the desktop computer or laptop. The mobile terminal may be one of mobile wireless communication devices such as a smartphone, tablet PC, or PDA.

As used herein, 'identifying information' refers to at least one of the shape, color, and pattern of the temperature calibrator and may include, for example, the shape of the case of the temperature calibrator (the edge line (outline) of the case or the boundary line between the infrared emitter and the case). ), the color of the case, and at least one of the patterns appearing on the case.

In the specification herein, 'subject real image' refers to a real image including a temperature calibrator and a subject to be photographed, and 'subject thermal image' refers to a thermal image including a temperature calibrator and a subject to be photographed. The ‘subject real image’ and ‘subject thermal image’ were obtained for the same subject and the same corrector, respectively.

In the present specification, the 'real image for registration' is a real image used to register the identification information of the temperature calibrator and includes a temperature calibrator, and the 'thermal image for registration' refers to the temperature of the black body included in the temperature calibrator. It is a thermal image used for registration and refers to a thermal image that includes a temperature calibrator. That is, the 'actual image for registration' is an image used to register the identification information of the temperature calibrator, and the 'thermal image for registration' is an image used to register the reference temperature, each obtained for the same calibrator.

The present invention will be described in detail below with reference to the drawings. In describing the specific embodiments below, various specific details have been written to explain the invention in more detail and to aid understanding. However, a reader with sufficient knowledge in the field to understand the present invention can recognize that it can be used without these various specific details. In some cases, it is mentioned in advance that parts that are commonly known but are not significantly related to the invention are not described in order to prevent confusion in describing the invention.

Figure 1 schematically shows a thermal camera system according to an embodiment of the present invention. Referring to the drawings, a thermal camera system according to an embodiment includes a camera module 10, an image processing device 20, and a display 30. The thermal camera system of the present invention can be used to measure and display the temperature of an arbitrary subject (S), and in this specification, the present invention will be described assuming the case of measuring a person's body temperature as an example. .

In one embodiment, the camera module 10 includes a real camera and a thermal camera. A real camera is a camera that generates image data by capturing images in the visible light range of the object being photographed. A thermal camera is a camera that detects infrared rays naturally emitted from the subject and converts the amount of infrared rays into heat to generate temperature data of the subject. In the drawing, it is shown that both a real camera and a thermal camera are provided within the camera module 10, but of course, the real camera and the thermal camera may be installed independently.

The image processing device 20 processes image data captured by the camera module 10. For example, the image processing device 20 pre-processes the real image and/or thermal image received from the camera module 10 (e.g., noise removal, filtering, synchronization, registration, etc.) and selects one or more shooting targets from the real image or thermal image. It is possible to perform image processing such as detecting the face of (S) and overlapping temperature information for each shooting target (S) with the real or thermal image, and using the processed real image data, thermal image data, and/or real image. Synthetic data of overheating can be output to the display 30. The specific operation of the image processing device 20 will be described later with reference to FIG. 2.

The display 30 is a device that displays image data processed by the image processing device 20 to the user, and may be implemented as any image output device, such as an LCD display or an LED display. In one embodiment, the image processing device 20 may output real images and thermal images to the display 30 in various ways. For example, only one of the real image or the thermal image can be output to the display 30, or a composite image that overlaps the real image and the thermal image can be output, for example, through the PIP (Picture-In-Picture) function. You can also split and output the real image and thermal image in parallel.

In one embodiment, the thermal camera system of the present invention may further include a temperature calibrator 40. The temperature calibrator 40 is equipment used for temperature correction of a thermal camera, and in one embodiment, it may be composed of an infrared radiator 41 and a case 42 that accommodates the same. The infrared emitter 41 is an object that emits infrared rays and is preferably a black body. An ideal black body is an object that absorbs and does not reflect all light coming from the outside and emits only light of the wavelength corresponding to the energy it has. Therefore, a black body can be used for temperature correction of an infrared camera (thermal camera) that takes infrared images of an object and measures the exact temperature of the object. In one embodiment, the black body can be made of a carbon material such as carbon black. The infrared emitter 41 is connected to an optional heating/cooling device and is configured to be maintained at a preset temperature.

The infrared emitter 41 is accommodated in the case 42 and is configured to be visible from the outside through a through hole on one side of the case 42. In one embodiment of the present invention, the case 42 has a predetermined shape, such as a hexahedron, and is configured to allow the infrared emitter 41 to be viewed through the front surface of the case. At this time, the remaining area of the front surface of the case 42 surrounding the radiator 41 is painted in a preset color (eg, red) or has a predetermined pattern formed thereon.

In one embodiment, the temperature calibrator 40 is arranged to be located within the camera viewing angle for photographing the object S, and thus the thermal camera of the camera module 10 simultaneously photographs the object S while the temperature calibrator 40 ) can also be taken. Temperature calibration of the thermal camera using the temperature calibrator 40 will be described later with reference to FIG. 2.

Meanwhile, the image captured with a thermal camera above is referred to as a 'thermal image', but in this specification, it is also called a 'thermal image', 'thermal image', or 'infrared image' with the same meaning. Additionally, in this specification, 'image data' is simply called 'video' or 'image'.

Figure 2 is a block diagram of a thermal camera system according to one embodiment. In FIG. 2, the camera module 10, the image processing device 20, and the display 30 correspond to the camera module 10, the image processing device 20, and the display 30 in FIG. 1, respectively.

The camera module 10 may include a real camera 11 and a thermal camera 12. The real camera 11 can generate image data by capturing an image in the visible light region of the subject S. In one embodiment, the real camera 11 is implemented with hardware and/or software, such as a lens, a CCD, and an analog-to-digital (A/D) converter. The light coming through the lens is converted into an electrical signal by the CCD, and this signal is converted into image data, a digital signal, by the A/D converter. Since the configuration and operation of the real camera 11 are known technologies, detailed description will be omitted below.

The thermal camera 12 can detect infrared rays naturally emitted from the subject S and convert the amount of infrared rays into heat to generate temperature data of the subject S. In one embodiment, the thermal camera 12 includes an infrared lens that passes infrared rays, a detection element array that detects infrared rays incident on the infrared lens and converts them into electrical signals, and an analog-digital camera that converts the electrical signals into temperature data, which is a digital signal. (A/D) converter included.

An infrared lens is a lens that transmits infrared rays, and the detection element array detects infrared rays incident on the infrared lens and converts them into electrical signals. For example, the detection element array may have a focal plane array (FPA) structure in which a plurality of pixels are arranged in a two-dimensional array, but is not limited to this structure. The A/D converter converts the electrical signal, which is an analog signal generated from the detection element array, into temperature data, which is a digital signal. The thermal camera 12 may additionally perform non-uniformity correction (NUC) and dead pixel processing on temperature data converted from the A/D converter. Since techniques for non-uniformity correction and dead pixel processing are known in the field of thermal cameras, detailed descriptions are omitted.

Real images and thermal images generated by the real camera 11 and the thermal camera 12 of the camera module 10 are transmitted to the image processing device 20 and processed. In one embodiment, the image processing device 20 may include an image processing unit 21, a face detection unit 22, a black body recognition unit 23, and a temperature correction unit 24.

The image processing unit 21 performs image processing operations such as noise removal, filtering, synchronization, image registration and synthesis on the real image and/or thermal image received from the camera module 10, and includes the following functions, for example: Contains one or more of

(1) Noise removal and image improvement: The image processing unit 21 may perform processing for image improvement, such as image quality improvement and contrast improvement of real images and/or thermal images. For example, the image processing unit 21 may remove noise or unevenness by performing processes such as low-pass filtering, image blurring, and image smoothing.

Additionally, the image processing unit 21 can also perform high-pass filtering on the image. For example, high-pass filtering can be performed on a real image using a spatial filter to extract the high spatial frequency component of the real image (i.e., the outline or borderline portion of the photographed object). As another method of extracting high spatial frequency components from an image, the image processing unit 21 may use a method of extracting the difference between two temporally consecutive images. That is, by obtaining the difference between the image at the first time and the image at the second time in the continuous image sequence, high spatial frequency components representing the edge or outline of the photographed object can be extracted.

(2) Image synchronization: The image processing unit 21 temporally synchronizes the real images and thermal images taken by the real image camera 11 and the thermal image camera 12 at the same time, respectively, to produce real images and thermal images corresponding to the same time. You can get it. In general, the real image camera 11 and the thermal image camera 12 have different image capture speeds due to differences in hardware/software such as CCD, detection element array, and A/D converter, so they cannot acquire real images and thermal images at the same time. The outputs of the real camera 11 and the thermal camera 12 are synchronized so that the output is synchronized.

(3) Image registration: The image processing unit 21 can match the real image and the thermal image so that the real image camera 11 and the thermal image camera 12 photograph the same subject at the same viewing angle and output the image in the same size. Additionally, in order to match the real image and the thermal image, the image processing unit 21 may change the resolution of one of the two images to have the same resolution.

(4) Color thermal image conversion: The image processing unit 21 can convert each pixel of the thermal image into a color according to the temperature data of the corresponding pixel and output it. In other words, the temperature data of each pixel is converted into a color matched according to the temperature band to generate colored thermal image data. For example, the image processing unit 21 generates a color thermal image by applying RGB values matched to each temperature value of the temperature data of each pixel. Accordingly, for example, a color thermal image can be generated in which high temperatures are displayed in red and low temperatures are displayed in blue.

Additionally, the image processing unit 21 can detect the temperature of a specific pixel and perform application processing accordingly. For example, a pixel with the highest temperature among all pixels can be detected, or pixels with a temperature within a preset temperature range can be detected.

(5) Image synthesis: The image processing unit 21 can generate one composite image by combining real and thermal images taken by the real camera 11 and the thermal image camera 12 for the same shooting target. For image synthesis, the above-described image synchronization and matching processing may first be performed, and then, for example, a composite image may be generated by overlapping part or all of the thermal image with the real image. Alternatively, a composite image can be created by overlapping information indicating the temperature of a specific pixel or specific pixel area on a real image or thermal image.

Some functions of the image processing unit 21 described above will be described further later with reference to other drawings.

In the image processing device 20, the face detection unit 22 may detect the face area of the subject to be photographed from a real image and/or a thermal image and calculate detection area information including the coordinates of the detected face area. In one embodiment, the face detection unit 22 may detect a face using predefined facial features. For example, a face can be detected using the size, shape, and correlation of facial feature components (e.g., eyes, nose, mouth, outline, brightness, etc.), and facial color and texture information. Alternatively, the face detection unit 22 may use not only facial feature points but also non-face feature points. For example, in a case where the subject of the photo is wearing a hat, sunglasses, or a mask, specific points about various types and sizes of hats, sunglasses, and masks can be defined in advance and used. Additionally, to first detect a person in an image and then detect the face area, feature points on the entire human body or upper body (for example, each joint and skeleton of the human body, etc.) can be used.

In one embodiment, the face detection unit 22 may detect a face using pattern recognition based on the appearance of the face. In other words, it is a method of learning face patterns through pattern recognition on a set of learning images of various facial poses and detecting faces using the learned model. For example, there are methods using the eigenface generated by principal component analysis (PCA), linear discriminant analysis (LDA), artificial neural network (ANN), Adaboost, and support vector machine (SVM). .

In one embodiment, the face detection unit 22 detects the texture or tone of the skin to distinguish the face from the skin of a person's hand or arm, and can determine the face based on this information. For example, the face detection unit 22 can detect the texture of an image using a local binary pattern (LBP) method and use it for face detection. Alternatively, the face detection unit 22 uses HSV (hue, saturation, brightness) color expression and applies a threshold according to the color and brightness of the entire image based on a predetermined threshold in the HSV color space to detect the image. The face area can also be detected.

The face detection unit 22 may detect faces in real and/or thermal images using various known methods and algorithms in addition to the face detection method described above.

The black body recognition unit 23 and the temperature correction unit 24 are functional units that recognize the temperature calibrator 40 in the captured image and correct the temperature of the object to be photographed based on this. An exemplary embodiment of recognizing the temperature calibrator 40 and performing a temperature calibration operation will be described later with reference to FIG. 8 .

The image processing unit 21, face detection unit 22, black body recognition unit 23, and temperature correction unit 24, which are each functional unit of the above-described image processing device 20, are equipped with software programmed to perform the corresponding functions and as necessary. Accordingly, it may be configured with hardware that supports it. In addition, these functional units 21, 22, 23, and 24 may each be implemented as independent software or may be implemented as one integrated software, and at least some of these functional units may be implemented by any arbitrary device outside the image processing device 20. It can also be installed and executed on a processing device.

Figure 3 is a flowchart of a method for displaying a real or thermal image according to one embodiment.

Referring to the drawing, first, in step S10, the subject S is photographed with the real image camera 11 and the thermal image camera 12 to generate a real image and a thermal image, respectively, and transmit them to the image processing device 20. . In step S20, the image processing device 20 may perform preprocessing on the received real and thermal images. For example, image noise removal, high-band/low-band filtering, etc. can be performed to improve images, and synchronization and registration processing of real and thermal images can be performed.

Then, in step S30, the face detection unit 22 of the image processing device 20 detects one or more faces in the image. In this regard, Figure 4 shows an example method by which the face detection unit 22 detects a face in an image.

Referring to Figure 4, in steps S31 and S32, the face detector 22 extracts the face shape from the real image. In this step (S31), the face detection unit 22 may extract the face shape by, for example, a method using facial feature points described with reference to FIG. 2 or a pattern recognition method such as an artificial neural network. At this time, in this specification, 'facial shape' is distinct from an actual 'face' and refers to a shape that may not be an actual face, but has a facial shape. That is, the 'face shape' extracted in step S31 may be an actual face or an object with a shape similar to a face.

Also, in this step (S31), the shape of the entire face can be extracted from the real image, but a shape similar to a part of the face can also be extracted as the face shape. For example, as described in Figure 2, if not only facial feature points but also non-face feature points such as a hat, sunglasses, mask, or upper body of the human body are used, part of the face can be recognized and extracted as a facial shape.

Simultaneously or sequentially with step S31, the face detection unit 22 extracts a body temperature region from the thermal image in step S32. For example, the body temperature area can be set to a predetermined temperature range, such as 35.5 to 39 degrees Celsius, and the pixel area representing the temperature value of this set temperature range can be extracted as the body temperature area.

Next, in step S33, the face is estimated based on the facial shape and body temperature region extracted in the previous steps (S31 and S32). As an example method for estimating a face, the face can be estimated using the facial shape extracted from the real image and the body temperature area extracted from the thermal image as an OR condition. In this case, not only the entire face but also when only a part of the face is visible is presumed to be a face, and a hypothermic face or an object with a temperature similar to body temperature is also presumed to be a face, so as many faces as possible can be detected in all possible situations.

In the next step (S34), additional information such as skin texture and color tone is obtained for the estimated face. For example, information such as texture information or color tone information can be additionally obtained from a real image. In an alternative embodiment, hot spots on the face may be detected in a thermal image. In other words, hot spots that appear in the blood flow of the face can be detected on a thermal image and used as additional information.

In one embodiment, in order to reduce the amount of data processing required to obtain additional information, this additional information may be extracted only for the pixel area extracted as the body temperature region from the thermal image. Additionally, this step (S34) may be executed after the face estimation step (S33), but alternatively, it may be executed before the face estimation step (S33).

Then, in step S35, the face is detected by determining whether the estimated face is a real face based on additional information such as texture or color tone. By using this additional information that distinguishes the skin from the face, such as the texture or tone of the face, the skin of the arms or hands, which are not the face, can be excluded from the estimated face, thereby increasing the accuracy of face detection.

Additionally, according to the above-described face detection method, low body temperature faces can also be detected. To avoid temperature measurement by a thermal camera, some people wash their face with cold water or spray water on their face right before the temperature measurement, or arrive in front of the camera after being in a cold place for a long time. In these cases, the temperature of the face may be lower than the normal body temperature range. It comes out low. Until now, fever monitoring has been aimed at detecting people with high temperatures, so these cases of hypothermia have been ignored. However, according to the present invention, hypothermic people can also be detected, preventing various attempts to avoid temperature measurement.

Referring again to FIG. 3, if a face is detected in the image as described above (S30), the real image and the thermal image can optionally be synthesized in step S40. In one embodiment, a composite image may be created by adding body temperature information to only one of the real image or the thermal image. In this case, if there is no need to composite the real image and the thermal image, step S40 may be omitted. However, if at least part of the real image and at least part of the thermal image need to be synthesized and output, step S40 can be performed. Additionally, even when only thermal images are output to the display, it may be necessary to composite a part of the real image into the thermal image to improve the thermal image. In this regard, Figure 5 shows an exemplary method of combining a part of a real image with a thermal image. When the thermal image is output to the display 30, preprocessing is performed to display the outline of the face or body appearing in the thermal image more clearly. The operation is shown as an example.

Referring to Figure 5, in step S110, high-band filtering is performed on the real image. By performing high-pass filtering on an image, you can extract the outline (edge) or boundary line of the subject. Therefore, through this step (S110), the edges (contours) of the entire face or part of the subject or the entire or part of the body can be extracted.

And step (S120) can be executed simultaneously with step (S110) or sequentially. That is, in step S120, the thermal image can be low-pass filtered, thereby removing noise from the thermal image. Then, in step S130, the edges of the extracted real image are overlapped with the thermal image. If the edges of the subject are overlapped and combined with the thermal image, the outline of the subject appears more clearly in the thermal image, making it easier to recognize the shape of the subject. Therefore, even when multiple faces are captured in a thermal image, face-to-face identification can be performed quickly.

Referring again to FIG. 3, after executing step S30 or step S40, body temperature information is added to the real image and/or thermal image in step S50 and transmitted to the display 30 for output. By adding body temperature information to real and/or thermal images, body temperature information can be displayed on the screen as shown in Figure 6 or Figure 7.

Figures 6 and 7 show a method of displaying body temperature on a thermal or real image according to an embodiment. Figure 6 schematically shows a method of displaying body temperature information in the form of a tag on the screen 31 of the real image or thermal image. It is shown as .

Referring to Figure 6, when one or more people are photographed on the output screen 31 of the display 30, their body temperature information is recorded for each photographing object S1 and S2. It can be displayed in the form of tags (T1, T2) at a certain distance from the face, that is, within a certain pixel distance. At this time, the body temperature information may be, for example, information expressing the body temperature in Arabic numerals as shown, but is not limited thereto.

In one embodiment, the body temperature of each photographed subject represents the body temperature at a predetermined feature point among the feature points of the face. The predetermined specific point may be, for example, one of the eyebrows or the center of the forehead. Conventionally, after detecting a face in a thermal image, the temperature of the entire face was averaged or the temperature of the pixel with the highest temperature was determined and displayed as the body temperature of the face. However, if the subject of the photo is wearing a hat, glasses, mask, etc. or has heavy makeup, the body temperature value may not reflect the actual body temperature if averaged over the entire face, and if the highest temperature point within the face is displayed as body temperature, the body temperature information is not displayed. There is a problem in that the position of the tag keeps changing, making it inconvenient for the user to read the body temperature information.

However, as in the present invention, if a specific feature point is set within the face, such as the forehead or the center point of the forehead, and the temperature value of the pixel at this point is determined as the body temperature, the exact body temperature of the subject can be stably displayed at a certain location. Meanwhile, the feature point used at this time can be, for example, the facial feature point used in the face detection step (S30) of FIG. 3. That is, when a face is detected using facial feature points in step S30, the temperature value of the pixel corresponding to the feature point representing the forehead among the facial feature points used at this time can be extracted as the body temperature value of the subject.

Figure 7 schematically shows a method of selecting only body temperature information of some photographed subjects and displaying them as tags on the screen 31 of the real image or thermal image.

Referring to Figure 7, in one embodiment, when a plurality of people are photographed on the output screen 31 of the display 30, body temperature is displayed in a selective display mode. Here, the “selective display mode” for body temperature is an image display mode in which body temperature information is displayed as a tag (T) for only some people among multiple shooting subjects, and body temperature information for the remaining people is not shown in the video or is displayed in a way other than tags.

In the selective display mode according to the embodiment of Figure 7, body temperature information of one or more faces according to a preset standard among a plurality of faces is displayed at a location adjacent to each face (e.g., a location within a predetermined pixel distance from the face of each photographing target S). ) can be selected and displayed with a tag (T), and the remaining facial body temperature information according to the above standards can be displayed as a thermal image.

According to this embodiment, first, it is determined whether the body temperature of each of the plurality of faces detected by photographing with the cameras 11 and 12 falls within a preset temperature band. At this time, the preset temperature band may be, for example, a normal body temperature band or a temperature range of 35.8 degrees Celsius to 37.2 degrees Celsius.

If the body temperature of each detected face does not fall within the preset temperature band, that is, if it is higher or lower than the preset temperature, the body temperature of the face is displayed in the form of a tag (T) as shown in Figure 7. At this time, the tag (T) can be displayed as a number at a position within a certain pixel from the face.

If the body temperature of the face falls within this temperature range, the body temperature information of the face is not displayed or displayed in a non-tag manner. For example, the temperature of this face can be displayed by overlapping it with a color thermal image. That is, a color thermal image in which high temperatures are displayed in red and low temperatures are displayed in blue within the preset temperature band for the face can be overlapped on the face.

At this time, among the images displayed, the body and background (B) excluding the face can be displayed as a black-and-white real image or thermal image, and only the face can be displayed as a color thermal image. In this case, the color resolution of the preset temperature band can be increased. there is. Therefore, according to this embodiment, even when many people are captured in one captured image, people whose temperature is higher or lower than the preset temperature band are classified as those with an abnormal body temperature and displayed with a tag (T), so that they can be easily identified. Even for people who fall within a preset temperature range, each face is displayed as a high-color resolution thermal image, making it easy to identify and track people with high and low body temperature.

Additionally, according to the present invention, various conditions for selective display with a tag (T) can be set. For example, only the body temperature information of the person with the highest temperature can be displayed as a tag (T), and for the remaining people, each person's face can be displayed as a color thermal image. As another example, only the people with the maximum and minimum temperatures can be tagged. It can also be indicated as (T). Additionally, since the temperature range can be set differently depending on the specific embodiment, it is possible to accurately classify and track people with and without an appropriate body temperature according to the specific situation in which the present invention is implemented.

In another embodiment of the present invention, body temperature information can be displayed while switching between a full display mode and a selective display mode depending on the number of people captured by the camera. According to this embodiment, for example, after detecting a plurality of faces in an image taken by the real camera 11 and/or the thermal camera 12, if the number of detected faces is less than or equal to a preset number, the body temperature of each of the plurality of faces Information is output to the display according to the full display mode, and when the number of detected faces exceeds the preset number, body temperature information for each of the plurality of faces is output to the display according to the body temperature selective display mode.

At this time, the 'full display mode' is a method of displaying body temperature information for all faces in the image with tags (T1, T2) at a location within a predetermined pixel distance from each face, as shown in Figure 6, and the 'selective display mode' is Among the plurality of faces in the image, body temperature information is displayed as a tag (T) only for some people who meet a predetermined standard, and body temperature information for the remaining people is not displayed or is displayed in a method other than a tag.

If there are, for example, two or three people being photographed by a camera, the body temperature information can be easily identified within the screen 31 even if the body temperature information is displayed as tags (T1, T2) for all people as shown in Figure 6, so in this case, all people are It is desirable to use the full display mode, which displays a person's body temperature information as a tag. However, for example, when a preset number of people (e.g. 5 or 6, etc.) or more are photographed at the same time and each person's body temperature is displayed, if all the body temperature information of all people is displayed as a tag, the screen 31 becomes complicated and it is difficult to identify each person's body temperature. Because it is difficult to do so, in this case, switch to the selective display mode as shown in Figure 7 to display body temperature information with a tag (T) only for some people according to preset conditions and do not display body temperature information for the remaining people or use color thermal imaging. It is desirable to display it in another way, such as:

Now, a temperature calibration method according to an embodiment will be exemplarily described with reference to FIG. 8 . Meanwhile, please refer to the descriptions mentioned in FIGS. 1 to 7 for exemplary configurations of devices or components (e.g., image processing unit, black recognition unit, temperature calibrator, temperature calibration unit, etc.) mentioned in this method. .

Referring to FIG. 8, the temperature calibration method according to an embodiment of the present invention includes the steps of installing a temperature calibrator in a camera shooting area (S210) and automatically recognizing the temperature calibrator (this step is 'automatically setting up the temperature calibrator'. (also known as 'recognition method'), recognizing a black body area in the thermal image of the subject based on the position of the temperature corrector 40 recognized in the real image of the subject (S240), and correcting the temperature based on the temperature of the black body area. It may include a step (S250). Here, the method of automatically recognizing the temperature calibrator includes registering identification information of the temperature calibrator 40 - at least one of shape information, color information, and pattern information (S220), and taking pictures with the temperature calibrator 40. A step of receiving a real image of the subject taken together with the subject, and a step of recognizing the temperature calibrator 40 in the real image of the subject using the information ('identification information') registered in the registration step (S220) (S230). It can be included. Meanwhile, the registration step (S220) is performed before the step of receiving the real image of the subject captured by the temperature calibrator 40 described above.

In this embodiment, the identification information of the temperature calibrator 40 includes, for example, information representing visual characteristics of the temperature calibrator 40 (eg, shape, color, pattern).

The step S230 of recognizing the temperature calibrator may be a step of identifying the temperature calibrator 40 in the real image of the subject described above using the identification information registered in step S220. For example, the step of recognizing a temperature calibrator (S230) is a step of determining whether the actual image of the subject is identical or extremely similar to the identification information registered in step (S220).

Below, the temperature calibration method will be described in more detail.

First, in step S210, the temperature calibrator 40 is installed in the camera shooting area. The temperature calibrator 40 is installed within the viewing angle of the thermal camera 12, and the installation location is not particularly limited. For example, the temperature calibrator 40 can be installed in a passageway where many people are measured.

Step S220 may be implemented in two forms, for example. The first form is performed at a location where the subject is photographed (i.e., the location where the temperature calibrator 40 is installed in step S210), and the second type is performed at any location other than the location where the subject is photographed. will be.

First, the first form of step S220 will be described. Step S220 includes obtaining a real image for registration by having the real image camera 11 photograph the temperature calibrator 40 installed in step S210, and obtaining a real image for registration. In the image, the black body recognition unit 23 may register identification information of the temperature calibrator 40, that is, at least one of shape information, color information, and pattern information, to the image processing device 20. .

For example, step S220 includes selecting a pixel area containing the temperature calibrator 40 shown on the real image for registration, recognizing identification information of the temperature calibrator 40 within the selected pixel area, and It includes the step of the image processing device 20 storing the recognized identification information. More specifically, when the image processing unit 21 selects a pixel area including the temperature calibrator 40 in the real image for registration, the black body recognition unit 23 selects the case 42 of the temperature calibrator 40 within the selected pixel area. At least one of the shape (edge line (contour) of case 42 or boundary line between infrared emitter 41 and case 42), color of front side of case 42, and pattern shown on case 42 It is recognized and registered in the image processing device 20.

Meanwhile, when registering the identification information of the temperature calibrator 40 in step S220, the image processing device 20 may measure the temperature of the infrared radiator 41 shown in the thermal image for registration and register it as the reference temperature. Here, the thermal image for registration is additionally acquired by the thermal image camera when the real image camera acquires the real image for registration with respect to the temperature calibrator 40.

The second form of step S220 will now be described, but only the differences from the first form described above will be explained. The only difference between the two forms is that in the second form of step S220, the place to acquire the real image for registration to the temperature calibrator 40 is any place other than the camera shooting area mentioned in step S210. . That is, the second form is to register identification information about the temperature calibrator 40 in the image processing device 20 at any place before going to the camera shooting area mentioned in step S210.

Step S230 is a step of acquiring a real image of the subject and a thermal image of the subject by photographing the temperature calibrator 40 together with the photographing object using the real camera 11 and/or the thermal camera 12, and step S220. It may include the step of determining whether the temperature calibrator 40 is present in the real image of the subject using the identification information registered in . For example, in the determining step, if an object identical to or extremely similar to the identification information registered in step S220 is included in the real image of the subject, such object is determined to be the temperature calibrator 40.

Thereafter, the image processing device 20 may identify the infrared emitter 41 of the temperature calibrator 40 in the subject thermal image (step S240). At this time, when the infrared emitter 41 is maintained at a preset temperature, the image processing device 20 determines that the temperature value measured from the infrared emitter 41 by the thermal camera 12 is the preset temperature. can do.

Afterwards, in step S250, the image processing device 20 corrects the temperature information of the thermal image of the subject using this predetermined temperature as a reference temperature. For example, if the temperature value of the pixel corresponding to the infrared emitter 41 is different from the actual temperature of the infrared emitter 41, the temperature of the subject thermal image can be corrected by applying this temperature difference value to all pixels. , In addition, other known temperature calibration methods may be used.

The automatic recognition and temperature correction operation of the temperature calibrator 40 as described above may be performed at predetermined time periods. Alternatively, the image processing device 20 may automatically determine whether to perform temperature correction and execute it. there is. For example, if the thermal camera 12 calculates the average value of the body temperature of the subject measured by shooting for a predetermined period of time, and this average value is outside the preset temperature range, the image processing device 20 executes the temperature correction method described above. It can be configured to do so.

Additionally, in one embodiment, the temperature calibrator recognition and temperature calibration operations may be performed simultaneously with the photographing, face detection, and body temperature measurement operations of the photographing subject (S). That is, when the subject S and the temperature calibrator 40 are simultaneously exposed to the viewing angle of the thermal camera 12 and the thermal camera 12 captures them, the image processing device 20 measures the body temperature of the subject S. Temperature calibration using the and temperature calibrator 40 can be performed simultaneously.

As described above, those of ordinary skill in the field to which the present invention pertains can understand that various modifications and variations are possible from the description of this specification. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the claims and equivalents thereof as well as the claims described later.

10: camera module 11: real camera
12: thermal camera 20: image processing device
21: image processing unit 22: face detection unit
23: black body recognition unit 24: temperature correction unit
30: Display 40: Temperature calibrator
41: infrared emitter 42: case

Claims (5)

In a method of automatically recognizing a temperature calibrator photographed together with a subject in an image by an image processing device,
The temperature calibrator has an infrared emitter that emits infrared rays,
The image processing device includes an image processing unit that processes real images and thermal images obtained by photographing a subject and a temperature calibrator with a real image camera and a thermal image camera, and a black recognition unit that recognizes an infrared radiator of the temperature calibrator in the thermal image,
The method is:
Receiving, by the image processing device, a real image of the subject obtained by photographing the subject and a temperature calibrator; and
Recognizing, by the image processing device, a temperature calibrator in the real image of the subject based on identification information of the registered temperature calibrator. According to claim 1,
A method for automatically recognizing a temperature calibrator, wherein the temperature calibrator includes a case of a predetermined shape surrounding the infrared radiator, and the front of the case has a preset color or pattern. According to claim 1,
A method for automatically recognizing a temperature calibrator, characterized in that the infrared emitter is connected to a heating or cooling device. According to claim 1,
The image processing device further includes registering at least one of the shape, color, and pattern of the case of the temperature calibrator as identification information,
A method for automatically recognizing a temperature calibrator, characterized in that the registering step is performed before the step of receiving the real image of the subject. According to claim 4,
The above registration steps are
selecting a pixel area containing a temperature calibrator on the real image for registration;
Recognizing at least one of a shape, color, and pattern of a case of the temperature calibrator within the selected pixel area; and
A method for automatically recognizing a temperature calibrator, comprising: storing at least one of information about the recognized shape, information about the color, and information about the pattern in the image processing device.