JPH0942924A - Thermal image sensor system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a human body discriminated from another heat source by binarizing image signals from a thermal type infrared matrix sensor detecting part, by carrying out labeling for detecting a heat source as a prospective human body, and by carrying out labeling for a heat source within an in-area pixel part. SOLUTION: Pixels of a thermal infrared matrix sensor detecting part 1 are divided into an end pixel part 1 an in-area pixel part 12. Pixel signals from the detecting part 1 are binarized (31), and a heat source is extracted. Successive pixels extracted from the pixel part 11 is subjected to an identical leveling process 32 so as to extract a heat source as a prospective human body, and a process of difference 33 between frames of sensor signals from the pixel parts 11 is carried out, and then the signals are delivered to a computing part 36. Meanwhile, the successive pixels extracted in the pixel part 12 are subjected to an identical labeling process 34, and are delivered to the computing part 36. Further, the computing part 33 determines the presence of human bodies, a number and positions thereof.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a thermal image sensor system for detecting human body information such as the presence, number and position of people in an area by detecting infrared rays from the human body.

[0002]

2. Description of the Related Art Conventionally, an infrared camera using a quantum infrared sensor has been well known as a device for forming a thermal image. However, the quantum infrared sensor cannot be used for consumer use because the element has to be cooled to the temperature of liquid nitrogen and thus has a limit to miniaturization and the price becomes very expensive. Therefore, a thermal image sensor using a small-sized and low-priced thermal infrared sensor is being developed. For example,
There are mechanically operated 8-pixel linear arrays that use PbTiO-based ceramic pyroelectric materials, and the number of pixels is 8 x 64.
Pixels and relatively high pixels are obtained. However, since mechanical operation is used, the life is short, and it is not suitable when a long time use such as lighting control or a crime prevention system is required. Further, since the ceramic pyroelectric material uses the principle that the variation of infrared rays from the heat source is captured by the dielectric polarization of the material, it is impossible to detect a stationary heat source. With a single element or linear array, it is possible to detect a stationary heat source by mechanically chopping it, but with a thermal pixel matrix sensor, how to chop becomes a problem, and it is difficult to detect a stationary human body. .

On the other hand, as a device which does not require such mechanical operation, there is a thermistor bolometer which detects a temperature change due to absorption of infrared rays by a change in electric resistance, which is a stationary heat source without mechanical chopping. It can be detected.

[0004]

However, in the detection of the heat source as described above, even if the steady heat source can be detected, the detected heat source is from the human body, the personal computer, the fluorescent lamp, the sunlight, and the human being. There is a problem in that it cannot distinguish it from other heat sources such as a chair that remains warm after sitting, which causes a considerable error.

The present invention has been made in view of the above points, and an object of the present invention is to provide a thermal image sensor system capable of distinguishing and detecting a human body from other heat sources.

[0006]

According to the first aspect of the present invention,
A thermal infrared matrix sensor detection unit in which pixels for detecting infrared rays are arranged in a matrix, a sensor control circuit unit that controls the thermal infrared matrix sensor detection unit, and a sensor image from the thermal infrared matrix sensor detection unit In a thermal image sensor system including a sensor signal processing unit that receives a signal and performs signal processing to detect a human body, each pixel of the thermal infrared matrix sensor detection unit is an edge pixel unit arranged in the periphery. And an in-area pixel section arranged inside, and the sensor signal processing section performs binarization processing for extracting a heat source from the sensor image signal from the thermal infrared matrix sensor detection section.
An end pixel for performing labeling for detecting a heat source as a human body candidate based on a signal from the binarization processing unit and a binarization processing unit, and an end pixel for obtaining a difference between frames of the labeled heat source as a human body candidate A frame difference processing unit, an in-area pixel labeling processing unit that labels the heat source in the in-area pixel unit, and an arithmetic unit that determines a human body based on signals from the respective units. It is a feature.

According to a second aspect of the present invention, in the first aspect of the present invention, the calculation section uses the output signals of the edge pixel frame difference processing section and the in-area pixel labeling processing section.
If there is a correlation between the two, it is determined that the labeled heat source as a human body candidate has moved from the edge pixel portion to the pixel portion in the area, and the heat source as the human body candidate is determined to be a human body. It is characterized by.

According to a third aspect of the present invention, in the first or second aspect of the present invention, an in-area pixel frame difference processing section for performing a frame difference process for detecting movement of a heat source in the in-area pixel section. Is added, and the heat source is traced by the calculation unit using the frame difference data from the in-area pixel frame difference processing unit.

According to a fourth aspect of the present invention, in the first to third aspects of the present invention, a pixel label area calculation processing unit for obtaining the area of the labeled pixel set is added, and the calculation unit has a pixel label area. It is characterized in that whether or not the heat source is the human body is determined based on the size of the area obtained by the calculation processing unit.

According to a fifth aspect of the present invention, in the first to fourth aspects of the present invention, the heat source as a human body candidate labeled by the end pixel labeling processing section is sampled a predetermined number of times in the computing section. When there is no change, it is characterized in that it is judged that it is not a human body.

According to a sixth aspect of the present invention, in the first to fifth aspects of the invention, a plurality of the thermal infrared matrix sensor detection units are arranged, and a sensor image from each thermal infrared matrix sensor detection unit is arranged. The feature is that the logical sum output of the signals is sent to the sensor signal processing unit.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a thermal image sensor system showing an example of an embodiment of the present invention. 1 is a thermal infrared matrix sensor detection unit,
It is divided into an edge pixel portion 11 arranged in the periphery and an in-area pixel portion 12 arranged inside. A sensor control circuit unit 2 controls the output of a sensor image signal from each pixel of the thermal infrared matrix sensor detection unit 1. 3 is a sensor signal processing unit, which is a binarization processing unit 31,
Edge pixel labeling processing unit 32, edge pixel frame difference processing unit 33, in-area pixel labeling processing unit 34, pixel label area calculation processing unit 35, calculation unit 36, in-area pixel frame difference processing unit 37, and pixel label data holding memory unit 38. The binarization processing unit 31 is for extracting the heat source by binarizing the sensor image signal output from the thermal infrared matrix sensor detection unit 1 with a predetermined threshold value. The edge pixel labeling processing unit 32
The same labeling is performed on consecutive pixels of the pixels extracted as the heat source by the edge pixel unit 11, and the heat source of the human body candidate is extracted. The edge pixel frame difference processing unit 33
The difference between the frames of the sensor image signal from the edge pixel unit 11 is taken, and when the heat source as a human body candidate disappears, it can be estimated that the heat source of the human body candidate has entered the area. The in-area pixel labeling processing unit 34 performs the same labeling on consecutive pixels among the pixels extracted as the heat source by the in-area pixel unit 12. The pixel label area calculation processing unit 35 uses the end pixel labeling processing unit 3
2 and the area of a pixel set to which the same labeling is performed by the in-area pixel labeling processing unit 34.
The calculation unit 36, based on signals from the edge pixel labeling processing unit 32, the edge pixel frame difference processing unit 33, the in-area pixel labeling processing unit 34, and the pixel label area calculation processing unit 35, the presence, number, position, etc. of the human body. Is to judge.
For example, when the area of the pixel set labeled by the edge pixel labeling processing unit 32 is calculated by the pixel label area calculation processing unit 35 and it is determined that this area is a predetermined size corresponding to the size of a human being. Recognizes that this heat source is a human body candidate, then recognizes that the heat source that has become a human body candidate is extinguished by a signal from the end pixel frame difference processing unit 33, and further recognizes it from the in-area pixel labeling processing unit 34. When a heat source appears in the area due to the image data and there is a correlation between the pixel data of the end pixel labeling processing unit 32 and the pixel data of the in-area pixel labeling processing unit 34, the heat source as a human body candidate is present in the area. It is judged that it has entered, and that this is a human body. Edge pixel part 1
When the area of the labeled pixel set of 1 is not a predetermined size, or when the position of the labeled pixel set does not move as a result of sampling a predetermined number of times, it is determined that the heat source is not the human body, The label is erased.

Further, the in-area pixel frame difference processing unit 3
Reference numeral 7 is a difference between frames of the sensor image signal from the in-area pixel unit 12, and movement of the labeled pixel group can be traced based on the result of the difference. The pixel label data holding memory unit 38 stores the labeled pixel data.
Even if the human body is tracked by the image data of the in-area pixel frame difference processing unit 37 and the human body is shielded by a shielding object on the way, the pixel label data holding memory unit 38
Then, the image data immediately before that is stored, and when the heat source appears again, the tracking of the human body can be continued by referring to the image data stored in the pixel label data holding memory unit 38. You can do it.

A plurality of thermal infrared matrix sensor detecting units 1 for monitoring the same area may be provided. In this case, each thermal infrared matrix sensor detecting unit 1 may be provided.
The reliability can be improved by performing the signal processing in the sensor signal processing unit 3 based on the logical sum of the binarized sensor image signals from the above.

Next, the operation of this embodiment will be described with reference to the flowchart of FIG. First, the sensor image signal from the thermal infrared matrix sensor detection unit 1 is binarized,
The end pixels are labeled (if there are a plurality of thermal infrared matrix sensor detection units 1, the end pixels are labeled for the logical sum), and the area of the pixel set with the same labeling is appropriate. If the label is not large or the position of the label does not move after a predetermined number of times of sampling, it is determined that it is not a human body and the label is erased.

Next, when it is determined that the heat source has disappeared from the edge pixel and the human body may have moved into the area, the in-area pixel is provisionally labeled and the in-area pixel and the edge pixel are correlated. If there is, the human body is judged to have entered the area, and labeling is performed again. further,
By tracking the label by taking the frame difference of the pixels in the area, even if the human body is blocked by a shield and reappears during the process, the labeling data is retained and the same label can be referenced. Even if a heat source other than the human body, such as a personal computer, fluorescent lamp, or sunlight, appears in the area, it is possible to track the presence of people, the number, and It is possible to accurately detect advanced human body information such as.

Hereinafter, the thermal image sensor system described above will be described with reference to FIG.
An example applied to the lighting control system shown in FIG. That is, the present invention relates to the lighting control of a large number of lighting fixtures 4 installed in a large indoor space such as a library or a conference room. By using the thermal infrared matrix sensor detection unit 1 corresponding to each pixel, it is possible to save unnecessary energy as much as possible and perform fine lighting control for a person in the room. FIG.
As shown in (a), the divided areas 5 of the room corresponding to each lighting fixture 4 are set so that there is no dead zone, and as shown in FIG. 4 (b), they correspond to the areas where the human body is located. The lighting equipment 4a is fully illuminated, the surrounding lighting equipment 4b is semi-illuminated, and the other lighting equipment 4c is stopped (off).
Control to turn on. Furthermore, in detail,
First, as shown in FIG. 5, the target room is divided into areas 5 each having the number of pixels of the thermal infrared matrix sensor detection unit 1, and one area 5 is provided for each pixel of the thermal infrared matrix sensor detection unit 1. Be prepared to respond. 20m
When the thermal type infrared matrix sensor detection unit 1 of 64 × 64 pixels is used in four rooms, each pixel is about 30 cm
It is divided into four areas. Each pixel of the thermal infrared matrix sensor detection unit 1 outputs 1 if there is a heat source in the corresponding area 5, and outputs 0 if there is no heat source. FIG. 6 shows respective outputs and their logical sums when two thermal infrared matrix sensor detection units 1A and 1B are provided in order to improve reliability. By providing a plurality of thermal infrared matrix sensor detection units 1 at different angles with respect to the area, a plurality of thermal infrared matrix sensor detection units 1A for one area of the room,
By associating the pixels of 1B with each other and taking the logical sum of them, even if the detection area of one of the thermal infrared matrix sensor detection units 1A is blocked by an obstacle such as a pillar or a desk, the other thermal infrared ray is detected. The matrix sensor detector 1A can detect the human body.

FIG. 7 schematically shows a process in which a human body enters the area from the edge of a room, is judged as a human body, and is labeled. First, as shown in FIG. 7A, when the human body 6 comes in from the edge of the room, only the edge pixels corresponding to the edge area detection area 5a are labeled as human body candidates. At this time, comparing the areas of the labeled pixels,
Since each pixel corresponds to 30 cm square, when the area of the label is 3 pixels or more, it is not judged as a human body and the label is erased. When the area of the label is less than 3 pixels, it is determined that the body is a human body, and the frame difference of the end pixel portion is taken to determine whether the heat source is moving or stopped. When it is determined that the heat source is not moving even after the label is operated 64 × 64 pixels / sec 600 times, it is determined that the heat source is not a human body and the label is erased. Further, when the heat source of the end pixel disappears due to the frame difference processing of the end pixel, it is possible that the human body has moved into the area.
In 6, the pixels in the area corresponding to the area detection area 5b are labeled. As shown in FIG. 7B, when the human body 6 enters the area detection region 5b, the position of the pixel labeled by the in-area pixel labeling process is adjacent to the position of the labeled pixel of the end pixel. Alternatively, if the distances are within 5 pixels, it is determined that there is a correlation between the two, the heat source is determined to be a human body, and labeling is performed again as a human body detection label. Since this human body detection label is tracked by the frame difference, it is possible to detect even if the human body stops in the area, and a heat source such as a personal computer or a fluorescent lamp suddenly appears in the detection area. Even if the heat source is moved,
Since there is no correlation with the end pixels, it can be distinguished from the human body, and thus no malfunction in illumination control will occur. Further, as shown in FIG. 8A, when the human body 6a with the human body detection label disappears from the end portion in the area, it is determined that the human body 6a has left the room. Erase the label. However, as shown in FIG. 8B, when the human body detection label 7 disappears in the center of the area, the human body 6a is placed on a shield such as a desk or a chair.
May be blocked. In this case, the label data is held in the pixel label data holding memory unit 38,
As shown in FIG. 8C, it is determined that the moving heat source that has appeared in the area again is the same human body 6a, the label data is called from the pixel label data holding memory unit 38, and tracking is performed with the same label. Here, another heat source that is not the human body 6
Even if b appears, the heat source such as a personal computer and a fluorescent lamp does not move, so labeling is not performed. Whether or not the reappearing heat source moves can be determined by the frame difference data in the area. Even if the human bodies overlap, the labels are tracked by the intra-area frame difference data, and therefore the number of people and the positions are not erroneously recognized.

By performing the above-described processing, the number and position of human bodies in the room are accurately detected, the lighting equipment corresponding to the area where the human body exists is fully illuminated, and the surrounding lighting equipment is semi-illuminated. By doing so, energy can be saved.

Next, a thermal image sensor system according to another embodiment of the present invention will be described. As the thermal infrared matrix sensor detection unit 1, only one column of the matrix is used as the end pixel unit 11a, not the surroundings, and the other pixels are used as the in-area pixel unit 12a. Then, as shown in FIG.
The area 5 on the side where the window W of the room is installed is the end pixel portion 11a.
If it corresponds to 1a, only the person who enters through the window W is a monitoring target, and the person who inputs and outputs through the door D on the opposite side is not detected as a human body. Therefore, as shown in FIG. 11A, when only an illegal entry from the window W is detected and an alarm is issued, sunlight is inserted from the window W or a small animal jumps in from the window W. The label may be erased by distinguishing it from the human body by the area of the labeled pixel, the moving speed of the heat source, or the like. Further, when a person approaches the vicinity of the window W from the room or opens the window W, the edge pixel portion 11a responds, but in this case, as shown in FIG. Since the label of 11a and the label of the in-area pixel portion 12a are correlated and the moving direction is detected from the in-area pixel frame difference data and the label is erased, no alarm is issued. Therefore, only illegal trespassers can be detected.

[0021]

As described above, according to the first aspect of the present invention, a thermal type infrared matrix sensor detecting section in which pixels for detecting infrared rays are arranged in a matrix, and the thermal type infrared matrix sensor detecting section are provided. In a thermal image sensor system comprising a sensor control circuit unit for controlling the sensor image signal from the thermal infrared matrix sensor detection unit and performing signal processing to detect a human body, Each pixel of the thermal infrared matrix sensor detection unit is divided into an edge pixel unit arranged in the periphery and an in-area pixel unit arranged inside, and in the sensor signal processing unit, from the thermal infrared matrix sensor detection unit Binarization processing unit that performs a binarization process for extracting a heat source from the sensor image signal of the above, and heat as a human body candidate based on the signal from the binarization processing unit. End pixel labeling unit for performing the labeling for detecting the difference, an end pixel frame difference processing unit for obtaining the inter-frame difference of the heat source as the labeled human body candidate, and labeling for the heat source in the area pixel unit. Since it has an in-area pixel labeling processing unit and a calculation unit that determines the human body based on the signals from the respective units, a thermal image sensor system capable of detecting the human body separately from other heat sources. Was able to provide.

According to the invention described in claim 2, in the invention described in claim 1, when there is a correlation between the output pixel signals of the end pixel frame difference processing section and the in-area pixel labeling processing section by the calculation section, Determines that the labeled heat source as a human body candidate has moved from the edge pixel portion to the in-area pixel portion, and determines that the heat source as the human body candidate is a human body. Can be done.

According to a third aspect of the present invention, in the first or second aspect of the present invention, an intra-area pixel frame difference for performing a frame difference process for detecting the movement of the heat source in the in-area pixel section. Since a processing unit is added and the heat source is traced by the arithmetic unit using the frame difference data from the in-area pixel frame difference processing unit, the human body can be traced.

According to a fourth aspect of the present invention, in the first to third aspects of the present invention, a pixel label area calculation processing unit for obtaining the area of the labeled pixel set is added, and the arithmetic unit calculates the pixel Since it is determined whether the heat source is the human body based on the size of the area obtained by the label area calculation processing unit, the human body can be accurately determined with respect to the heat source such as a small animal.

According to a fifth aspect of the present invention, in the first to fourth aspects of the invention, the heat source as the human body candidate labeled by the end pixel labeling processing section by the computing section is sampled a predetermined number of times. If there is no change, it is determined that the heat source is not the human body, so that it is possible to distinguish between the heat source such as sunlight and fluorescent light and the heat source from the human body.

According to a sixth aspect of the present invention, in the first to fifth aspects of the present invention, a plurality of thermal infrared matrix sensor detecting portions are arranged, and the sensors from the respective thermal infrared matrix sensor detecting portions are arranged. Since the logical sum output of the image signals is sent to the sensor signal processing unit, even if the detection area of one thermal infrared matrix sensor detection unit is blocked by an obstacle or the like, another thermal infrared matrix sensor detection The detection reliability is improved by the detection by the part.

[Brief description of drawings]

FIG. 1 is a block diagram of a thermal image sensor system showing an example of an embodiment of the present invention.

FIG. 2 is a flowchart showing the operation of the above.

FIG. 3 is a schematic diagram showing an example in which the thermal image sensor system of FIG. 1 is applied to illumination control.

FIG. 4 is a schematic diagram showing an operation of the lighting fixture according to the above.

FIG. 5 is a schematic diagram showing a correspondence relationship between a detection area and each pixel of a thermal infrared matrix sensor detection unit.

FIG. 6 is a schematic diagram showing a sensor image signal when two thermal infrared matrix sensor detection units are used.

FIG. 7 is a schematic diagram illustrating a determination method in a calculation unit.

FIG. 8 is a schematic diagram illustrating a determination method in a calculation unit.

FIG. 9 is a schematic diagram showing a thermal infrared matrix sensor detection unit according to another embodiment of the present invention.

FIG. 10 is a schematic diagram showing a correspondence relationship between a thermal infrared matrix sensor detection unit and areas according to the above.

FIG. 11 is a schematic diagram showing an operation according to the above.

[Explanation of symbols]

 1 Thermal Infrared Matrix Sensor Detection Section 2 Sensor Control Circuit Section 3 Sensor Signal Processing Section 11 Edge Pixel Section 12 Area Pixel Section 31 Binary Processing Section 32 Edge Pixel Labeling Processing Section 33 Edge Pixel Frame Difference Processing Section 34 Area Pixel Labeling processing unit 35 Pixel label area calculation processing unit 36 Calculation unit 37 In-area pixel frame difference processing unit 38 Pixel label data holding memory unit

Claims (6)

[Claims] 1. A thermal type infrared matrix sensor detecting section in which pixels for detecting infrared rays are arranged in a matrix, a sensor control circuit section for controlling the thermal type infrared matrix sensor detecting section, and a thermal type infrared matrix sensor detecting section. In a thermal image sensor system including a sensor signal processing unit that takes in a sensor image signal from the unit and performs signal processing to detect a human body, each pixel of the thermal infrared matrix sensor detection unit is arranged in the periphery. 2 for extracting a heat source from the sensor image signal from the thermal infrared matrix sensor detection unit in the sensor signal processing unit.
A binarization processing unit that performs a binarization process, an end pixel labeling unit that performs labeling to detect a heat source as a human body candidate by a signal from the binarization processing unit, and a frame of the heat source as a labeled human body candidate. An end pixel frame difference processing unit that obtains an inter-difference, an in-area pixel labeling processing unit that labels the heat source in the in-area pixel unit, and an arithmetic unit that determines a human body based on signals from the respective units. A thermal image sensor system comprising. 2. The calculation unit, when there is a correlation between the output signals from the edge pixel frame difference processing unit and the in-area pixel labeling processing unit, the labeled heat source as a human body candidate is output from the edge pixel unit. 2. The thermal image sensor system according to claim 1, wherein it is determined that the heat source as the human body candidate is the human body, and that the heat source is the human body candidate. 3. An in-area pixel frame difference processing section that performs frame difference processing for detecting movement of a heat source in the in-area pixel section is added, and the calculation section causes the in-area pixel frame difference processing section to The thermal image sensor system according to claim 1 or 2, wherein the heat source is tracked by using the frame difference data of. 4. A pixel label area calculation processing unit for calculating the area of a labeled pixel set is added, and in the calculation unit, whether the heat source is a human body according to the size of the area calculated by the pixel label area calculation processing unit. 4. The thermal image sensor system according to claim 1, wherein it is determined whether or not the thermal image sensor system is determined. 5. The calculation unit determines that the heat source labeled as a human body candidate by the end pixel labeling processing unit is not a human body when there is no change for a predetermined number of times of sampling. The thermal image sensor system according to any one of claims 1 to 4, which is characterized. 6. A plurality of the thermal infrared matrix sensor detection units are arranged, and a logical sum output of sensor image signals from each thermal infrared matrix sensor detection unit is sent to the sensor signal processing unit. Claim 1 characterized by
A thermal image sensor system according to claim 5.