JP2978374B2 - Image processing device, image processing method, and control device for air conditioner - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for detecting information on a person or an environment from a thermal image obtained mainly from an infrared sensor. It can be used in many industrial fields, such as equipment, intelligent building systems (IBS). Further, the control device for an air conditioner of the present invention can be used for an air conditioner that uses information on people and the environment mainly obtained from an infrared sensor as a control index.

[0002]

2. Description of the Related Art A first field to be solved by the present invention relates to device control using information obtained by using a thermal image of an infrared sensor and a person obtained therefrom. 2. Description of the Related Art Conventionally, there has been an apparatus that detects a position of a person using an infrared sensor and performs air conditioning control. For example, Japanese Patent Application Laid-Open No. 2-143047
Is known. The configuration will be briefly described below. The room is divided into a plurality of areas, and an infrared sensor for detecting infrared rays in each of the areas is provided to detect a person in each area and a temperature of an area corresponding to a floor surface. Then, based on the relationship between the position where the person is detected, the temperature of the floor near the person, and the room temperature, set temperature control is performed so that the room temperature normally reaches the set temperature. Change based on If the room temperature falls within the set temperature range and the difference between the floor surface temperature and the room temperature is large, the floor surface is blown to warm the floor surface. This allows
Control corresponding to the region where the person is located and control for reducing discomfort when the temperature has reached the set temperature but there is cooling near the feet of the person are performed.

[0003] In addition to the air conditioner, accurate information about humans is required in a surveillance system or the like. A method for detecting information about humans is to install a visible surveillance camera to monitor by humans or use video. It is common to record. Although there are monitoring systems using image processing, detection is often difficult due to problems such as illuminance and color. In addition, there are surveillance systems that use infrared cameras. However, in order to obtain a high-resolution thermal image, a large number of detecting elements or a cooling device is generally required, and complicated processing and high cost are required. Is required.

A second field to be solved by the present invention relates to a control index for air conditioning. In recent years, as for the air conditioning control index, not only a temperature index but also a PMV (ISO 7730) that takes into account factors related to comfort other than temperature.
(1984). For example, a configuration described in an embodiment of Japanese Patent Application Laid-Open No. 4-84055 is known. The configuration will be described below. Air temperature, radiation temperature, humidity, measure the airflow as the surrounding environment side element, and also estimate the amount of movement of the person, the amount of clothing as the human side element, and perform control based on the human side element and the environment side element, or , A PMV as a comfort index is calculated, and control is performed based on the calculated PMV. In this case, the method of deriving the input element is to measure all the surrounding environment-side elements using a sensor that measures each element, which leads to an increase in cost in commercialization,
In addition, there are many elements that are difficult to measure in terms of the measurement method.
Guess V. In other words, using a radiation sensor or an airflow sensor for PMV measurement leads to an increase in cost as a product, and arranging a sensor in a room is actually
Have difficulty. Even if the detection and estimation of these multiple elements can be performed, complicated calculations are required to calculate the PMV. In order to improve these points, six elements such as outside air temperature, room temperature, room temperature change over time, air volume, set temperature, and human position, which are relatively easy to measure, are input, and the PMV at that time is used as neural data in advance as teacher data. By making the network learn, a neural network for outputting the PMV with respect to the six types of inputs is formed, the PMV is estimated using the neural network, and the estimated PMV is used as a control index for air conditioning. In this case, since the teacher data used for learning has a large number of cases, the activity amount and the clothing amount use fixed values of a plurality of combinations.

A third field that the present invention seeks to solve is:
It is related to air conditioning when not present. Conventionally, a human detection sensor is provided or an absent button is provided in the air conditioner operating means, and when the air conditioner in which a person is absent is determined, control is performed such that a predetermined temperature level is maintained at a minimum room temperature. There was something to do. As a result, it was possible to prevent the building frame from being overcooled during heating and from being overheated during cooling, and to reduce discomfort when entering the room. In addition, since there is an unpleasant time from the natural room temperature or the lowest temperature level during absence or bedtime to the set temperature, some timers use the timer to reach the set temperature at the time of entry or wake-up. Was.

[0006]

However, the control device for an air conditioner using an infrared thermal image shown in the prior art is effective for detecting the direction in which a person is present, but is comfortable for individuals in a room. It has been difficult to detect accurate information on the position of an individual, the determination of the posture of the individual, and the like for accurately calculating the degree.

Further, since the measurement range is only a scene fixed in advance, if the room is different from the assumed room, the wall may be erroneously recognized as the floor, so that an accurate floor surface temperature may not be detected. There were challenges.

That is, these problems are roughly divided into the following two problems. (1) Image resolution is low (2) Individual or indoor environment information cannot be detected accurately (1) is a problem caused by the resolution of an image reading sensor. If the resolution of the image is low, it is difficult to accurately detect individual and indoor environment information. Therefore, in the present invention, it is desirable that the image has such a resolution that one person in the room is detected as a cluster of a plurality of pixels.

[0009] Next, the problem (2) arises even in a high-resolution image in which the problem (1) does not matter because the personal information is accurately detected and a control device based on the information is not used. It is. In the present invention, (2) is raised as a major problem of the conventional method.

With respect to the control device for an air conditioner shown in the second field of the prior art, the method of estimating comfort using a neural network estimates the average comfort of the entire room. It was not a control index that considered individual comfort. As an input element for calculating the degree of comfort, a method for detecting airflow, radiation, or the like may be considered in which a sensor for measuring each element is arranged, but a plurality of sensors are arranged indoors in an actual home. It is difficult. In addition, it is conceivable to install a sensor for measuring these surrounding environment-side elements on a remote controller or the like. In addition, as for the human element, the momentum that is the human element can be estimated from the number of detections of the peak value of the output of the infrared sensor, as is already used in some products, and the activity level of the whole room can be detected However, in the conventional method, for example, it is presumed that the activity level is high even in a situation where five occupants are in the room and four are stationary and one person is moving around the room.
There is a problem in the accuracy of activity amount detection for each individual. There is also a method of estimating the amount of clothing from sunlight, calendar, outside temperature, etc., but it is not actually measuring the human body, so it is far from the amount of clothing of the person who is actually in the room There was also a problem in terms of accuracy.

In the air conditioner control device shown in the third field of the prior art, control has been performed to maintain a certain minimum temperature in the absence of the air conditioner. It took a long time, during which time there was a problem of being uncomfortable. Also, with respect to the minimum temperature level to be maintained, in the case of heating, if it is too high, the electricity cost is uneconomical, and if it is too low, there is a problem that it takes a considerable time to reach the set temperature. On the other hand, the timer has a problem in that it is troublesome to set and reset the timer by the time input format, and there is a problem that the change of the time setting is small, and the timer is not used much except when the wake-up is effective.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems of the accuracy of detecting the position of a human body, the accuracy of detecting an input element for calculating a degree of comfort, and the inconvenience of a timer set. A second object is to detect human personal information and detailed environmental information in a detection area using an infrared thermal image. The second object is to detect air information based on human information in a detection area and detailed environmental information in a room. A third object of the present invention is to determine fine control indices of the harmony device, and to eliminate a timer operation.

[0013]

Means for Solving the Problems To achieve the above object, the technical solution of the present invention relates to an image processing apparatus comprising an infrared image (hereinafter, referred to as an infrared image) for measuring at least the temperature of a person and an environment in a detection area. A sensor unit having input means and a thermal image having a pixel value of a detected temperature output from the sensor unit are processed to extract a feature amount of an image used in an image information detecting unit at the next stage. An image processing unit, having a neural network or a pattern recognition mechanism as a component, and based on a signal from the image processing unit, a human personal information signal or an environmental information signal equivalent or more detailed than an output signal from a sensor unit. , An image information detecting section for estimating and outputting the information by the neural network or the pattern recognition mechanism.

The image processing section includes a human area correcting section for complementing and erasing a human pixel block obtained by the human area detecting section at a pixel level, and a representative of a human pixel block obtained by the human area detecting section. Accumulation processing unit that accumulates points for a predetermined period, and at least floor and wall, or room representative points that are boundaries between walls are determined from the accumulation of representative points of the human pixel block obtained from the accumulation processing unit for a predetermined period, It is also possible to adopt a configuration in which an indoor part determining unit that associates the correspondence between each pixel on the thermal image and the floor and wall in the room is added. .

[0015]

According to another aspect of the present invention, a control unit for an air conditioner includes the image processing device as a component, and further includes a sensor unit for detecting temperature, humidity, and the like. It has a control index determining unit that determines a control index for performing air conditioning from a personal information signal obtained from the device and an environmental information signal, and an air conditioning control unit that controls air conditioning equipment. The control index determination unit includes: an activity amount detection unit that detects an activity amount representative of an individual or a room from the personal information output from the personal information extraction unit of the image processing apparatus; the personal information; and the environment information extraction. Environmental information obtained from the unit, wind direction and air volume information obtained from the air conditioner, from the activity amount obtained from the activity amount detection unit, has a comfort level detection unit for detecting individual or representative comfort level And outputs a control target value of the air conditioning control unit from the comfort level.

Further, a life scene estimating unit for estimating a living scene from the activity amount obtained from the activity amount detecting unit, time, and environmental information output from the environmental information extracting unit is added to the control index determining unit. Estimate a transient life scene in which the comfort level is not a control index and a stable life scene in which the occupant is in the room for a predetermined period and the comfort level can be used as the control index, and according to the life scene. A predetermined control index that is predetermined is selected for each comfort level and each scene.

In order to achieve the third object, the occupancy data storage means, the occupancy pattern classification means, and the base temperature which is the minimum maintenance temperature level of the absence time can be obtained from the occupancy pattern classification means. It has an absent-time control index determining unit and a calendar, each of which has a configuration of base temperature conversion means determined according to the use state of the room.

[0019]

According to the image processing apparatus of the present invention,
First, the image is input as a thermal image by the infrared image input means of the sensor unit. Based on the input image data, the human region detection unit performs extraction of the human body part based on the extraction temperature of the human body part, and calculates these human pixel blocks and their representative points. Next, for example, regarding a method of detecting a position and a posture of a person, which is one of the signals detected by the image processing apparatus, the image processing unit uses the human pixel block and its representative point and the surrounding environment of the representative point. And the feature amount for position determination and the feature amount for posture determination. Next, the image information detection unit detects the position and orientation of a person by using the above-mentioned respective characteristic amounts as input. The detection of the position and the posture of the person is performed by the posture detecting means and the foot position detecting means each constituted by separate neural networks. Further, in the foot position detecting means, according to the type of the posture, for example, the classification is performed in accordance with the number of postures to be distinguished, such as for standing and for sitting, and a determination is made from the neural network for estimating the foot position for each posture. The output is selected and sorted by the output from the attitude detecting means. By performing such processing, more accurate position detection becomes possible.

Further, in an actual home, it is conceivable that there is an obstacle such as furniture between the human body to be detected and the sensor. In this case, it is erroneously assumed that the human body is separated and a plurality of occupants are present. The human area correction unit searches around the pixels cut out as a human pixel block, complements the pixels, or deletes pixels considered as noise. Further, in order to remove a stationary heating object such as a television, the accumulation processing section accumulates the representative points of the cut out human pixel block for a predetermined period, and when only a specific pixel exceeds a preset frequency threshold value, Assuming that there is a heating object, the position on the thermal image is stored, and when the representative point of the human pixel block matches the position of the still heating object at the time of detecting a human area, the human pixel block is treated as a human body. Perform no processing. This enables highly accurate detection of the human body. In addition, the accumulation processing unit accumulates the representative points of the human pixel block for a period longer than the accumulation period used for the detection of the stationary heat-generating object, and calculates the three walls and the floor except for the wall on which the air conditioner is installed. A representative point for specifying the boundary is detected, and the indoor part determining unit associates the floor surface and the wall surface in three directions with the correspondence of the thermal image based on the representative point. As a result, the environment information extraction unit detects the temperature of each part of the floor and the walls in three directions even in rooms having different sizes, and furthermore, by using the foot position information of the person, accurate radiation for each individual can be obtained. The temperature can be detected.

Further, it is difficult to detect environmental information in the detection area of the thermal image input means, for example, structural information such as the size of the room and the position of the wall surface with respect to the sensor if the room is indoors. Can be directly input from the console unit and used as auxiliary information for accurate position detection and posture detection.

Next, the image processing apparatus of the present invention not only processes a thermal image but also employs a system using a visible image input device (camera) so that human-related information obtained from the thermal image can be visualized. It can be conveyed on the image to provide the user with the position of the person in the visible image.

In the opposite case, it is possible to detect the position of a person on the thermal image using information of the person obtained from the visible image, for example, motion information, and to perform each processing of the thermal image and the visible image. Adaptive learning is performed on the state of the neural network in the image information detection unit that is performing using each other's information as a teacher, and the performance can be further improved.

Further, by using a plurality of image processing apparatuses that combine the thermal image processing and the visible image processing of the present invention, it is possible to more accurately detect information about a person in an area where the plurality of apparatuses are installed. Although it is difficult for one image processing apparatus to accurately separate people who overlap, this problem can be solved by using a plurality of apparatuses. In addition, a person can be tracked in a wide area.

Next, with respect to the control device of the air conditioner, according to the present invention, first, with this configuration, the image processing device first detects the information of the person and the environment in the detection area. In the control index determination unit, for example, when the degree of comfort is used as the control index, the personal information extracted by the personal information extraction unit of the image processing apparatus and the individual information based on the environment information calculated by the environment information extraction unit. The degree of comfort is calculated, and when there are a plurality of occupants, a control index of the wind direction, the amount of air, and the degree of comfort is calculated so that the occupant having a low degree of comfort is preferentially comfortable. The air-conditioning control unit performs, for example, control of the compressor frequency, control of the vertical and horizontal flaps, and control of the indoor unit fan based on the instruction of the control index determination unit.

Although the control index determining section has shown an example in which the comfort level is used above, the addition of a life scene estimating section makes it possible for a transition such as sleeping, getting up, entering and leaving the room to become a control index. And a stable scene where the degree of comfort after a person has entered the room for a while becomes a control index,
A predetermined control index is used as a control index for a transient scene according to each scene, and a calculated comfort level is used as a control index for a stable scene. When only the scene estimating unit is used as the control index, in addition to estimating the transient scene, the stable scene is further cleaned and cleaned according to the activity amount representing the room calculated from the number of persons and information of each individual. , Relaxation, etc., and control indices predetermined for each scene are used.

Next, with this configuration, the presence / absence of an occupant in the room is initially set to 1 when the occupant is present and 0 when the occupant is not present by the number detection unit at predetermined time intervals. Since the data of 0 and 1 greatly change when a person enters and exits the room, a correction is made such that 10 minutes is set to 1 unit, and for example, 0 is set only when 0 is continued for 10 minutes or more. The occupancy data storage means stores the supplemented occupancy data for one day for a certain period from the current day. The occupancy pattern classification means collects occupancy data on the same day and the same day attribute based on information on the attributes of the day such as weekdays, holidays, and days of the week from the calendar, and occupies the occupancy rate by time (the occupancy pattern and the occupancy pattern). Call). The base temperature conversion means calculates the value of the base temperature based on the occupancy pattern. In the air conditioning control means,
Based on the output of the human detection means, the target temperature is set to the target temperature while the person is in the room, and the target temperature is set to the target temperature for each time from the current time when the person is absent. Switch the target value to. As a result, since the indoor environment is on standby before a person enters the room, the air conditioner is excellent in comfort, and the air conditioner is economically excellent since the base temperature is low during a time when there are many absences.

[0028]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An image processing apparatus according to a first embodiment of the present invention will be described below.

FIG. 1 is a block diagram of the image processing apparatus of the present embodiment. Reference numeral 1 denotes a sensor unit for measuring a thermal image. Reference numeral 2 denotes an image processing unit that detects a human region and a feature amount from the thermal image. Reference numeral 3 denotes an image information detection unit that estimates a human posture, a foot position, and the like from a thermal image. The image information detection unit 3 is configured by a plurality of neural networks and the like.

FIG. 1 shows a basic flow of information shown in FIG. Outside world 4
Is a region where a human is located, and is scanned by a two-dimensional infrared sensor 11 included in the sensor unit 1 to detect a temperature of an area that can be detected as thermal image information. In some cases, a room as an area where people live is shown. The heat distribution of the external world 4 is scanned by the two-dimensional infrared sensor 11 and input to the image processing unit 2 as a thermal image 6. Since the entire thermal distribution including the person in the scanning area becomes an image in the thermal image 6, the image processing unit 2 detects the region of the person. The image processing unit 2 creates feature amount data of the person's area to be transferred from the person's area to the subsequent image information detection unit 3, and is input to the image information detection unit 3 as feature amount data 7. The image information detection unit 3 estimates a personal information signal 8 such as a posture determination and a foot position from the feature amount data 7.

The operation of the air conditioner incorporating the image processing apparatus configured as described above will be described. First, a one-dimensional pyroelectric infrared sensor in which a plurality of elements are arranged in a vertical direction, for example, is installed in the body of a wall-mounted air conditioner, and scans the room in the frontage direction at a sampling timing for a predetermined time. Using the two-dimensional infrared sensor 11 configured in such a manner, a two-dimensional thermal image including the occupants in the room is obtained. FIG. 2A is an example of a measured thermal image in the first embodiment. In this case, three people are present in a quadrangular room, and each person is in a different posture. 21 is an image block of the occupant in the lying position, 22 is an image block of the occupant in the standing position, and 23 is an image block of the occupant in the chair sitting position. 24 is a floor image block, 25 is a left wall image block, 26 is a back wall image block,
Reference numeral 27 denotes an image block on the right wall surface. FIG. 2B is a plan view of the room at this time. 28 is a person in a lying position, 29
Is a standing occupant, 30 is a chair sitting occupant, 31 is a floor,
32 is a left wall, 33 is a back wall, 34 is a right wall, 35 is an air conditioner having the image processing apparatus as a component, 36 is an infrared two-dimensional sensor installed in the air conditioner 35, and 37 is an infrared sensor The scanning direction and the detection range of the two-dimensional sensor 36 are shown. Thus, by scanning the two-dimensional infrared sensor 36 in the frontage direction, a distant person is detected at the upper portion on the thermal image, and a near person is detected at the lower portion. Most of the floors and walls are detected on the thermal image, except for the wall of the air conditioner mounting surface.

Next, FIG. 1 will be described in detail. First, the image processing unit 2 will be described. The thermal image detected by the two-dimensional infrared sensor 11 is stored in the image storage unit 1 for performing subsequent processing.
Remember once in two. Next, in the human area detecting unit 13, FIG.
The configuration shown in FIG. 1 detects a human area. Conventionally, when detecting a human area from thermal image data, if it is assumed that there is no non-human heat generation in the room and that the temperature of the non-human room is 25 ° C. or less, a temperature of 26 ° C. to 34 ° C. The region of the human body has been detected by detecting pixels. However, with the above-described configuration, there are cases where a human cannot be cut out accurately from a two-dimensional thermal image. For example, the temperature distribution in a room may be different depending on the floor or each wall surface.
There is also a method of detecting a person from the boundary of the person without using the temperature range for cutting out the person. However, accurate detection is difficult if a good edge cannot be detected.

Therefore, in the present embodiment, the thermal image stored in the image storage unit 12 is divided into M (M is a positive integer) and divided into regions (hereinafter, each region is referred to as a block). The human cut-out temperature range is determined for each block to calculate the position of the human, and the cut-out temperature of the human is calculated in conjunction with the temperature distribution in the room, so that the detection of the human can be easily calculated. I made it.

In addition, a process of easily capturing a heat-generating object of a human or a living being, which is a characteristic of a thermal image, can be effectively used.
Further, when capturing the human as a heating object, and the human region to ambient temperature, Ri Do convexly case high, If no low is concave. Therefore, adjacent pixels of the thermal image are represented by G (n) and G (n).
(n + 1), a difference is obtained as D (n) = (G (n) -G (n-1)) / 2, and the absolute value of the difference D (n) (ABS (D
(n)) is detected as ABS (D (n))> TH greater than the appropriate threshold value TH as a boundary of a human area, and a human area is cut out based on the temperature inside and outside the boundary. There is also. In addition, if the entire thermal image is noisy, a method of capturing large irregularities in the thermal image is to remove the noise using a polynomial smoothing method as one of the well-known moving average methods in image processing. locate the unevenness corresponding to the human in the pixel, Ru method mower cutting the region based on the temperature of the portion of the irregularity. This method is described in "Waveform Data Processing for Scientific Measurement" published by CQ Publishing Co., Ltd.

As a method of connecting a pixel having a similar temperature around each pixel of the thermal image to form a human region, a simple region expansion method is used as a region dividing method disclosed in "Image Analysis Handbook" published by The University of Tokyo Press. There is also a method of extracting a heat distribution area on a thermal image surface using a method or an iterative area expansion method, and detecting an area having a size and a temperature distribution that can be determined as a human area among the areas as a human.

An example of the configuration of the human area detecting section 13 will be described with reference to FIG. In FIG. 3, reference numeral 40 denotes a region division representative temperature calculating means for dividing a thermal image into M blocks and calculating an average temperature of each block; Determining means, 42 is a human detecting means, 43 is a representative point calculating means for calculating a representative point of the human area.

The thermal image stored in the image storage section 12 is divided into M blocks by a region representative temperature calculating means 40, and the average temperature of each block is calculated. The calculated average temperature of the block is passed to a cut-out temperature determining unit 41 that determines a cut-out temperature range of a person, and a cut-out temperature range for each block is determined. Here, the reason why the block is divided into a plurality of M blocks is that it is difficult to accurately cut out the human body portion at a uniform cutout temperature such as when the wall is partially warm or when the clothes of the human body are cold. Therefore, by dividing the thermal image into M images, it is possible to cut out the image in consideration of the background temperature. Next, the cut-out temperature range and the thermal image are input to human detection means 42 for detecting a human area, and a human pixel block indicating a human area is detected. Representative point calculation means 43
In this example, a human pixel block is input, and a representative point representative of the block, for example, a center-of-gravity point pixel weighted by temperature, or a representative point such as a lowest point pixel corresponding to a foot position is detected. In addition, the coordinates of the pixel closest to the floor surface or the coordinates of the pixel farthest from the floor surface may be used as the representative point so that the calculation is reduced. Further, the position of the Y coordinate of the center of gravity and the position of X closest to the floor surface may be set as the position of the representative point. By knowing the representative points, it is possible to roughly determine where the human is and how many are.

FIG. 4 shows an example of dividing the thermal image data of the present invention into eight blocks and calculating the average temperature of each block. Area division representative temperature calculating means 40 for each block
Then, the average temperatures T1 to T8 are calculated. For example, when the average temperature TN of the block N is 20 ° C. <TN <22 ° C., the cut-out temperature range of the person in the block N is 25
The cutout temperature of a person is determined for each block, such as from 34 ° C to 34 ° C. In other words, the human exposed skin temperature is 36
The temperature is between 37 ° C and 37 ° C. However, if the clothing, background and human body are included in one pixel, they are averaged and the detection temperature becomes low. Become. In the present invention, an appropriate human cut-out temperature range is determined based on the average temperature value for each block and the position of the block in the entire image.

On the other hand, in the above configuration, in order to correctly extract a human area, it is necessary to set a temperature range for determining that a pixel includes a human body for each infrared element.
When the number of elements is large, it is difficult to set a temperature range for each detection element in terms of manufacturing the device, or an object having the same temperature range as a human, such as a television or a heater, is mistaken for a human. Had also. Therefore, there is a method of determining an appropriate human cut-out temperature range from continuous images. FIG. 5 shows a configuration of the human area detection unit 13 using such a method. 3 is different from the configuration of FIG.
The following is the same as the configuration of FIG. The operation will be described below with reference to FIG.

In this configuration, the image storage section 12 stores a plurality of thermal images continuously. The difference image creation means 50 creates a difference image between the stored plurality of thermal images. The moving object image creating means 51 creates a moving object image from the plurality of thermal images and the detected difference image.
Next, in the frequency distribution creating unit 52, the moving object image creating unit 51 obtains the pixel blocks of the image corresponding to the moving object from among the pixel values of the plurality of thermal images. Create a distribution. The cut-out temperature determining means 53 determines a cut-out temperature range in which a human region can be detected using an appropriate threshold value set therein and the frequency distribution obtained from the frequency distribution creating means 52. Thereafter, the human pixel block and its representative point are calculated by the human detecting means 42 and the representative point calculating means 43 shown in FIG.

Next, a detailed processing procedure will be described with reference to FIG. The image stored in the image storage unit 12 is an image obtained by continuously reading the temperature distribution in the room as a thermal image. In the present embodiment, the continuous measured thermal images are referred to as an image A and an image B, respectively.

First, the images A and B are read in order.
It is assumed that image. In the image, pixels existing at coordinates (x, y) are represented by P (x, y). A difference image is created from the image A and the image B by the difference image creation means 50. In this case, if the image contains a noise component, the threshold value h may be used. That is, the coordinates in the image A (x,
When the pixel value in y) is T1 (x, y), the pixel value of the same coordinates in the image B is T2 (x, y), the threshold value is h, and the value which cannot be absolutely determined as the pixel value is TN, If | T2 (x, y) −T1 (x, y) | <h, the value dT of the pixel at the same coordinates in the difference image is given as dT (x, y) = TN | T2 (x, y) −T1 ( If x, y) | ≧ h, dT (x, y) = T2 (x, y) −T1 (x, y) (| x | indicates that the absolute value of x is to be obtained). A differential image can be created by performing the process on all the pixels in the middle. FIG. 6 shows an example of creating a difference image from images A and B. By taking the difference between the two images, it can be seen that the area having a positive value is the area after the movement, and the negative area is the area before the movement. For TN, for example, if the temperature value is measured in degrees Celsius, TN = -1
Or a value less than absolute zero such as 000, or TN =
What is necessary is just to make it very high temperature, such as 10,000. Also the difference
If each pixel in the minute image has a temperature value other than TN, it indicates that the pixel has captured a moving object, and if the temperature value is TN, it indicates that a stationary object has been captured. I have. Next, a moving object image is created from the difference image 2 between the image A and the image B by the moving object image creating means 51. Creation of a moving object image is performed as follows. That is, when the temperature difference value of each pixel in the difference image is dT (x, y) and the temperature value of the pixel at the same coordinates in the moving object image is V (x, y), dT (x, y) <0 Then, if V (x, y) = T1 (x, y) dT (x, y)> 0, then V (x, y) = T2 (x, y) If dt (x, y) = TN, then V ( By performing a process such as (x, y) = TN for all pixels in the difference image, a moving object image can be created. next,
From the moving object image, the frequency distribution creating means 52 uses the frequency distribution H (x, y,
T) is created. The frequency distribution is created as follows. That is, when the pixel value at the coordinates (x, y) in the moving object image is V (x, y), and the frequency distribution of the pixels at the same coordinates in the thermal image is H (x, y, T), By performing the process of H (x, y, V (x, y)) ← H (x, y, V (x, y)) + 1 on all the pixels in the moving object image, each pixel in the image is obtained. A frequency distribution can be created for each temperature.

By performing the processing from the detection of the thermal image to the creation of the frequency distribution for all the measured thermal images, the frequency distribution is constructed for each pixel, and if sufficient time elapses, , The frequency distribution reflects the temperature range in which the temperature of the pixel can be determined to be human. However, depending on the accuracy of the two-dimensional infrared sensor 2, a frequency threshold may be provided as described below, and a temperature value having a small frequency may be excluded.

Further, a temperature range from which a human is cut out is determined by the cut-out temperature determining means 53 from the image A, the frequency distribution, and an appropriate frequency threshold value k contained therein. Then, the human detection means 42 calculates the temperature value of the pixel at the coordinates (x, y) in the image A as T1 (x, y) and the frequency distribution of the pixel as H (x, y, T). When the threshold value is k and the temperature value of pixels at the same coordinates is M1 (x, y) by the temperature cutout temperature determining means 53 indicating the temperature cutout range for each pixel, H (x, y, T1 (x, y)) <k M1 (x, y) = TNH (x, y, T1 (x, y)) ≧ k If M1 (x, y) = T1 (x, y) By performing for all pixels of
A human pixel block can be extracted from the image A.
In addition, a human pixel block is created by extracting only pixels representing a human from the image B by using the temperature determination unit 53 and the human detection unit 42 that are cut out from the image B, the frequency distribution, and the frequency threshold. In this process, a human region is extracted in the same manner as when a human pixel block is extracted from the image A.

Further, the minimum value and the maximum value are selected from the temperature values having the frequency equal to or higher than the frequency threshold value k. There is also a method of judging that there is. This method is effective when the apparatus has just started operation and the frequency distribution does not sufficiently reflect the temperature range in which it can be determined that the temperature of the pixel captures a person. With the configuration of the present embodiment described above, even when the variation between the detection elements of the sensor is large or when noise is added to the image, the human sets the temperature range in which it can be determined that the human is captured for each pixel. The temperature range in which the human region detection unit can automatically determine that a person is captured can be set for each pixel as the device is used.

In this embodiment, two continuous images are used. However, even when three or more thermal images are used, the same processing can be performed, and more difference images can be obtained. It has the advantage of faster construction.

In the embodiment, the frequency distribution arranged two-dimensionally is used. However, for example, the detection means in which the detection elements are arranged one-dimensionally in the vertical direction is mechanically scanned in the horizontal direction. When the two-dimensional thermal image detection means is used as a result, the frequency distribution need not be created for each pixel, but may be created for each row of the image. The extraction of the human region from the thermal image is performed according to the common temperature range for each row obtained from the frequency distribution. The same applies to the case where the detection means in which the detection elements are arranged one-dimensionally in the horizontal direction is mechanically scanned in the vertical direction. The extraction of the human region from the thermal image is performed by the common temperature range.
Further, when a single detecting element is mechanically scanned in the horizontal and vertical directions, resulting in a two-dimensional thermal image detecting means, it is not necessary to create a frequency distribution for each pixel. Needless to say, only one frequency distribution is required, and a human region is extracted from a thermal image by using a temperature range common to all pixels obtained from the frequency distribution.

Next, the configuration of the personal information extraction unit 14 of FIG. 1 will be described with reference to FIG. Personal information extraction unit 14
In the image information detection unit, a feature amount for estimating more detailed information about a person is determined from the human pixel block obtained from the human region detection unit and its representative point, and the thermal image stored in the image storage unit 12. calculate. Reference numeral 60 denotes a number-of-persons characteristic detecting unit, 61 denotes a posture determining characteristic amount detecting unit, 62 denotes a position determining characteristic amount detecting unit, 63 denotes a skin temperature determining characteristic amount detecting unit, and 64 denotes a clothing amount determining characteristic amount detecting unit.

First, a method for detecting the number of people will be described. Conventionally, when detecting the number of people in the image from the image,
A human-like area is detected from the image, and whether or not the area is a human is determined based on the shape of the area, and the number of persons is calculated based on the number of humans. However, with this method, it is difficult to detect a good number of people unless the shape of the region can be accurately recognized from the image. Also, when people overlap, it becomes difficult to calculate the number of people. Therefore, in the present embodiment, the number of persons is obtained by the number-of-pixels calculating means, the block form determining means, and the human pixel block counting means.

FIG. 8 shows the structure of the number-of-people detecting means 60. In FIG. 8, reference numeral 70 denotes a pixel number calculating unit, 71 denotes a block type determining unit, and 72 denotes a human pixel block counting unit. The number of humans is determined from the number of human pixel blocks in the room detected by the human region detection means 13 and the number of pixels in each block. The number of persons is determined using the number of detected pixel blocks. In addition, since there is a possibility that a human is detected overlapping, the number of persons is determined using the shape of each pixel block. In this case, the shape is different from the shape shown in FIG.
It is assumed that the image information is to be classified into portrait, landscape, and square shape types, and the number detection means using a neural network of the image information detection unit 3 is used. FIG. 9 shows a simple method of determining the number of persons when people overlap. Assuming that the number GA of pixels in the pixel block is a simple method of determining the number of persons, the number of persons is set to one when GA <TH1, and to two persons when TH1 ≦ GA <TH2. N thresholds (TH1, TH2,..., THN) are prepared in accordance with the number N of persons required for discrimination. However, TH1, TH2, ..., TH
N is a positive integer.

In this configuration, even in the image input means for inputting a visible image, the number of persons can be determined by judging the human area and the number of pixels constituting the area from the edges, lines and colors.

Next, in the above-described determination of the number of persons from the pixel block, the determination of the threshold value requires adjustment depending on the image input device, its installation position, image quality, image resolution, and the like. Therefore, in order to simplify such a regulation, there is a method to take advantage of the learning function, such as a neural network.
FIG. 10 shows an embodiment of the neural network. 73
Is a neural network for estimating the number of people. The neural network 73 receives the number of pixels of the human pixel block, the representative point, and the shape type obtained from the number-of-persons characteristic detection unit 60 as inputs, and calculates the number of people in the output. Various engineering models can be applied to the neural network 73. For example, when using a multilayer perceptron,
As a method of learning, back propagation (inverse error propagation method) is famous. In this learning, input information and desired output information at that time are supplied to the neural network, and the internal state is adjusted. In the above case, the feature signal 2,
3 and the human pixel block representative point and the number of persons in the room at that time are given to the neural network for learning. This configuration,
The work of empirically determining the threshold value for adjusting the internal state by learning can be reduced. Further, as an alternative to the neural network, a statistical process such as a regression analysis can be used.

Next, a method of detecting a foot position will be described. FIG. 11 is a diagram showing a configuration in the first embodiment for detecting a foot position. In FIG. 11, reference numeral 90 denotes a column data extracting unit for extracting column data, and 91 denotes a foot position estimating unit for detecting a foot position. The foot position estimating means 91 includes an algorithm method, a statistical method such as regression analysis, a neural network method, and the like. In this embodiment, a method using a neural network will be described.

The human pixel block detected by the human area detecting unit 13, the coordinates of the representative point of the pixel block, and the peripheral pixel values of the human pixel block on the thermal image are input to the position determining feature detecting unit 62. First, the human pixel block
The data is input to the column data extracting means 90. In the column data extracting means 90, one vertical column of data including the lowermost pixel of the human area is input to the foot position estimating means 91. When there are a plurality of pixels at the lowermost end, there are a method of extracting a column including a pixel having the highest pixel value, and a method of extracting the column based on the center and left and right positions. If the resolution of the thermal image is eight pixels vertically, eight temperature values represented by each pixel are extracted. Based on the column data extracted by the column data extracting means 90, the foot position estimating means 91 using a neural network is used to calculate the human foot position. The foot position estimating means 91 using the neural network performs learning by providing a sufficient number of data and an appropriate teacher signal using the back propagation method in advance, and it is assumed that the error is considered to have converged. . In this step position estimating means 91, in addition to the multilayer perceptron,
It is also possible to use other neural network models such as LVQ.

FIG. 12 is a schematic diagram showing the correspondence between a standing human thermal image and a human body. a to o represent pixels of the thermal image, and hatched portions represent regions corresponding to the human body. In this case, f, g, and h are cut out well because most of the pixels are occupied mostly by the human body, but in the case of i, the upper half is the temperature because the position of the foot is near the center of the pixel. And the lower half has a lower temperature. At this time, since the area of the human body portion occupying one pixel is small, the temperature of the human portion and the temperature of the background portion overlap, and as a result, the pixel value becomes approximately intermediate, and falls below the cutout temperature range. Only three pixels, f, g, and h, are obtained. Even if the foot position is estimated from this, it cannot be accurately obtained, but by inputting the value of i to the neural network,
An accurate foot position can be obtained.

In the case of using an algorithm as a method other than the neural network, the foot position estimating means 9
1, using only the values of the three pixels h, i, and j, h
Is the temperature of the human, and j is the temperature of the background. If α is the ratio of the human to the pixel i, α is obtained from i = α * h + (1−α) * j, and based on that, The position can also be calculated.

FIG. 13 is a diagram for explaining the characteristic amount in the position determination characteristic amount detecting section 62. The outside world information 110 including the person is addressed as a thermal image 112 by the infrared sensor 111, and the person's block in the hatched portion is cut out by detecting the region of the person. The center of gravity of the detected person's block temperature value (pixel value) is obtained, and the value of the pixel A at the lowest point of the person's block whose center of gravity is lowered in the vertical direction is defined as the pixel value (A). The value of the pixel B one immediately above the pixel in the vertical direction is the pixel value (B), and the pixel C one immediately below the pixel in the vertical direction.
Is the pixel value (C), and the value of the adjacent pixel D is the pixel value (D). In addition, the background temperature of the human block, or the average value of several pixels in the horizontal direction of the pixel C is used as the foot temperature, the average temperature of the entire human block, and the reference temperature inside the sensor as the feature amount. calculate. These feature amounts are input to a position estimation neural network 113 inside the image information detection unit 3 to estimate a position 114 of a person.

Next, a method of detecting a posture will be described.
In the posture detection, three types of postures to be detected are provided: standing, lying, and sitting. FIG. 14 is a diagram showing a configuration in the first embodiment for detecting a posture. In FIG. 14 , reference numeral 120 denotes a human pixel block obtained from the human region detection unit 13 and a rectangular block including a human from the thermal image, and side pixel number detecting means A and 121 for detecting the vertical size of the rectangular block. Side pixel number detecting means B 122 for detecting the horizontal size of the rectangular block is a converting means for standardizing the rectangular block into a block of 3 × 3, and 123 is a posture estimating means. Posture estimating means 123
There are various methods such as statistical processing and a method using a neural network. In this embodiment, a multilayer perceptron that has been learned in advance by a back propagation method is used.

The human pixel block detected by the human area detection unit 13 is extracted from the human pixel block and the thermal image by the side pixel detection means A and B of 120 and 121, and a rectangular block including the periphery is extracted. The number of horizontal pixels is detected, and the number of vertical pixels and the number of horizontal pixels are output. Conversion means 1
At 22, the human pixel blocks of various sizes are
The signal is standardized to a signal corresponding to a block of × 3 pixels and output as a fixed pixel signal. The number of vertical pixels, the number of horizontal pixels, the standard pixel signal, and the foot position information obtained from the foot position detecting means 17 as required are input, and the posture estimating means 123 using a neural network calculates the position information. Human posture is obtained. The posture estimating means 123 using the neural network performs learning by giving a sufficient number of data having different postures at various positions and appropriate teacher signals by using a back propagation method in advance, and the error converges. It is assumed that it is in a state considered to be. In the posture estimating means 123, other neural network models such as LVQ can be used in addition to the multilayer perceptron.

FIG. 15 shows an example of processing for detecting a posture. 130 is a measured thermal image, and 131 is a human pixel block obtained from the human area detection unit 13. The human pixel block is converted into a rectangular block 132 by detecting the length and width of the human pixel block using the side image detecting means A120 and the side pixel detecting means B121 using the thermal image. Rectangular block 1
Since the size of the human area varies depending on the position of the person, the size of the area 32 may be various. Since this cannot be directly input to a multi-layer perceptron having a fixed number of neurons, the data must be standardized by some method. Here, 122
A case where the conversion means standardizes data of 9 pixels of 3 × 3 using the area ratio will be described. In FIG.
The procedure for standardizing a 4 × 5 rectangular block is shown. 4x
If the values of each of the 20 pixels of 5 are a to t and the values of each of the standardized 3 × 3 9 pixels are A to I, the values of A are standardized using the area ratio. 0.25 * 0.2 * a + 0.25 * (0.33-0.2) * e + (0.33-0.25) * 0.2 * b
+ (0.33-0.25) * (0.33-0.2) * f} / (0.33 * 0.33). Similarly, B to I can be obtained.

In addition, as auxiliary data, together with the foot position information and the number of pixels in the vertical and horizontal directions, a 12-input, 4-output multi-layer perceptron, which has been learned in advance by the back propagation method, is used to obtain the posture of a human in the room. Can be

When learning with a neural network,
By adding the data obtained by changing the direction of the human body from the two-dimensional infrared sensor to the learning data, it is possible to output including the direction in each posture. Further, as the target posture, it is possible to handle a posture in which the orientation is taken into consideration and a posture in which the type of the posture is subdivided.

Next, the characteristic amount of the posture determination characteristic amount detecting section 61 will be described with reference to FIG. FIG. 17 shows the flow of the present invention from the thermal image to the posture determination. In FIG. 17, the area of a person in the thermal image 140 is indicated by oblique lines.
When the above-described human region is detected, a pixel region within a square (thick solid line portion) that interpolates the region is cut out, and the feature extraction unit 141 performs region determination for nine 3 × 3 posture determinations. There are various methods for calculating nine (3 × 3) feature amounts. For example, assuming that the square of the thermal image is 2 × 2 as shown in FIG. 17, each pixel has G (1), G (2), G ( 3), G
Assuming (4), nine feature values, F (1), F (2),..., F (9) F (1) = G (1) F (2) = (G (1) + G (2)) / 2 F (3) = G (2) F (4) = (G (1) + G (3)) / 2 F (5) = (G (1) + G (2) + G (3) + G (4)) / 4 F (9) = G (4). Also, other blocks of position and human blocks in human block center of gravity or thermal image of the pixels of the lowest point of the human blocks down in the vertical direction from the center of gravity of a characteristic quantity, the human block Calculate the average temperature and so on. These feature amounts are input to a neural network 143 in the image information detecting unit 3 to determine a posture.
Is output. The outputs 145 and 146 of the neural network in FIG. 17 are used as a selection signal or a correction signal for position determination described later.

Next, another configuration of the image information detecting section 3 will be described with reference to FIG. It is assumed that the image information detecting unit 3 has a configuration of a foot position detecting unit 17 having a posture detecting unit 18 and a position determining unit for each posture. The position discriminating unit can be configured by one configuration when the posture classification is not performed, and when the posture is classified into a sitting position or a flat sitting position, for example, when the sitting position is finely classified into a chair sitting position and a flat sitting position, the number of posture classifications can be increased. In the embodiment of the present invention, the flow of processing will be described based on the configuration of FIG.

The individual for posture determination calculated by the image processing unit 3
Feature amount A 0.99 is input to the neural network configured as a posture determining unit as shown in FIG. 17, attitude signal 14
4 is output. The neural network of the posture judging unit detects the feature data of many people with different clothes and body shapes as much as possible in the scanning area of the thermal image at many positions, and learns the posture at that time as a teacher signal to the neural network. I do. The learning method differs depending on the type of the neural network. For example, a well-known method is backpropagation, which uses a steepest descent method that adjusts the internal state so as to reduce the square error between the output and the correct answer. . The signals 152 and 153 from the attitude determination unit will be described in the overall flow after the description of the position determination unit. In the position determination units 161 and 162, 161 is for a sitting position,
162 is for standing. Each estimating unit is formed of a neural network as shown in FIG. 13, and outputs the estimated positions 154 and 155 of the person. The neural network of each position judging unit actually detects a person with as many different clothing and body types as possible in the scanning area of the thermal image, detects feature data at many positions, and uses the position from the sensor at that time as a teacher signal. To learn by neural network. Although the learning method differs depending on the type of the neural network as described in the posture determination unit, for example, it is well known that the internal state is adjusted so as to reduce the square error between the output and the correct answer called back propagation. There is learning using the descent method. The personal characteristic amount B 151 for position determination is input to the position determination units 161 and 162, and the position determination result 15
4, 155 are output. It is necessary to select or process these two outputs and output them.
There are the following two methods for selecting 55 items. (1) 154 or 155 is selected based on the output result 144 of the attitude determination unit. (2) Output values 152, 153 and 154, 1 of the attitude determination unit
55 is processed and output. In many cases, the method (1) is generally performed based on the IF-THEN rule. In other words, the simplest method is that the IF output 144 outputs the sitting position THEN 154 and the IF output 144 outputs the standing position THEN 155. However, in the method (1), even when the posture has an ambiguous value (when it is not possible to determine whether the person is sitting or standing), the user is forced to select one of them.

[0066] Therefore, whether the posture determination unit as shown in FIG. 18
Output D152 for processing X154 and X155,
D153 is output, and the following processing is performed to output the position estimation value Y of the image information detection unit.

Y = D152 × X153 + D153 × X154 The advantage is that X152 and X153 output 1 or 0 when the posture discrimination result is clear, but 0 to 1 when ambiguous.
In order to output a value corresponding to the ambiguity up, it is possible to estimate the position estimate 16 3 good Ri exact position. In addition,
The adjustment of the outputs 152 and 153 of the attitude determination unit 160 is performed by the position determination units 161 and 162 as learning of the neural network.
After learning by sitting and standing, you can divide into sitting and standing
It can be implemented by learning together with the learned data. These learning methods are methods for using the above-described back propagation as a learning method for the structured neural network. Although the personal information extraction unit 14 is included in the image processing unit 2 in the present invention, it is naturally possible to incorporate the personal information extraction unit 14 into the image information detection unit 3 at the subsequent stage.

Next, the personal information extracting section 14 will be described in detail. The personal information extraction unit 14 detects the number of people and the position of the feet, detects the posture of the occupant, detects the temperature of the skin, and detects the amount of clothes in the first embodiment. FIG. 19 shows the first
FIG. 5 is a configuration diagram of a personal information extraction unit 14 in the example of FIG.

Next, a method of detecting skin temperature will be described. The skin temperature detecting means 20 detects the skin surface temperature of the occupant in the room. FIG. 20 shows the skin temperature detecting means 2
FIG. Hereinafter, the skin temperature detecting means 20 will be described. Reference numeral 360 denotes a block dividing unit that divides the human pixel block according to the body part based on the human pixel block obtained from the human region detecting unit 13 and the posture obtained from the posture detecting unit 18, and 361 denotes a block that is divided by the block dividing unit 360. The block temperature calculating means 362 for calculating the average temperature of the blocks 362 compares the respective temperatures obtained by the block temperature calculating means 361, and determines the block corresponding to the skin temperature at which the body surface appears and the block covered with clothes or the like. This is a temperature comparison means for separation.

The operation of the skin temperature detecting means 20 configured as described above will be described below. The human pixel block obtained from the human area detection unit 13 is divided into a maximum of six human bodies using the posture information so that the human body includes the head, chest, left arm, right arm, waist, and legs. In each block division, a division standard pattern of a human pixel block is determined in advance for each posture, and pattern recognition is performed. At this time, since the size of the human pixel block changes depending on the position of the person, block division is performed using the ratio used for standardization of the conversion unit 122 of the posture detection unit 13. FIG. 21 (a) and FIG.
(B) shows an example of block division. FIG. 21A shows a human pixel block of a standing occupant. The size is standard for simplicity. In this case, A is a block including the head,
B is the block containing the chest, C is the block containing the right arm, D
Is a block including the left arm, E is a block including the waist, and F is a block including the legs. The division into six blocks is because each block corresponds to a part of the body where exposure and covering can occur independently. FIG.
(B) is a human pixel block of a occupant sitting on a chair facing left. In this case, since the right arm is hidden, there are five divisions. Next, the block temperature calculating means 3
At 61, the average of the pixel values of each divided block is determined. The average block temperature increases when the body surface is exposed, and decreases when the body surface is covered with clothing.
Therefore, the temperature comparison unit 362 compares the predetermined threshold temperature with the block average temperature, and outputs the block average temperature having the maximum average value among the blocks higher than the threshold temperature. . In other words, the average temperature of the block is high when the part where the body surface is exposed, such as the face, is high, but the average temperature of the block is low when the part covered by the background or body clothes is in the same pixel. Therefore, the block having the highest average temperature can be determined to be the body part having the largest exposed portion. The skin temperature is detected with the above configuration.

Next, a method of detecting the amount of clothes will be described. The clothing amount detecting means 19 detects a human clo value extracted from the thermal image. FIG. 22 is a configuration diagram of the clothing amount detection means.

In FIG. 22, reference numeral 363 denotes area comparison means .
This is the same as the configuration of the skin temperature detecting means 20 of FIG. Here, the amount of clothing to be detected is called a clo value, and is, for example, 0 in a naked state and 0. in a state of only pants.
1, 0.3 in upper and lower underwear, 0 in short sleeves and summer pants.
5. The value of 0.7 for light work clothes with long sleeves and long pants, 1.0 for clothes in winter in the room, 1.5 for 3-piece suit, and 3.0 for over in outdoor clothes in winter. Have been. The amount of clothing is determined by how much of the body surface area is covered with clothes and the like and how thick the covered part is. Therefore, regarding the thickness of the clothes, the temperature comparing means 362 in the skin temperature detecting means 20 sets a temperature threshold value so that the clothes block whose body surface is covered with clothes or the like and the exposed body block are provided. Since the surface block can be separated, it is estimated by taking the temperature difference. That is, when the temperature difference is large, it can be determined that the clothing is thick, and when the temperature difference is small, it can be determined that the clothing is thin.
On the other hand, as to how much the body surface area is covered by clothes or the like, the area comparison unit 363 counts the number of pixels of the block separated into the clothing block and the body surface block, and compares the area ratio. Thus, the ratio of the clothing surface area to the naked body surface area can be estimated from the area ratio. Further, the area comparing means 363 associates the clo value with the clo value based on the ratio of the thickness of the clothing to the surface area of the clothing to the surface area of the naked body, and outputs the clo value. Here, it outputs the clo values for comfort calculation, in the case of using a comfort equation using a coating area ratio and clothes surface temperature of the body surface can also output these values directly. Furthermore, it is also possible to make a neural network learn the relationship between the feature amount and the amount of clothing for detecting the amount of clothing in advance, and to obtain the clo value using the neural network.

Next, FIG. 23 shows the configuration of the environment information extracting unit 15. The environment information extraction unit 15 is not an indispensable component of the image processing apparatus, but is a component added as necessary. 370 is a foot temperature detecting means for detecting a temperature near the foot from the foot position information and the thermal image;
Reference numeral 71 denotes an indoor part temperature detecting means for calculating an average temperature for each part of the wall and floor from the thermal image, and 372 denotes a foot temperature detecting means 37
0 and indoor part temperature detecting means 3 detected by 0
Radiation temperature detection means for detecting the radiation temperature of each occupant from the temperature of each part in the room detected from 71 and the foot position information output from the foot position detection means 17. less than,
Each means will be described. The step temperature detecting means 370 will be described. The object of the present invention is to easily and accurately detect a floor temperature near a human in a room. When trying to detect the floor temperature near a human based on the thermal image of the room, the problems are that it is difficult to identify where the floor is, and the temperature of the floor is not always uniform. The point is that the temperature of the floor near a human must be extracted because there is no human. In the present invention, a method is used in which a person in a room is first cut out, the foot position of the person is detected, and then the temperature of the floor surface, which is considered to have a large effect on the person, is extracted from surrounding pixels.

FIG. 24 is a configuration diagram showing the configuration of the foot temperature detection means 370. In FIG. 24, reference numeral 380 denotes peripheral pixel extracting means for extracting pixels around the human foot, and 381 denotes a foot temperature calculating means for detecting the foot temperature. The foot temperature calculating means 381 includes a method using an algorithm, a method using statistical processing, and a method using a neural network. In this embodiment, a neural network will be described as an example.

First, a signal (foot position information) indicating a human pixel block in the room and a representative point of the pixel block and a thermal image are input to the peripheral pixel extracting means 380. The peripheral pixel extracting unit 380 extracts, from the thermal image, values of four pixels that are considered to be closest to the human foot region, and sets them as peripheral pixel signals. Information about the periphery of the human foot, such as the peripheral pixel signal and the foot position information, is input to the foot temperature calculating means 381 which is a multilayer perceptron. The foot temperature calculating means 381 using the neural network calculates the input information and the teacher information corresponding to the input information as a radiation temperature actually measured at the position, and in advance a sufficient number of data and an appropriate teacher signal using the back propagation method in advance. It is assumed that learning is performed by giving the error and that the error is considered to have converged . In this step temperature calculating means 381, a neural network model such as LVQ can be used other than the multilayer perceptron. In this way, the foot temperature detecting means 370 performs processing on the basis of the peripheral pixel signal, and obtains the floor temperature at the foot of a person in the room.

FIG. 25 shows an example of processing in the foot temperature detecting means 370. 390 is thermal image data taken into the image storage unit 12. Reference numeral 391 denotes a human pixel block obtained from the human area detection unit 13. Next, based on the human pixel block 391, the peripheral pixel extracting means 380 detects a pixel corresponding to the position of the foot of the human and extracts pixels around the foot of the human. This, based on the detected foot position, the <br/> by 2 pixels outside the human region on both sides is the basic. However,
If the human area is located at the end of the thermal image data or if there is another human area in the vicinity, effective pixels cannot be obtained. In this case, the foot temperature may be detected from a small number of pixels. However, when a multilayer perceptron is used in the foot temperature detection device, it is desirable that the number of input data is uniform. Then, a method was used in which a priority list was created centering on the pixel corresponding to the foot position, and four pixels were extracted in descending order of priority. The values of the four pixels extracted in this way, the foot position information, and the like become peripheral pixel signals.

Instead of the foot temperature detecting means 381 using the neural network, there is a method using a foot temperature detecting means for averaging the values of the four pixels obtained from the peripheral pixel extracting means 380 to obtain the foot temperature. Can be realized by simple processing.

Next, the room temperature detecting means 371 will be described. If information on the mounting position of the two-dimensional infrared sensor 11 on the wall is at the center, right, or left and the size of the room, the boundary between the wall and the floor appears at which position on the thermal image. Can be calculated in advance. Therefore,
The indoor part temperature detecting means 371 obtains information on the size of the room and the mounting position of the air conditioner by switching on the air conditioner, and calculates a pixel of a boundary line of a wall or a floor based on the information. The area surrounded by each boundary pixel is defined as a wall and a floor in three directions. Further, an average temperature is calculated for each region, and an average temperature is calculated for each part in the room. Here, since the temperature may be partially different even on the same floor or wall due to the influence of a heat-generating object, adjacent room conditions, solar radiation, etc., each part may be divided into a plurality of blocks. FIG. 26A is an example of the appearance of thermal images of walls and floors when the air conditioner is installed centrally on one wall surface of an 8-tatami square room. Here, the floor is divided into six. 390 is the left wall, 391 is the back wall, 392 is the right wall, 39
3 to 398 are floor blocks divided into six.
FIG. 26B is an example of how a thermal image of a wall and a floor is viewed in the case of the left installation. 400 is a left wall, 401 is a back wall, 402 is a right wall, and 403 to 408 are floor blocks divided into six. In this way, by obtaining information on the size of the room and the installation position of the air conditioner, it is possible to know in advance where the respective parts in the room appear on the thermal image.

Next, the radiation temperature detecting means 372 will be described. The radiation temperature for each individual is calculated from the temperature from each part of the wall and floor in the three directions and the temperature near the feet. Conventionally, there is an air conditioner that measures the temperature of a plurality of distant areas on the floor surface and preferentially warms the colder floor area as equipment that considers radiation. When it is used for a part of the floor, the floor alone is not enough, and the radiation from the wall or the window is often large on a cold outdoor day.
In addition, floors and walls are made of materials and walls with different physical properties, and the direction of radiation to the human body is different between the floor and each wall. Is more accurate in calculating the radiation. FIG.
FIG. 7 shows the configuration of the radiation temperature detecting means 372. 410 is
Average temperature calculating means for calculating the average temperature of the pixel block of each part in the room obtained by the indoor part temperature detecting means 371;
Reference numeral 411 denotes a radiation temperature calculation unit that calculates a radiation temperature for each individual from the foot temperature obtained from the foot temperature detection unit 370 and the average temperature of each part obtained from the average temperature calculation unit 410. The radiation temperature calculation unit calculates the foot temperature near the occupant,
Radiation from each part is calculated for each occupant by multiplying the average temperature of the floor block and the average temperature of the walls in the three directions in the room by a coefficient such as a form factor that sums up to 1, for example. As for the floor block, it is also possible to use a part of the floor divided according to the position of the occupant.

Next, a different configuration of the environment information extraction unit 304 will be described. In FIG. 28, reference numeral 420 denotes an auxiliary heating detecting unit, and 421, a floor heating detecting unit. The above is the same as the configuration of the environment information extracting unit 304 in the first embodiment. Here, the auxiliary heating is a combustion or electric heating type heater such as a stove or an oil fan heater.
In addition, floor heating includes electric carpet in addition to normal floor heating. Since the auxiliary heating is higher than the temperature of the human body, the auxiliary heating detection unit 420 detects this by setting a certain temperature threshold higher than that of the human body in the measured thermal image.

Further, since the floor heating is at a temperature close to the human body temperature, it is detected once as a human pixel block by the human area detection unit 13. Since the area is larger than the human body and the form is usually a rectangle, the presence or absence of floor heating is determined by providing a range in the size and form of the extracted human pixel block. In addition, since the floor heating does not change much with time except when the power is turned on and off when the floor heating is detected, the change in the human pixel block is determined by examining the change in the extracted human pixel block for a predetermined period. It is also possible to detect on condition that there is no such condition. Thereby, it is possible to correct the influence of the heat transfer directly from the floor surface due to the floor heating and the influence on the calculated comfort due to the influence of the high radiation due to the combustion of the stove. In addition, for example, when there is floor heating or auxiliary heating, it is possible to control the air conditioner with a low degree of comfort while saving energy. Next, different configurations of the image processing unit 2 will be described. Hereinafter, since the configuration is different, the image processing unit 430 is used. The image processing unit 430 will be described with reference to FIG. 4
31 is a human area correction unit, 432 is an accumulation processing unit, and 433 is an indoor part determination unit.

First, the human area correcting section 431 will be described. The human region detecting unit 13 determines that each pixel of the image representing the temperature distribution is valid if the temperature value of the pixel is included in a temperature range in which the pixel can be determined to capture a human. A human area image was created by performing processing such as leaving the pixel as a normal pixel and removing it as an invalid pixel otherwise. However, due to the condition of human clothes, the temperature value of the pixel that originally captures the human does not match the temperature range in which it can be determined that the human is captured, and the pixel is processed as an invalid pixel. There is a case where a human region image in which a human region to be one region is divided into a plurality of regions is created. Such a defect of the human area detecting unit 13 is compensated by the human area correcting unit 431.

An embodiment of the human area correcting section 431 will be described with reference to the drawings. In FIG. 30, 4
40 is a cut-out pixel connection means, 441 is an isolated point search means, and 442 is a peripheral pixel search means.

The human pixel block detected by the human area detecting unit 13 is cut out together with the representative points together with the cut-out pixel connecting means 440.
Is input to It is the cut-out pixel connection means 440 that generates information as to whether or not the cut-out pixel exists in eight pixels around the two-dimensional coordinates of each pixel of the human pixel block. This connection information is input to the isolated point search means 441. Isolated point searching unit 441 checks if there is isolated points to a pixel which is cut out around 8 pixels on the basis of the connection information, and outputs the isolated-point information. The isolated point information is input to the peripheral pixel search means 442, and searches whether there is a pixel cut out in 16 pixels around the isolated point two-dimensionally one pixel apart. If there is a pixel cut out from the surrounding 16 pixels, it is connected to the cut out pixel. If there is no such point, a process of deleting the corresponding isolated point is performed. As a result, the corrected human pixel block is output from the peripheral pixel search means 442, and does not include an isolated point. However, if there is a possibility that an isolated point may exist depending on the position of a pixel on the two-dimensional coordinates of the image, an exception process in which erasure and connection are not performed is inserted depending on the coordinates of the isolated point. Finally, a representative point is calculated based on the corrected human pixel block.

The above-described human area correction unit 431 is supposed to be a single human area, depending on the noise removal added to the image and other heating equipment is determined as a small human area, or human clothes, etc. Even when things are divided, by removing small human areas as isolated points or complementing between divided human areas,
A human region can be correctly extracted, and human information can be detected from the human region image.

Next, the accumulation processing section 432 will be described. When it is difficult to accurately recognize the shape of the area from the image, the accumulation processing unit 432 can hardly detect information about a person or a room. easily calculating information for human and indoor Te.

FIG. 31 shows a configuration diagram of the accumulation processing section. FIG.
1, reference numeral 450 denotes a cumulative storage means A, 45 of a human position.
Reference numeral 1 denotes a stationary object determining unit, 452 denotes a stationary object pixel position storing unit, 453 denotes a cumulative storing unit B, and 454 denotes an indoor representative point determining unit.

The representative point of the human pixel block obtained by the human area correcting section 431 is stored in the accumulating storage means A450 having the same resolution as that of the image. This accumulation storage means A45
“0” stores the number of pixel positions of a representative point of a human detected from the K thermal images. The cumulative position information from the cumulative storage unit A450 in which the cumulative storage of the K thermal images has been completed is discriminated between a human and a stationary object by the stationary object determination unit 451, and the detected pixel position of the stationary object is stored. 452. Further, the accumulation storage means A450 processes the data and stores it in the accumulation storage means B453. In the accumulation storage means A450, 1 is stored in a place where a human is, and 0 is stored in a place where no human is. Cumulative storage means A450
Is deleted when data is stored in the accumulating storage means B453, and the accumulating storage is performed again. Cumulative storage means B453
When the data from the cumulative storage unit A450 is written n times , the indoor representative point determination unit 454 executes a process described later and is deleted. After the accumulation storage means B 453 is also deleted, the same processing is executed from the beginning. By processing the data in the accumulating storage unit B453, the installation position of the two-dimensional infrared sensor in the room, the position of the wall surface and floor surface in the room, the volume, and the like are calculated.

Next, the detailed processing of the stationary object determining means 451 shown in FIG. 31 will be described. The number of times of detection as a human position is stored in the cumulative storage A450 for each pixel of the thermal image. For example, the result at the coordinates (X2, Y2) on the two-dimensional accumulation storage means A450 is Sx2, y2. This number Sx2, y2 is compared with a threshold value TH1 (positive integer), and TH1
For those indicating a larger value, a still object list (memory) is created as a still object. In FIG. 32, K is 120
Since the example of turning is shown, TH2 was set to 100. As described above, of the K thermal images, the position of an object having a value close to M in the cumulative storage in which the number of cuts at the same position is stored is determined as a person and a stationary object erroneously cut out (for example, a stove). Or an electric carpet). That is, since the stationary object does not move, a numerical value close to the K value is stored in the stored position, so that it can be distinguished from a human body having a slight movement.

Next, FIG. 33 shows a process of storing the information of the cumulative storage means A450 in the cumulative storage means B453. The value of Sx2y2 in the accumulative storage means 1 is compared with the threshold value TH2, and if Sx2y2> TH2, 1 is written to the coordinates (x2, y2) on the accumulative storage means B453, otherwise nothing is performed. By repeating this process N times, if one area of the accumulative storage unit B453 is confirmed, it indicates the moving range of the human. The comparison with the threshold value TH2 is performed in order to remove the result of the process in which a person is erroneously detected as a person. The information in the accumulation storage means B 453 is processed by the indoor representative point determination means 454,
For example, the processing as shown in FIG. 34 can be performed. Hereinafter, the indoor representative point determination unit 454 will be described. Since 1 is stored in the accumulation storage means B 453 at a position where a person is present and 0 is stored at a position where no person is present, this is divided into three in the horizontal direction.
A total of 1 is obtained for each of the divided blocks A, B, and C. The sum of the obtained blocks is represented by SA, SB, S
The value of C is determined as follows, and it is determined whether the air conditioner is installed at the center on the wall, left, or right.

When SA <TH and SC ≧ TH, the C side is set. If SC <TH and SA ≥ TH, install on the A side. In other cases, it will be installed at the center.

Further, each part such as a floor and a wall surface in the room is also executed by the room representative point determining means 454. FIG. 35 shows an example of determination of each part in the room. In FIG. 35, the cumulative storage B45
The moving range of the human obtained from 3 is a range surrounded by A, B, C, and D. The coordinates of A are (X1, Y1) and the coordinates of B are (X2, Y
2) If the coordinates of C are (X3, Y3) and the coordinates of D are (X4, Y4),
A is a point where X1 is equal to or larger than the threshold TX and Y1 is the largest. B is a point where X2 is less than the threshold value TX and Y2 is the largest. C is a point where X1 is equal to or larger than the threshold value TX and Y1 is the smallest. D is a point where X2 is less than the threshold value TX and Y2 is the smallest. These A, B, C, and D become representative points that represent the form of the floor, which is the range of movement of the person in the room, when accumulated for a sufficiently long time. Therefore, the coordinates of each representative point are output. In addition, a point representing the floor shape can be determined for a room whose room is not square.

The accumulation processing section 432 detects the position of a person using edges, lines, and colors even when the image input means inputs a visible image, thereby acquiring the information of the person and the room by the accumulation storage section 432 of the present invention. can do.

Next, the indoor part determining section 433 will be described. When points A, B, C, and D are obtained as shown in FIG.
The area enclosed by CD is the floor surface, the area enclosed by ADD'-A 'is the left wall surface, the area enclosed by ABB'-A' is the front wall surface, and B- The range surrounded by CC′-B ′ is determined as the right wall surface. The correspondence between floors and walls and pixels is the same as the number of pixels in the thermal image.Preparing an array data structure with the same form and assigning different symbols to each region in the array, We can know whether we cope.

Hereinafter, a second embodiment of the present invention will be described. FIG. 36 is a configuration diagram of the second embodiment of the present invention. FIG. 36 is provided with a console unit 460 for inputting various information by the user in FIG. 1 described above. Console part 4
Processing other than the information from step 60 is performed in the description of FIG. The console unit 460 corrects information that cannot be detected from the thermal image based on the information input by the user, and outputs the output values of the detection of the foot position and the detection of the posture in the image information detection unit illustrated in FIG. , Will be able to do more accurately. As an example, information from the console unit is shown below.

When the infrared sensor is installed at a height that allows the room to be seen, and the heat distribution in the room is scanned as a thermal image, the position of the left, right, front, and front walls of the room as viewed from the sensor is determined from the thermal image. Is difficult. For this reason, when the user inputs the distance of the front wall surface or the left and right wall surfaces from the sensor, the position determination result from the position determination unit estimates the position farther than the front wall surface. It can be corrected to the position, or can be treated as a non-human heating object on the wall. In addition, since the temperature distribution for each wall surface can be identified, the degree of comfort of the thermal environment at the position of the person estimated by the position determination unit can be estimated.

Hereinafter, a third embodiment of the present invention will be described. FIG. 37 is a configuration diagram of the third embodiment of the present invention. FIG. 37 shows the sensor section 470 (camera) for the visible image in FIG.
, A visible image processing unit 472, and a visible image information detecting unit 47
3 to enhance the function as an image processing apparatus.

As a method of using the image processing apparatus using the visible camera, there is a monitoring system. 2. Description of the Related Art Conventionally, in a monitoring system, a person constantly monitors a visible image. For this, a technology for recognizing a person from a visible image is being developed, but few have come to practical use. However, many techniques for detecting a moving object from a visible image have been put to practical use. This is because, since the S / N ratio of the visible image is improved, the detection of the moving object can be easily performed by taking the difference between the temporally continuous visible images. However, since an object captured by motion includes a person other than a person, it is difficult to accurately capture a person from a moving object. Also, it is impossible to detect a person with little movement. On the other hand, it is very easy for an infrared image to capture a person as a heating object. However, since a heating object other than a person is also detected, this classification is difficult. In this way, the apparatus shown in FIG. 37 is intended to detect human or environmental information more accurately by combining detection that is good at visible image processing and detection that is good at infrared image processing.

The flow of the process will be described with reference to FIG. Of the information 5 obtained from the outside world 4, visible light is detected by the camera 470 to obtain a visible image 473. The visible image 473 is transferred to the visible image processing unit 471 and subjected to preprocessing, temporal difference processing of continuous images, and the like. Next, the visible image information detecting section 47
In 2, the motion information is detected. This information 475, 4
As 76, the presence or absence of the moving object, the number thereof, the position of the moving object, and the like are transmitted to the image processing unit 2 and the image information detecting unit 3.
Further, based on the information 477, the image processing unit 2 performs area detection with reference to the number and position of the areas in the human area detection processing. Further, the visible image information detecting unit 472 associates the presence or absence of the moving object, the number thereof, the position of the moving object with the position of the person detected by the image information detecting unit 3, and outputs
Is detected.

The visible image processing section 472 and the image information detecting section 3
An example of a target to be detected based on the mutual information is shown below. Example 1: Whether the moving object detected by the visible image processing unit 471 or the visible image information detecting unit 472 is a person,
The determination is made by matching the position with the position of the person detected by the image information detection unit 3 that detects information from the thermal image. Example 2: A human detection mark is shown at the position of the visible image calculated from the position of the person detected by the image information detection unit 3 so that the user can easily check the visible image. Example 3: Only when the posture of a person detected by the image information detecting unit 3 is sitting, the position of the person is marked on the visible image because the possibility that the target person is a stationary person is high. Example 4: When the posture of the person detected by the image information detection unit 3 is sitting, the image information detection unit 3 transmits the information to the information 476, and the visible image processing unit 471 processes the information. , And creates the person information 474. Example 5: The position and orientation of a person detected by the image information detecting unit 3 are transmitted to information 476, and the information 476 is used as a teacher signal, and the visible image processing unit is configured with a processing system having a learning mechanism such as a neural network, Evolve human recognition processing. Example 6: The position of the detected person is transmitted as information 475 from the motion information signal detected by the visible image processing unit 471 and the visible image information detection unit 472, and the information 475 is used as a teacher signal, inside the image information detection unit 3 described above. Learn and evolve the capabilities of processing systems with learning mechanisms such as neural networks.

Next, FIG. 38 will be described. FIG.
A monitoring information detection unit 480 that performs integrated processing by using the information 474 and the output of the image information detection unit in FIG. 7 described above as inputs is provided, and a suspicious person warning signal 481 and an abnormal signal 482 are output. As an example of the information integration, when the area measured by the camera 470 or the sensor unit 1 indicates that a person should not stay or that there is a movement of a person who has entered, the information 4
The motion information is output from 74, and the position information of the person is output from the image information detecting unit 3. However, after the posture is detected as a sitting position in the area, the position of the person does not change, and for a certain period of time, if such state continues, you output suspicious individual alert signal 481 and error signal 482.

Further, it is also possible to set an area where no person is allowed to enter the measurement area in the sensor section by using the console section 460 and detect a person entering the area.

The image processing apparatus is not only a monitoring system but also a visible image information detecting section 472 as shown in FIG.
The information is integrated from the output of the adaptive learning unit 490 shown in the above example with the output of the image information detecting unit 3 and the personal identification unit 49.
In step 4, identification or tracking of an individual in the detection area can be performed. Thus, for example, the visible image information detecting section 47
In the case where the information 491 on the color or the physique of the clothes of the human is obtained from 1 and the posture and the foot position information 492 are obtained from the image information detecting unit 3, the adaptive learning unit 490 determines the position, the posture, the physique, and the clothes of the human. Learn features to identify individuals from their colors and the like. The individual identification unit 494 performs pattern recognition based on the feature amount obtained from the adaptive learning unit 490 to identify an individual. As a result, highly accurate personal identification becomes possible.

Finally, an example of the configuration of a system integrating the above-described image processing apparatus of the present invention will be described. The overall configuration will be described with reference to FIG. FIG. 40 includes an integrated processing unit 503 that collectively processes information from the above-described image processing apparatuses 500 to 502 of the present invention and three image processing apparatuses. Each of the three image processing apparatuses 500 to 502 has a different angle or a different measurement area. Information 504 to 506 obtained from the three devices is processed temporally and spatially by the integrated processing unit 503, and information 507 or environmental information 508 such as tracking of a person and position and posture of the person is output. Two specific examples will be described below.

Example 1: The image processing apparatuses 500 to 502 measure and process substantially the same area from different points. Each observation point and arrangement relation are input from each console unit of the image processing apparatuses 500 to 502. In the image processing apparatus 500, when a person overlaps and determines the position and posture as one person, the image processing apparatuses 501 and 502
If the information obtained from the information indicates the positions and postures of a plurality of persons, the integrated processing unit 503 uses the image processing apparatuses 500 to 50
From the arrangement 2, accurate information 507 about a human is detected and output.

Example 2: Each of the image processing apparatuses 500 to 502 measures and processes an area in a different direction. Image processing device 5
The observation points and arrangement relations are input from the console units 00 to 502. A person detected by the image processing device 500 leaves the measurement area of the image processing device 500, enters the measurement area of the image processing device 501, leaves, and then enters the measurement area of the image processing device 502 and then stands still in a sitting position. . In such a case, the motion, position, and posture of a person who is temporally continuous are processed by the integration processing unit 503 and output as information 507.

In order to execute Examples 1 and 2 simultaneously, a large number of image processing apparatuses 500 may be prepared to perform spatial and temporal processing.

As a specific application device of the above image processing apparatus, a monitoring apparatus for detecting human or environmental information from an image having a small number of pixels can be considered. In the case where the number of pixels is not so required, the image processing is easy and there is no problem in the amount of information to be detected, the smaller the number of pixels, the lower the cost and the easier to use. Further, by using both the visible image and the thermal image, the detection accuracy of human information is particularly improved.

Also, in the nursing of a person such as medical care or nursing care, the state such as whether the care recipient is asleep, awake, awake, or asleep is determined.
By using the image processing apparatus, an apparatus capable of nursing without a nursing person being around for 24 hours is possible. It is also possible to detect the amount of clothing and the skin temperature, and determine whether the bedding has shifted or the body temperature is not higher than usual.

Hereinafter, a fourth embodiment of an air conditioner control apparatus using the image processing apparatus of the present invention will be described.

FIG. 41 is a block diagram of the fourth embodiment. 6
Reference numeral 01 denotes a sensor unit, which includes a two-dimensional infrared sensor 602 for measuring a temperature distribution in a room, a room temperature sensor 603 and a humidity sensor 6.
04 and an outside air temperature sensor 605. Reference numeral 606 denotes an image processing device, and the two-dimensional infrared sensor 602 is shared with the sensor unit of the image processing device 606. Reference numeral 610 is wind direction / wind amount information. The comfort level detection unit 613 detects the foot position on the floor for each occupant in the image processing device 606. Therefore, if the wind direction and the air volume from the outlet are known, the air conditioner at an arbitrary position on the floor is detected. Can calculate the airflow (m / s). Therefore, by obtaining from the air conditioner the wind direction and wind amount information 610 such as the vertical and horizontal flap angles for determining the wind direction and the rotation speed of the indoor unit fan for determining the air flow, the airflow for each occupant at each position is obtained. Is calculated. Reference numeral 611 denotes a control index determination unit, and an activity amount detection unit 612 and a comfort level detection unit 6 for calculating a comfort level PMV.
13. Reference numeral 614 denotes an air-conditioning control unit.
Is a calendar.

Next, the control index determining section 611 will be described. The control index determination unit 611 in the first embodiment calculates the degree of comfort for each occupant, and selects a control object to be preferentially air-conditioned. In order to calculate the comfort level, the activity amount calculation and the clothing amount estimation are performed for each occupant as a human element. The environmental factors such as room temperature, humidity, radiation temperature, and airflow are obtained from the environmental information extraction unit. Hereinafter, the activity amount detection unit and the comfort level detection unit will be described in this order.

It is known that when PMV is used as a control index as a control index, the amount of activity is an important factor as a human state. The amount of activity used in PMV is called the MET value, and the relationship between the MET value and the state of a person is:
When lying down in a lying position and relaxing, the MET value is 0.
8, walk 1.0 at a time sitting quietly in a chair, 1.4 at a state of relaxation in a standing position, 1.6 at a state of light work in a standing position, and 3 km / h. 2.0 when the vehicle is running, and 3.0 when the vehicle is running at 5 km / h or when the vehicle is moving violently. Normally, if you live a standard life in your home, the ME between 0 and 2.0
Take the T value. In the conventional activity amount detection method, the movement of a heating object in a certain area is detected by an infrared sensor, and the magnitude of the activity amount is determined based on the magnitude of the output value. However, in the above conventional activity amount detection method, for example, even if there are a plurality of occupants in the room and some of them are stationary, if other occupants move around the room, the activity amount is uniformly large. There is a problem that the air conditioner is controlled according to a person who has a large activity amount. In addition, the detection area of the infrared sensor for detecting the activity amount is not only a part of the room, but the relation between the output value of the infrared sensor and the actual movement of the person is ambiguous, so the detection accuracy of the activity amount is Was low. Therefore, in the present invention, in order to solve the above-mentioned conventional problems, the position of the occupant is detected over a wide range of the room, and the stay frequency, which is the continuation duration of the occupant with respect to the current position, is determined for each occupant. The activity amount for each room is detected.

FIG. 42 shows an activity amount detecting section 61 of the first embodiment.
FIG. 620 is a personal information storage means for storing the foot position information of the occupants, 621 is a stay frequency calculation means for counting the stay frequency for each floor area divided into a plurality of areas in advance, and 622 is a stay frequency and It is an activity amount calculation unit that associates an activity amount. The operation of the activity amount detection device configured as described above will be described.

When the room is a square room of 8 tatami mats, the floor surface is previously divided into 6 × 6 in units of 60 cm, for example, as shown in FIG.
Number them as position numbers. Image processing device 6
The floor position for each occupant obtained in step 06 is stored in the personal information storage means 620 as personal information. FIG. 43 (b)
9 shows an example of personal information stored in the personal information storage means 620. The personal information includes the measured time, the number of occupants at that time, and the floor number of each occupant for the number of persons. Next, the stay frequency calculating means 621 counts the frequency of the visitor for each floor number from the floor position number in the personal information when using the foot position information for the past three minutes from the current time at a sampling period of 30 seconds. I do. Then, only the part of the stay frequency corresponding to the floor number of the occupant obtained by the image processing device 606 at the present time is extracted.

FIG. 44A shows an example of the stay frequency corresponding to the floor position. It is considered that the stay frequency of the position number 32 is 6 and the stay frequency is 1, and the stay frequency of the person of the position number 17 is 1 and the person has just moved from some other position. It can be considered that the person at the position number 9 has stayed at the position 9 and only two minutes have elapsed, or has moved to another place for one minute during the three minutes. The stay frequency obtained here serves as an index indicating how much the occupant has been stationary with respect to the floor position in the past. In other words, when a person is moving, the measurement interval is too long with a sampling period of 30 seconds, so that it is not possible to calculate the moving distance or the like of each occupant, and the moving distance of the person cannot be used as the activity amount. However, in a room, a person is often stationary rather than moving. Therefore, if there is a person in the same position even at a long time interval of about 30 seconds, it may be determined that the same person continues to stay. Often.

The activity amount calculating means 622 associates the current staying frequency of each occupant obtained by the stay frequency calculating means 621 with the activity amount. FIG. 44 (b) shows an example of a correspondence relationship for association. When the stay frequency is 6, which is the maximum value, it is considered to be stationary, so the activity amount is 1.0. When the stay frequency is 5, it may be possible that the user has moved a little while standing still. Since there is a possibility that the activity has been imported, medium 1.6 and stay frequencies 1 and 2 are associated with a high activity amount 2 since they have just moved from another place. This association changes depending on the number of samples stored in the personal information storage unit 5. In this way, by sequentially calculating the stay frequency, in the case of a person with a high degree of stillness, the stay frequency frequently outputs a value near the maximum frequency, and often moves like cleaning. Since the value of staying frequency is continuously output, the activity amount can be detected with high accuracy. In addition, since a stationary person and an active occupant can be separated, when a occupant with a high activity amount and a occupant with a low activity amount are present, the comfort level, which is a control index, is determined by the activity amount. It can be adapted to low occupants.

Further, when a room is divided into regions, there is always a case where a person is present at the boundary between the regions. In this case, if the movement of a person is slightly higher or the accuracy of detecting the position of the foot in the image processing device 606 is not very high, the detection position may be shifted at each measurement even if the person is stationary. Therefore,
By defining the peripheral floor area for each floor position in advance and considering the frequency of stay in the peripheral area when calculating the stay frequency, a person who is stationary can be captured accurately. FIG. 45A shows an example of the peripheral definition for the floor surface.
Since the floor number 31 is at a corner, the floor numbers 25, 26, and 32 surrounding it are set as the periphery. Since floor number 3 is located on the side, floor number 2 surrounding it is
4, 8, 9, and 10 are defined as peripheral areas. Floor number 23
Is inside, so floor numbers 16, 17, 1 surrounding it
8, 22, 24, 28, 29, and 30 are defined as peripheral areas. The stay frequency calculating means 621 counts the stay frequency of the floor number in the peripheral area in addition to the floor number of the current occupant. FIG. 45 (b) is an example of stay frequency calculation including the peripheral area for the personal information of FIG. 43 (b). The stay frequency of the person at the floor number 17 is 1 when the surrounding area is not included, but the stay frequency is 4 when the surrounding area is included, and the way of movement is not moving around in a distant place but staying now. It can be seen that the subject moves around the point, and a value that is not so high as the amount of activity can be related.

Further, when the sampling period of the thermal image measurement is a short period of about 1 or 2 seconds or less in the activity amount detecting section 613, there is a limit to the moving speed of the person in the room. The occupants are associated with each other from the extracted feet of the occupants, and the moving speed can be obtained (hereinafter, referred to as the moving amount). In this manner, the accuracy of activity amount detection can be improved by obtaining the amount of movement for the occupant who is moving. Hereinafter, an example of activity amount detection using the movement amount will be described.

FIG. 46 shows a configuration diagram of the activity amount detection unit 613 using the movement amount according to the first embodiment. 630 is continuous 2
Moving amount detecting means for calculating the moving distance from the foot position information at two measurement intervals; 631 a personal information storing means for storing information on the position and moving amount for the past T1 minutes from the current time;
Reference numeral 32 denotes a period movement amount calculation unit for obtaining a movement amount representative of a predetermined period, 621 denotes a stay frequency calculation unit for detecting a stationary person similar to that shown in FIG. 42, and 633 denotes an activity amount calculation unit.

The activity amount detector 61 configured as described above
The operation of No. 3 will be described. FIG. 47 is a graph showing a scene of cleaning using a vacuum cleaner which is considered to be the most active in a room, in which video is recorded at an actual home and the moving distance per second is plotted with 60 cm as one unit. The horizontal axis represents the time elapsed from the start of cleaning and is sampled every second. The vertical axis is distance and one scale is 60 cm. The feature of the cleaning is that the moving distance per unit time, that is, the moving speed of the unit is shorter than the moving speed of the unit time. The amount of movement is faster in the case of an action such as going for a thing in the room or passing through the room, but since it is completed in a few seconds, it is often not possible to say that it is active. Therefore, when judging the amount of activity of the occupants, it is better to judge based on a certain instantaneous travel distance.
It can be seen that it is appropriate to make a judgment by integrating the moving distance for a certain period. Now, regarding the operation of the activity amount detection unit 613, the obtained foot position information for each occupant is input to the movement amount detection unit 630. Hereinafter, the movement amount detection means 630 will be described.

FIG. 48 is a block diagram of the moving amount detecting means 630. Here, the sampling time is Δt, and the time t
The foot position information detected at step S1 is converted to the foot position information F (t),
The foot position information detected before the sample is converted to the foot position information F
(T−Δt). Reference numeral 640 denotes a position storage unit that stores information on the foot position of the occupant. Numeral 641 is a maximum value of a distance that a human can move within a predetermined time Δt with the input of the foot position information F (t) and the foot position information F (t−Δt) stored in the position storage means. This is a movement determination unit that searches for a occupant within the maximum movement distance from the foot position information F (t−Δt) using the maximum movement distance. Numeral 642 searches the foot position information F (t) output from the movable determination means for the foot position information closest to the foot position information F (t), and determines two pairs of combinations in order. Nearest neighbor search means. Reference numeral 643 denotes a moving speed calculating unit that receives a distance output from the nearest neighbor searching unit and outputs a moving amount for each occupant.

The moving amount detecting means 6 configured as described above
The operation of 30 will be described. First, the foot position information detected by the image processing device 606 at each sampling timing is sequentially stored in the position storage unit 640. next,
The foot position information F (t) and the foot position information F (t−) stored in the position storage unit 640 by the movable possibility determination unit 641.
Δt) distance calculation is performed. At that time, distance calculation is performed for all combinations of the occupants of t and t−Δt, and a combination within the maximum movement distance is selected. Next, in the nearest neighbor search means 642, the foot position information F (t−Δt) and the foot position information F output from the movable judgment means 641 are obtained.
From the combinations of (t), a pair of combinations at the closest position is output. At this time, if the number of persons increases during Δt, there is an extra pair of foot position information F (t) that cannot be paired, but in this case, it is considered that the person is entering a room from another room. The moving distance is substituted for the moving distance. Also, if the number of persons decreases during Δt, there is a surplus in the foot position information F (t−Δt). However, in this case, there is no problem because the number of persons has decreased at this time.
Next, the moving speed calculating means 643 calculates the nearest neighbor searching means 6.
The moving speed of each occupant is calculated from the moving distance of each set and the time interval Δt output from 42. With the above configuration, by calculating the amount of movement, even if a person is present, for example, at both corners of a room, the correct person's moving speed can be achieved without exceeding the man's possible moving speed. Can be requested. Further, it is also possible to calculate the foot position by coordinates, calculate the moving distance from the coordinates, and calculate the moving amount without dividing the room into a plurality of regions. After the movement amount of each occupant in the room is obtained in this way, a movement amount representative of a certain period of the room (hereinafter, referred to as a period movement amount) is calculated. FIG. 49 shows an example of personal information stored in the personal information storage means 631. The personal information describes the measured time, the number of occupants at that time, the floor number of each occupant, and the distance for each occupant. The difference from FIG. 43B is that a distance is included. The period movement amount calculating means 632 associates the movement amount for each individual with the period movement amount. The period movement amount is represented by ranks A, B, and C in descending order of activity. Figure 50: How to find rank A
Explanation will be given while looking at (a). First, the average value of the movement amount of each occupant is determined, and it is determined whether or not the average exceeds the threshold value H1. This process is performed for each measurement sample for T1 minutes, and the number of times during T1 minutes is counted. And
When the value is larger than the predetermined frequency threshold N1, the period movement amount rank A is set. Next, FIG.
A method of obtaining the period movement amount rank C will be described with reference to FIG. The frequency at which the average value of the movement amount falls below the threshold value H2 is counted for T1 minutes, and a case where the value is larger than a predetermined frequency threshold value N2 is defined as a period movement amount rank C. If both the ranks A and C are not included, the period movement amount rank B is set. here,
The reason why the threshold value is set not only for the amount of movement but also for the frequency is that the object may not move in an active life scene or may move in a relaxing scene.

The amount of period movement calculated by the above method is combined with the presence or absence and the number of stationary persons obtained from the stay frequency calculation unit 621, and the activity amount calculation unit 633 associates the amount with the activity amount. FIG. 51 shows an example in which the period movement amount obtained from the period movement amount calculation unit 632, the stay frequency obtained from the stay frequency calculation unit 621, and the number of people obtained from the image processing device 606 are related to the activity amount. In FIG. 51, the value in the table is the amount of activity, and the stationary person indicates a case where the stay frequency calculating unit 621 has a occupant whose stay frequency is 90% or more of the maximum value. By detecting the amount of activity by the above method, the amount of activity can be detected along the life scene.

Next, the comfort level detector 613 will be described. Conventionally, when calculating the degree of comfort, there has been a method of calculating the degree of comfort representing a room. Here, the degree of comfort is mainly calculated for each individual, and these are used as control indexes. Comfort PM
As input elements of V, environmental factors include room temperature, humidity,
Airflow and radiation temperature are required, and factors related to humans include the amount of activity and the amount of clothing. Also, P for calculation
MV calculation formula is required. Each element, each room temperature using the measured value of the room temperature sensor 603 for measuring the intake temperature of the air conditioner, the humidity using the measured value of the humidity sensor 60 4 installed in the main body of the air conditioner. The airflow is obtained from the wind direction information and the foot position of each individual obtained from the image processing device 606. The radiation temperature is also obtained from the image processing device 606. The activity amount for each individual is obtained from the activity amount detection unit 612. The clothing amount is obtained from the image processing device 606. At that time, clothes are generally changed depending on the outside temperature and the season. Therefore, each month is made to correspond to each of the four seasons of spring, summer, autumn and winter based on the value of the calendar 615, and the rank is conditionally classified based on the values of the month and the outside temperature. It is also possible to make corrections. This makes it possible to detect the amount of clothing using the thermal image of the target person. Examples of the PMV calculation formula are shown below.

PMV = (0.303e-2.1M ⁺ 0.028) ｛58.
15 (MW)-3.05 · 10 ^-3 [5733-406.7 (MW)-
Pa] -24.42 [(M-W ) -1] -10 -3 · M (5867-P
a) −0.0814 · M (34−ta) −3.96 · 10 ⁻⁸ fcl [(t
cl + 273) ⁴ − (tr + 273) ⁴ ] −fcl · hc (tcl−t
a)｝ Here, the meaning of each symbol is as follows. M: Metabolic rate (Met value) W: External mechanical work (can be assumed to be zero for most metabolic rates) Pa: Water vapor pressure ta: Air temperature tr: Average radiation temperature fcl: Part covered with clothes And the exposed area ratio tcl: surface temperature of clothes hc: convective heat transfer coefficient The convective heat transfer coefficient hc satisfies the condition of 2.38 (tcl-ta) ^0.25 > 12.1 kgVr when Vr is the airflow (m / s). For example, hc = 2.38 (tcl-ta) ^0.25 , and in the case of 2.38 (tcl-ta) ^0.25 ≤ 12.1 kgVr, hc = 12.1 kgVr can be determined.

Also, as for the area ratio fcl of the area covered with the clothes and the area of the exposed part, if the clo value is Icl, if the condition of Icl <0.5 is satisfied, fcl = 1.00 + 0.2 · Ic
l If the condition of Icl ≧ 0.5 is satisfied, fcl = 1.05 + 0.1 · Ic
l.

As described above, since each input element for calculating the PMV can be detected, the comfort level PMV for each individual can be calculated. Next, a control index for controlling the air conditioner is determined based on the comfort level calculated for each individual. When a plurality of occupants are in a room, the comfort level of each individual may differ depending on internal and external conditions such as the degree of activity and the effect of spatial radiation. In that case, if there is a room occupant whose comfort level detected by the above method is outside the predetermined comfort range, for example, PMV ± 0.5, the room occupant is preferentially air-conditioned. In addition, when all the occupants enter the comfortable area, wind direction control is performed centering on the occupant's position.

An example of air conditioning control in the fourth embodiment will be described below. FIG. 52A shows an example of air conditioning at the time of heating in a case where there are two occupants in the room and their PMVs are almost the same. 650 is a room, 651 is an air conditioner,
652 is an infrared two-dimensional sensor, 653 is a sitting occupant,
654 is another occupant in the sitting position, 655 is a door, 65
Numeral 6 denotes a wind direction and air volume, and 657 denotes a detection range of the two-dimensional infrared sensor. In this case, since the occupant 653 and the occupant 654 are stationary in the sitting position, they have the same amount of activity.
Assuming that the intensity of the radiation received from the wall surface and the intensity of the air flow are also the same, the PMVs of the two persons are the same, the blowing temperature is such that the indoor thermal environment is maintained, the wind direction is wide, and the air volume is medium. FIG. 52 (b) is an example of air conditioning in a case where time has elapsed from the state of FIG. 52 (a), the occupant 658 has moved, and the door in the room has opened, and cool air has entered the room. 658 is an occupant who moved in a standing position, 659 is an occupant who is still in a sitting position, 660 is an open door, 66
1 is the wind direction and the air volume at this time. In this case, the door 66
The floor temperature near the room occupant 659 cools due to the cold air entering from 0, and the temperature drops on the floor and the wall.
Observed by

Further, since the 658 occupant starts to move in the standing position, a higher amount of activity is observed than the occupant in the 659 sitting position. In this case, since the PMV is high near the occupant 658 and low near the occupant 659, the wind direction is
59 is centered. In addition, since the cool air enters from the door 660, the blowing temperature is controlled to be higher. In addition, even if the occupant 658 does not move, the door 660 is not moved.
When the temperature of the floor and walls near the occupant 659 decreases due to the cold wind from the room and the radiation temperature to the occupant 659 is low, the wind direction is directed toward the occupant 659. When is closed, the control returns to the air conditioning control in a stable state in a short time. In this way, by estimating the comfort level of each occupant and performing air conditioning based on the estimation, fine air conditioning can be performed in accordance with the comfort level of the people. If there is no occupant, it is assumed that there is an occupant having a predetermined amount of clothing with a zero activity amount in the center of the room and a control index is determined, or a place where the floor or wall temperature of the room is low, Alternatively, control such as blowing hot air to a high place can be performed. The same control can be performed in the case of cooling.
If it is unstable to change the wind direction for each sample for the wind direction control, the position movement of the occupant is checked while looking at the output relating to the foot position information of the image processing device 606, and the position is continuously measured at the same position. Only when there is a room occupant, it is possible to control the wind direction.

Next, a fifth embodiment of the air conditioner control apparatus in which a life scene estimating unit is added to the fourth embodiment will be described. FIG. 53 is an overall configuration diagram of the fifth embodiment. The difference from FIG. 41 is that a life scene estimating unit 670 is added to the control index determining unit 611, an optical sensor is added to the sensor unit 601, and a clock is added.

The life scene estimating section 670 will be described below. In the life scene estimation unit 670, the activity amount detection unit 6
And the image processing device 60
6, the indoor illuminance level obtained from the optical sensor 671, and the time obtained from the clock 672, the life scene in the room is estimated. Here, the life scene is a name of a living activity or a living state performed by the occupant when viewed in indoor units, and is generally called by a name such as danran or relaxation. If there is only one person,
The life act of the person is a life scene, but in the case of a plurality, the estimation must be made in consideration of the degree of activity of each person. Estimated life scenes include leaving, entering, going to bed, getting up,
There are six transitional scenes of hot and cold occupants, and three stable scenes of relaxation, reunion, and active housework. The transitional scene referred to here is a scene in which PMV has no validity as a control index, and a stable scene is a scene in which a occupant is in a room for a certain period of time and the PMV can be used as a control index. is there.

A method for estimating each scene will be described below. First, a method for estimating a transient scene will be described. In the exit scene, when the number of persons obtained from the image processing device 606 becomes 0, the absence duration time starts to be counted, and when the number of persons continues to be 0 for 10 minutes or more, the exit scene is determined. Here, the absence duration is usually counted because the room often moves in contact with an adjacent room or the outside.Therefore, the exit is determined for each of these short absences, and the control index is changed. Becomes unstable, so that a leaving scene is determined after a certain period of time elapses in order to set a scene as one section. Next, the entry scene is a state in which the current life scene is the exit scene and one or more persons are detected as an entry scene, and is held for 20 minutes. Next, in the case of a bedtime scene, when the illuminance obtained from the optical sensor is equal to or less than a predetermined brightness, the time is between 9:00 pm and 6:00 am, and the amount of activity is 1.0 or less for 30 minutes. If you do, it is determined that you go to bed. Also, if the current life scene is a bedtime scene and the state of the activity amount 1.2 or more continues for 5 minutes,
Alternatively, if the illuminance exceeds a certain value, it is determined that the scene is a wake-up scene, and is held for 20 minutes. In the hot occupant scene and the cold occupant scene, the skin temperature obtained from the image processing device 606 is
Judgment is made based on whether the temperature exceeds a standard human skin temperature range predetermined for each month previously obtained from a calendar.

Next, a stable scene will be described. As for the relaxing scene, when the activity level is 1.0 or less for 10 minutes or more, the group scene is active when the activity level is 1.2 to 1.7 for 10 minutes. For housework, the case where the activity amount is 1.8 or more for 10 minutes is set as a scene determination condition.

For the life scene estimated as described above, the control index of air conditioning is determined in advance. less than,
An example of a control index determined for each scene will be described. Here, the standard target comfort level is 0. In the case of the leaving room, the target comfort level is reduced to -1.0 in the case of heating, and the wind direction is directed to the center of the room. In the case of the entry scene, the target comfort level is set to 1.0,
Turn the wind toward the occupants. In the case of a bedtime scene, the target comfort level is reduced by 0.5 every two hours, and the wind direction is to avoid the human body and to reduce the airflow. In the case of the wake-up scene, the target comfort level is set to 2.0, and the wind direction is directed to the occupant. In the case of a hot occupant scene during cooling, lower the target comfort level to -2.0,
Cool with the wind direction mainly for the occupants with high skin temperature. In the case of a cold occupant scene at the time of heating, the target comfort level is increased to 2.0, and heating is performed in a wind direction centered on the occupant whose skin temperature is low. However, directing cold air or hot air to the human body for a long time is not good for health, so the hot and cold room scenes are limited to 3 minutes. In the case of a relaxing scene, the target comfort level is set to a slightly warmer level of 0.5. The standard comfort level is set for the group scene and the target comfort level is set to -0.5 for the active housework scene. By performing air conditioning in accordance with each scene in this manner, air conditioning suitable for actual living conditions can be achieved.
When a stable scene is estimated, the control index is changed to the fourth.
In the case of a transient scene, it is also possible to switch to the control index for each scene described above. The air conditioning control unit 614 obtains these control indices,
Control the equipment.

The image processing apparatus or the activity amount detection unit 612 according to the first to third embodiments detects presence / absence and number of persons in the room, foot position, posture, clothing amount, skin surface temperature, and activity amount. It is also possible to use the present invention in an application device of an image processing apparatus for performing the following. For example, a security device that detects an intruder using the human area detection unit of the present embodiment. A device for detecting the degree of congestion in a room using a number of people detecting means. Further, an apparatus that controls an environment such as lighting and sound according to a position of a person, using a foot position detecting unit or an activity amount detecting unit. An apparatus that detects the state of a patient or a sick person used in medical care or nursing using the posture detecting means, the clothing amount detecting means, and the skin temperature detecting means can be considered.

Hereinafter, a sixth embodiment of the present invention will be described. FIG. 54 is a configuration diagram of the control index determination unit according to the sixth embodiment of this invention. Other configurations are the same as those of the fifth embodiment shown in FIG. Reference numeral 680 denotes an absent control index determination unit that outputs a control index when no occupant is present in the room. FIG.
2 shows the configuration of the absence control index determination unit. Reference numeral 690 denotes an occupancy data storage unit that determines presence / absence of occupants (hereinafter referred to as occupancy data) based on the number of people obtained from the image processing apparatus 606 and stores the data as time-series data. 692 is a base temperature conversion means. here,
The base temperature is a minimum maintained temperature level, which differs between heating and cooling.

The operation of the absence control index determining unit 680 will be described below. The occupancy data storage unit 690 detects the presence or absence of a person as 1 if the occupant is present and 0 if not, based on the number of people obtained from the image processing device 606. The data of 0 and 1 indicate that the actual presence / absence greatly changes when a person enters / exits the room.
0 minutes as one unit, for example, if the number of people is 0 for 10 minutes or more
It is 0 only for humans and 1 otherwise. It is also possible to perform data compression or the like to reduce the storage capacity of the occupancy data storage means 690. In-room data storage means 690
Then, data management is also performed so that the room occupancy data for one day is stored for a certain number of days from the current day. The reason why the data is stored for a certain period from the day and the old data is not used is that the use of too old data for calculating the occupancy rate for obtaining the later data is likely to not reflect the recent life situation. Here, the upper limit is 60 days. Next, based on the attribute information of the day from the calendar, the occupancy pattern classification means 691 collects the occupancy data of the same day and the attribute of the day, and obtains the occupancy rate by time. FIG. 56 (a) shows actual room occupancy rates in the living room of the home, and FIG. 56 (b)
Shows the occupancy rates on holidays. The abscissa represents the time at 4:00 am and the unit is 30 minutes. The vertical axis is the occupancy rate, which is obtained by multiplying 100 by a value obtained by dividing the frequency of occupancy in a certain time period by the number of days on all weekdays or all holidays. As can be seen from these figures, if the attribute of the day is different, the occupancy pattern will be different. Therefore, here, the occupancy patterns are classified based on the attribute of the day.

FIG. 57 shows an example of the processing for classifying an occupancy pattern. 700 is occupancy data stored in the occupancy data storage means 690, 701 is a calendar, 702 is day information obtained from the calendar 701, and 703 is an occupancy pattern for each day. For example, when the current day is Wednesday, only data on Wednesday is extracted from the in-room data stored in the in-room data storage unit 609, and the frequency is counted for each time zone. Then, the occupancy rate by time is obtained by dividing the frequency by time by the number of days on all Wednesdays. Then, it is stored as a room presence pattern on Wednesday.

Next, the base temperature conversion means 692 calculates the value of the base temperature based on the occupancy pattern. FIG. 58 shows the relationship between the occupancy pattern and the base temperature.
710 is an occupancy rate by time, 711 is a base temperature as an output, and 712 is a standard set temperature at the time of occupancy. The horizontal axis of the graph is time, the left vertical axis is the occupancy rate, and the right vertical axis is temperature.
In the base temperature conversion method, the range of possible values of the base temperature is defined as lower limit p degrees to upper limit q degrees, the current time is defined as t, the base temperature at t as y (t), and the occupancy at t as z (t). In this case, y (t) = (q−p) × z (t + 1) + p. However, the unit of time is one hour. Fifth
If the life scene estimating unit 670 shown in the embodiment shows the exit scene and it is determined that the occupant is absent, the base temperature value for each time is set as the target value and the base temperature is changed according to the schedule. Then, when the entry of a person is detected, the target value is switched to the control index when there is a occupant. By reflecting the occupancy rate on the base temperature in this way, standby can be performed before a person enters the indoor environment, and a lower base temperature can be set during times when there are many absences, so economy is improved. Excellent air conditioning can be realized. Also, there is no need to use a timer except in special cases.

In this embodiment, the day of the week is used as the basis of the classification attribute of the room occupancy pattern. Can be obtained from the occupancy data. Further, except for the image processing unit, using the output of a human body detection sensor that detects only the presence or absence of a human body, to determine the presence or absence of the room, the in-room control index determination unit to determine the control index when in the room, An absent air conditioning control index determination unit having the room pattern classification and the base temperature conversion may be provided. In this case, the in-room control index may be temperature or comfort level. In addition, even when the user is absent, the comfort level PMV is maintained by using the minimum comfort level instead of the base temperature.
May be used as a control index.

Hereinafter, a seventh embodiment of the present invention will be described. In the seventh embodiment, as shown in FIG. 59, a remote control learning unit 720 and a remote control 721 are provided between the control index determination unit 611 and the air conditioning control unit 614 of the fourth embodiment, and Is learned as the user's preference, and the control index calculated by the control index determination unit 611 is corrected.

FIG. 60 is a block diagram of a remote control learning section 720 according to the seventh embodiment of the present invention. In FIG. 60, reference numeral 731 denotes a learning means. There are various methods for the learning means 731. In this embodiment, a neural network is shown as an example. 732 is a comfort level correction unit. 733 is the comfort level calculated by the comfort level detection unit 613, 734 is the life scene estimated by the life scene estimation unit 670, 735 is the time and calendar, and 736 is an operation panel obtained from a remote controller to which the user has declared the comfort level. An operation signal, 737 is a comfort level correction value, and 738 is a target comfort level as a final control index.

The operation of the above configuration shown in FIG. 60 will be described below. The calculated comfort level 733 obtained from the control index determination unit 611 and the remote controller 721,
Life scene 734, time / calendar 735, operation panel operation signal 73 from remote control declaring user's comfort level
6 is input as an input signal to a neural network which is a component of the learning means 731. The neural network estimates the operation of the remote controller for which the user has reported the comfort level. In other words, for the comfort level calculated by the control index determination unit 611, the user may select, for example, "hot"
Declaring "cold" etc. is judged to be the user's preference, and when similar environmental factors and human factors are observed, the control index is corrected based on the content of the user's preference. is there. The correction value 737 of the user's comfort output from the neural network is input to the comfort correction means 732, and the calculated comfort 733 is calculated to generate the target comfort 738. As for the calculation method, the simplest method is to add. When the calculation becomes complicated, the comfort correction value 737 and the calculated comfort 7
It can also be performed by a linear or non-linear function using the information of 33 as an argument. The learning means 731 has an input value 733 of each means.
To 735 using the remote control operation signal 736 as a teacher signal to perform neural network learning. After the learning, the state in the neural network is changed. In the neural network, various models of a pattern classification type can be used, but in this embodiment, an LVQ (learning vector quantu- lation) is used.
m: References, T. Kohonen, "Self-Organization and Asso
c-iative Memory ", 2ed, Springer-Verlag, 1988) The forward calculation of the neural network used in this embodiment will be described with reference to FIG.
Numeral 740 denotes an input signal normalizing unit, and 741 denotes a reference information unit. There are three categories according to human intention and m reference information in each of the three categories. 742 is a distance calculation / calculation unit, and 743 is a category calculation unit. The operation of the above configuration shown in FIG. 61 will be described below. The input signal 744 is normalized in an input signal normalizing section. That is, in the input signal normalizing section 740, Sx1 = (xmax-x1) / (xmax-xmin) (1) Sx1: normalized signal value xmax: maximum value of the input signal xmin: minimum value of the input signal The processing shown in the value equation (1) is performed on all input signals. The input signal is temperature, comfort level, life scene, time, and the like.
The signal normalized by the input signal normalization unit 740 is output to the distance calculation unit 742 as an input vector 745.
In the distance calculation unit 742, the input vector and the reference information unit 74
The distance to the reference information of each of the hot, cold, and satisfying categories representing one human intention is calculated. That is, the distance (dAj) from the reference information of the hot category is obtained by Expression 2.

DAj = Σ (xi-Raji) ² (2) dAj: distance from j-th reference information xi: i-th input vector value Raji: i-th vector value of j-th reference information Reference information The number of categories 3 and the number of pieces of reference information of each category are calculated with reference to the directory 746 of the unit 741.

The distance between the reference information and the input vector is calculated for each category, and the same processing is performed for all the categories to calculate the distance m . Distance calculator 742
The total number (dnum) of the distances m obtained by is: dnum = m1 + m2 +... + MN (3) Next, the distances dNj and d obtained by the distance calculation unit 742
num is output to the category calculation unit 743. In the category calculation unit 743, the shortest distance d among the dnum distances d
min is calculated, and the category to which the corresponding reference information belongs is calculated.

The category to which the reference information located at the shortest distance from the input vector belongs is set as the output of the category calculating section 743.
That is, it is an output of a neural network which is a component of the learning means 731 in FIG. This output is subjected to calculations such as being added to the comfort level calculated by the comfort level detection section 613 as a comfort level correction value 737 by the comfort level correction section 732.

The learning of the neural network will be described in detail with reference to FIG. 62, an input normalizing section 750 normalizes an input signal common to the input signal normalizing section 740 of FIG. 61 and converts it into an input vector. The reference information section 751 is also a reference information section common to FIG. Reference numeral 752 is a reference information search unit, 753 is a reference information generation unit, and 754 is a reference information learning unit. The operation of the above configuration will be described below.

The input signal normalized by the input signal normalizing section 750 is used as an input vector as a reference information search section 752.
Is entered. Further, the correct category (teacher data) of the input vector is also input at the same time. Reference information search unit 7
At 52, the directory of the reference information section 751 is referred to,
If the correct category (TC) of the input vector is, for example, a hot category, the following processing is performed when the number of pieces of reference information of the hot category has the following relationship.

If ARNUM <ARNUMmax, the processing of the reference information generation unit 753 is performed.

If ARNUM ≧ ARNUMmax, the processing of the reference information learning unit 754 is performed. ARUNM: the number of reference information of a hot category ARUNUM: the maximum number of reference information of a hot category The reference information generation unit 753 generates an input vector as reference information of a correct category (TC). Here, reference information of a hot category is generated, and ARUNM is increased by one.
The reference information learning unit 754 compares the output value of the remote control operation unit with the correct category of the input vector, and if they match, brings the shortest reference information closer. If the output value of the remote controller does not match the correct category of the input vector, the reference information having the shortest distance is kept away. That is, if OC = TC, then RV = RV + α (RV−Sx) if OC ≠ TC, then RV = RV−α (RV−Sx) RV: the input vector and the shortest reference vector OC: the output category (RV TC: correct category of input vector Sx: input vector α: learning rate Accordingly, the reference information learning unit 754 refers to the correct category of the input vector until the number of reference information exceeds the maximum value of the reference information. Generate as information. Also,
After the maximum value is exceeded, when the output category and the input category match, the reference information is slightly moved in the direction of the input vector, and when they do not match, the reference information is slightly moved away from the input vector. The distance to approach or move away is set by the learning rate α.

As described above, by learning the remote control operation using the neural network, by changing the combination of the input data, it is possible to correct the comfort index and the control index for each life scene. Air conditioning can be realized. In addition, by installing a remote control position sensor on the remote control and checking the correspondence between the foot position information and the remote control position sensor and using it as an input element of the neural network, it is possible to correct the control index for each individual and the control index for each position of the human body It is also possible to do.

Hereinafter, an eighth embodiment of the present invention will be described. The eighth embodiment relates to a traffic flow control device using the image processing device of the present invention. The traffic flow referred to here is a flow of a person related to transportation or transportation of a person, such as an elevator, a traffic light, and a transportation vehicle. Conventional traffic flow control means detects changes in elevator car load and predicts and controls traffic flow, but this method delays the detection of the number of people, and the control is a follow-up control. There were many things. This traffic flow control device accurately detects the number of people waiting for a means of transportation, distinguishing them from passers-by, and controls the traffic flow using the result. Figure 63 eighth
1 shows a configuration diagram of an embodiment of FIG. 800 is an image processing device, 80
1 is a traffic flow classification means, 802 is a traffic flow prediction means, 803
Is a traffic flow control means. With the above configuration, the number of people, particularly the number of people waiting for a vehicle or an elevator, is obtained from the image processing apparatus 800. Next, the traffic flow classification unit 801 extracts a feature amount related to the increase / decrease of the number of persons from the time series data of the number of persons obtained from the image processing apparatus 800 and classifies the pattern. In other words, it is classified into which time zone is crowded. In the traffic flow prediction means 802, the image processing device 80
The traffic flow is predicted from the current number of people obtained from 0 and the traffic flow classification pattern. For example, in the case of an elevator or the like, a time zone and a floor that are likely to be crowded are predicted. Traffic flow control means 80
In step 3, based on the predicted traffic flow, for example, the operation of a plurality of elevators is optimally controlled so that the waiting time of the person on each floor is reduced.

[0154]

According to the image processing apparatus of the present invention, the inside of the position judging unit is divided according to the type of posture in the step position detecting unit, for example, according to the number of postures to be judged as standing and sitting. The determination is performed, and the output from the position determination unit is selected and sorted based on the output from the posture determination unit. By performing such processing, more accurate position detection becomes possible.

Further, since it is difficult to detect environmental information in the detection area, for example, structural information such as the size of the room and the position of the wall surface with respect to the sensor if the room is indoors, it is difficult to detect from the thermal image.
The information can be directly input from the console unit by the user and used as auxiliary information for accurate position detection and posture detection.

[0156]

[0157]

[0158]

As described above, the control device of the air conditioner according to the present invention includes an image processing apparatus having an infrared image input means for measuring the temperature of a room occupant and the environment, and personal information obtained from the image processing apparatus. It has a control index determination unit that determines a control index for air conditioning from environmental information and environmental control, and an air conditioning control unit that controls air conditioning equipment. Comfortable air conditioning according to the position and condition of the individual can be realized. Further, since personal information and environmental information are extracted by performing image processing on the thermal image, it is not necessary to dispose a plurality of sensors indoors, and it is easy to realize the embodiment.

Further, by adding an absent-control index determining unit having the configuration of the occupancy data storage means, the occupancy pattern classification means, and the base temperature conversion means to the above configuration, it is possible to eliminate the timer and to improve comfort. The air-conditioning operation which has economy can be performed.

Further, by adding a remote control operation learning unit to the above configuration, it becomes possible to perform air conditioning control reflecting the user's preference.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an image processing apparatus according to a first embodiment of the present invention.

2A is a diagram illustrating an example of a thermal image according to the embodiment; FIG. 2B is a diagram illustrating an example of a plan view according to the embodiment;

FIG. 3 is a configuration diagram of a human area detection unit in the embodiment.

FIG. 4 is a diagram showing an example of block division of a thermal image in the embodiment.

FIG. 5 is a configuration diagram of a human region detection unit using a difference image in the embodiment.

FIG. 6 is a diagram showing an example of creating a difference image in the embodiment.

FIG. 7 is a configuration diagram of a personal information extraction unit in the embodiment.

FIG. 8 is a configuration diagram of a number-of-persons characteristic detecting unit in the embodiment.

FIG. 9 is a diagram showing an example of the number of persons determined in the embodiment.

FIG. 10 is a configuration diagram of a number detection means using a neural network in the embodiment.

FIG. 11 is a configuration diagram of detection of a foot position in the embodiment.

FIG. 12 is a schematic diagram of a thermal image of a standing human.

FIG. 13 is a view showing a method for calculating a position determination feature amount in the embodiment.

FIG. 14 is a configuration diagram of posture detection in the embodiment.

FIG. 15 is a diagram showing a processing example of a posture detecting unit in the embodiment.

FIG. 16 is a view showing a procedure of standardizing a human pixel block in the embodiment.

FIG. 17 is a view showing a method for calculating a posture determination feature amount in the embodiment.

FIG. 18 is a configuration diagram of a part of an image information detection unit in the embodiment.

FIG. 19 is a configuration diagram of a personal information extraction unit in the embodiment.

FIG. 20 is a configuration diagram of skin temperature detection in the embodiment.

FIG. 21 (a) is a diagram showing an example of block division of a standing person in the embodiment. (B) is a diagram showing an example of block division of a sitting person in the embodiment.

FIG. 22 is a configuration diagram of detection of a clothing amount in the embodiment.

FIG. 23 is a configuration diagram of an environment extraction unit in the embodiment.

FIG. 24 is a configuration diagram of a foot temperature detection unit in the embodiment.

FIG. 25 is a diagram showing a processing example of a foot temperature detecting unit in the embodiment.

FIG. 26 (a) is a diagram showing a thermal image of each part of the room in the case of a central installation in the embodiment. (B) A diagram showing a thermal image of each part of the room in the case of a left installation in the embodiment.

FIG. 27 is a configuration diagram of a radiation temperature detection unit in the embodiment.

FIG. 28 is a configuration diagram of a different environment extraction unit in the embodiment.

FIG. 29 is a configuration diagram of a different image processing unit in the embodiment.

FIG. 30 is a configuration diagram of a human area correction unit in the embodiment.

FIG. 31 is a configuration diagram of an accumulation processing unit in the embodiment.

FIG. 32 is a diagram showing an example of position accumulation in the embodiment.

FIG. 33 is a diagram showing a relationship between cumulative storage A and cumulative storage B in the embodiment.

FIG. 34 is a diagram showing a method for determining an air conditioner installation position in the embodiment.

FIG. 35 is a diagram showing a method of determining a room representative point in the embodiment.

FIG. 36 is a configuration diagram of an image processing apparatus according to a second embodiment of the present invention.

FIG. 37 is a configuration diagram of an image processing apparatus according to a third embodiment of the present invention.

FIG. 38 is a configuration diagram of an image processing apparatus in which a monitoring information detection unit is added to the image processing apparatus according to the third embodiment of the present invention.

FIG. 39 is a configuration diagram of an image processing apparatus in which a personal identification unit is added to the embodiment.

FIG. 40 is a configuration diagram of an integrated processing unit that integrates outputs of a plurality of image processing apparatuses according to the present invention.

FIG. 41 is a configuration diagram of a control device of an air conditioner using an image processing device according to a fourth embodiment of the present invention.

FIG. 42 is a configuration diagram of an activity amount detection unit in the embodiment.

FIG. 43 (a) is a diagram showing floor position division in the embodiment. (B) is a diagram showing an example of stored personal information in the embodiment.

FIG. 44A is a diagram showing stay frequency in the embodiment. FIG. 44B is a diagram showing correspondence between stay frequency and activity amount in the embodiment.

FIG. 45 (a) is a diagram showing an example of the definition of the periphery of the floor in the embodiment. (B) is a diagram showing an example of stay frequency calculation including the periphery in the embodiment.

FIG. 46 is a configuration diagram of an activity amount detection unit using a movement amount in the embodiment.

FIG. 47 is a diagram illustrating an example of a moving distance in an actual cleaning scene.

FIG. 48 is a configuration diagram of a movement amount detection unit in the embodiment.

FIG. 49 is a view showing an example of personal information stored in the embodiment.

50A is a diagram showing a method of deriving a period movement amount rank A in the embodiment. FIG. 50B is a diagram showing a method of deriving a period movement amount rank C in the embodiment.

FIG. 51 is a diagram showing a correspondence between a period movement amount and the like and an activity amount in the embodiment.

FIG. 52 (a) is a diagram illustrating an example of air conditioning when the embodiment is stable, and FIG. 52 (b) is a diagram illustrating an example of air conditioning when conditions in the embodiment are changed.

FIG. 53 is a configuration diagram of a control device for an air conditioner according to a fifth embodiment of the present invention.

FIG. 54 is a configuration diagram of a control index determination unit according to the sixth embodiment of the present invention.

FIG. 55 is a configuration diagram of an absence control index determination unit in the embodiment.

FIG. 56 (a) is a diagram showing the occupancy rate by weekday in a real home, and (b) is a diagram showing the occupancy rate by time on a holiday in a real home.

FIG. 57 is a view showing an example of classification of occupancy patterns in the embodiment.

FIG. 58 is a view showing the relationship between the occupancy pattern and the base temperature in the embodiment.

FIG. 59 is a configuration diagram of a control device for an air conditioner according to a seventh embodiment of the present invention.

FIG. 60 is a configuration diagram of a remote control learning unit in the embodiment.

FIG. 61 is a configuration diagram of a forward calculation of the neural network in the embodiment.

FIG. 62 is a configuration diagram of learning of a neural network in the embodiment.

FIG. 63 is a configuration diagram of a traffic flow control device according to an eighth embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Sensor part 2 Image processing part 3 Image information detection part 460 Console part 470 Visible sensor part (camera) 471 Visible image processing part 472 Visible image information detection part 480 Monitoring information detection part 503 Integrated processing part 611 Control index determination part 612 Activity amount Detector 613 Comfort detector 614 Air-conditioning controller 670 Living scene estimator 680 Absence control index determining unit 720 Remote control operation learning unit

 ──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Takehiko Shida 3-10-1, Higashi-Mita, Tama-ku, Kawasaki-shi, Kanagawa Prefecture Inside Matsushita Giken Co., Ltd. (72) Masaaki Sato 3-chome, Higashi-Mita, Tama-ku, Kawasaki-shi, Kanagawa No. 10 No. 1 Matsushita Giken Co., Ltd. (72) Inventor ▲ Yoshi ▼ Kunio Tadashi 3-chome, Tama-ku, Kawasaki City, Kanagawa Prefecture No. 1 Matsushita Giken Co., Ltd. (72) Inventor Ikuo Akamine Kadoma, Osaka 1006 Kadoma Matsushita Electric Industrial Co., Ltd. (72) Inventor Makoto Shimizu 1006 Kadoma, Kadoma City, Osaka Prefecture Matsushita Electric Industrial Co., Ltd. (72) Yoshiaki Uchida 1006 Kadoma Kadoma, Kadoma City, Osaka Matsushita Electric (56) References JP-A-3-247939 (JP, A) JP-A-2-157551 (JP, A) JP-A-62-240823 (JP, A)

Claims (26)

(57) [Claims] 1. An infrared image for measuring at least the temperature of a person or an environment in a detection area (hereinafter referred to as a thermal image)
And a temperature distribution from the thermal image and a human part
The area and its representative point are detected, and the size and shape of the area
Temperature and its surrounding area and surrounding temperature values as features
An image processing unit to be extracted and extracted;
Human information based on the feature amount and the thermal image
At least one of foot position, posture, amount of clothing, and skin temperature
And environmental information as the number of people and the ambient temperature from the foot position
-An image processing apparatus comprising: an image information detection unit that estimates a radiation temperature . Wherein the image processing unit includes an image storing unit for storing the thermal image measured even without to Ku, the region of human parts from the thermal image (hereinafter, referred to as human pixel block) is detected, humans pixel and a human region detecting section that detects a block and its representative point, the human human pixel block detected by the region detecting section of the area based on the size and shape and the area and the surrounding temperature
A personal information extraction unit that extracts the degree value as a feature
The image processing apparatus according to claim 1, wherein a. 3. A human region detection unit includes a region division representative temperature calculating means for calculating a representative temperature of each region by dividing the thermal image measured into a plurality of regions, determining a human body cutout temperature for each representative temperature of each area and clipping the temperature determining means, and the human detecting unit that performs temperature cut out thermal image cutout temperature range defined for each of the regions, to have a representative point calculation unit that calculates a representative point of the excised human pixel block the image processing apparatus according to claim 2 Symbol mounting features. 4. The number of persons for detecting a feature quantity for detecting the number of persons from a thermal image obtained from the image storage unit, a human pixel block obtained from a human area detection unit, and a representative point thereof, a feature detection unit, the number feature detection unit
Wherein the number obtained by the image information detecting unit, and the posture determining feature amount detecting section that detects a characteristic quantity for determining the human attitude of at least the detection area, the foot position of the human number feature amount based on from A position determination feature value detection unit that detects a feature value for determining the amount of clothing, a clothing amount determination feature value detection unit that detects a feature value for determining the clothing amount of the same person, and determines the skin temperature of the same person the image processing apparatus according to claim 2 Symbol placement and having any one of the skin temperature determination feature detector for detecting a feature quantity for. 5. A number-of-people characteristic detecting unit, wherein a number-of-pixels calculating means for counting the number of pixels of a human pixel block from a human pixel block obtained from the human region detecting unit and a representative point thereof, 5. The image processing apparatus according to claim 4 , further comprising: a block type determining unit that classifies the image into a shape type based on an aspect ratio, and a human pixel block counting unit that counts the number of human pixel blocks. 6. The detection of the number of persons by the image information detection section includes the step of detecting the shape type obtained from the number of persons feature detection section and the detection of the human area.
Claim to the number of pixels obtained from the output unit and wherein the estimating the number using the neural network from the representative point 5
The image processing apparatus according to any one of the preceding claims. 7. The image information detecting unit estimates a foot position, a posture, a skin temperature, and a clothing amount from each of the feature amounts extracted from the personal information extracting unit by using a neural network. Item 2. The image processing apparatus according to Item 1 . 8. The detection of the skin temperature of the image information detecting section is performed individually.
Based on the skin temperature estimation feature amount calculated by the skin temperature determination feature amount detection unit of the human information extraction unit and the detected posture, the human pixel is matched to a human body part by block dividing means having a pattern recognition mechanism. The block is divided into a plurality of parts, the average temperature of each block obtained from the block dividing means is calculated by the block temperature calculating means, and the temperature of each block obtained by the block temperature calculating means is compared by the temperature comparing means. Extract the block corresponding to
The image processing apparatus according to claim 4, wherein skin temperature estimation is performed. 9. The detection of the amount of clothing by the image information detection unit is performed by an individual.
From the clothing amount estimation feature amount calculated by the clothing amount determination feature amount detection unit of the information extraction unit, a human pixel block is divided into a plurality of blocks according to a human body part by block dividing means having a pattern recognition mechanism, and the block is divided into blocks. Calculate the average temperature of each block obtained from the block dividing means in the temperature calculating means, compare the temperature of each block obtained in the block temperature calculating means by the temperature comparing means, and extract the block corresponding to the skin surface, 5. The image processing apparatus according to claim 4, wherein the amount of clothes is determined by calculating an area ratio of a block corresponding to an exposed portion of the human body obtained from the temperature comparing means and a block corresponding to a surface portion of the clothes by the area comparing means. apparatus. 10. A difference image creating means for calculating a difference between two consecutive thermal images by a human region detecting section, and a moving object image creating means for creating an image of a moving object from the two thermal images and the difference image Means, a frequency distribution creating means for creating a frequency distribution of pixel values of the moving object image, a cutout temperature determining means for determining a cutout temperature of a person from the frequency distribution, and a human detecting section for performing temperature cutout with the cutout temperature width. The image processing apparatus according to claim 2, further comprising: a representative point calculation unit configured to calculate a representative point of the extracted human pixel block. 11. Environmental information extraction section provided in the image processing section includes a thermal image obtained from the image storage unit, and the foot temperature detection means for detecting the temperature in the vicinity of the foot position from the detected foot position information, the An indoor part temperature detecting means for detecting the temperature of each part from the correspondence between the pixels in the thermal image and the parts such as walls and floors in the room, and the foot position and the detected room interior
3. The image processing apparatus according to claim 2 , further comprising a radiation temperature detecting unit configured to detect a radiation temperature near a person by using a positional relationship with each part . 12. An environment information extracting section, comprising: an accumulating section for accumulating representative points of a human pixel block obtained by a human area detecting section for a predetermined period; and a predetermined point of a representative point of a human pixel block obtained from the accumulating section. At least floors and walls, from the cumulative period
Or to determine the room representative point is the boundary wall and the wall, according to claim 11, characterized in that the addition of the indoor part determination unit that associates correspondence between pixels and indoor floors and walls on the thermal image
Serial mounting image processing apparatus. 13. An accumulation processing unit, comprising :
Accumulation storage means for accumulating the representative points of the block, and accumulation for each pixel.
Stationary object determination means for determining a stationary object from the value of the value,
A still object image that stores the pixel position of the determined still object
13. The apparatus according to claim 12, further comprising a raw position storage unit.
Image processing device . 14. An environmental information extracting unit, comprising: a floor heating detecting unit that distinguishes a human body from floor heating according to a size of a human pixel block obtained by a human region detecting unit; and a thermal image obtained from the image storage unit. 12. The image processing apparatus according to claim 11 , wherein at least one of auxiliary heating detection means for detecting a high-temperature heating object is added. To 15. The image processing apparatus according to any of claims 1 to 14, by adding a console unit for the user environment information about the detection area can not be detected directly from the thermal image is input, less the information An image processing device for inputting the information to an image processing unit or an image information detection unit to correct information related to a human. 16. A method for detecting a temperature distribution from an infrared image (hereinafter, referred to as a thermal image) for measuring at least the temperature of a human or an environment in a detection area, a human part area, and a representative point thereof.
The size and shape of the area and the area and surrounding
The temperature value is extracted as a feature value, and the extracted features
Based on the amount and the thermal image,
At least one of position, posture, amount of clothing, and skin temperature
Ambient temperature and radiation from the number of people and foot position
An image processing method characterized by estimating a temperature . 17. The environmental information may include a thermal image and a foot position.
Temperature near the feet, indoor wall, floor, and ceiling temperatures
17. The image processing method according to claim 16, wherein 18. The method according to claim 18 , wherein the posture detection is performed on a large area of the human part.
Detecting standing, sitting, and lying postures based on size and shape
17. The image processing method according to claim 16, wherein: 19. An individual or proxy based on the amount of movement of the foot position from the personal information and the environment information obtained by the image processing method according to claim 16.
Determine the amount of activity to represent, the amount of activity, the personal information and the
Environmental information and wind direction and volume from air conditioner, room temperature, humidity
Temperature or at least one of the outside temperatures
A control device for an air conditioner, comprising: a control index determining unit that determines an appropriate control index; and an air conditioning control unit that controls an air conditioner based on the control index. 20. includes an image processing apparatus according to any one of claims 1 to 15, and personal information obtained from the image processing apparatus, the environment information, feet positions from the personal information
The amount of individual or representative activity is determined from the amount of movement of
Activity amount, personal information, environmental information and air conditioner
At least one of wind direction and volume, room temperature, humidity, and outside temperature
A control index determined tough for determining the control index as individual or representative Comfort from One, the air conditioner based on the control indication
A control device for an air conditioner, comprising an air-conditioning control unit for controlling an air conditioner. 21. control index determination unit adds the amount of activity obtained from at least the activity amount detecting unit, the time and the life scenes estimation unit that estimates a living from the environment information which is the output of the environmental information extraction unit scenes Estimating a transient life scene in which the comfort level is not a control index and a stable life scene in which the occupant is in the room for a predetermined period and the comfort level can be used as the control index, and 21. The control device for an air conditioner according to claim 20 , wherein a predetermined control index predetermined for each comfort level and scene is selected. 22. A remote controller capable of reporting a temperature or a comfort level, at least a control index calculated by the control index determining unit, and a user's report from the remote controller, and outputting a correction value of the control index. 22. The control device for an air conditioner according to claim 20 , wherein a remote control operation learning unit having a neural network is added to learn the environmental conditions when the user performs a remote control operation with the above configuration. . 23. An occupancy data storage means for storing occupancy data as a time series detection result of a person entering and exiting a room in a control index determining unit, and an occupancy pattern for classifying the occupancy data into similar patterns. A classifying means, and an absence control index determining unit having a configuration of a base temperature conversion means for determining a base temperature, which is a minimum maintenance temperature of the absence time, according to the use state of the room obtained from the presence pattern classifying means. The control device for an air conditioner according to any one of claims 20 to 22, wherein: 24. The detection of the amount of activity by the control index deciding section is performed by
A personal information storage means for storing a predetermined period of personal information from the personal information extraction section of Kiga image processing apparatus accumulates the foot position information obtained from the personal information extraction means by a predetermined time period position, a still's determination Stay frequency calculation means, having a configuration of activity amount calculation means for calculating the activity amount from the stationary person determination result obtained from the period movement amount and the stay frequency calculation means,
21. The control device for an air conditioner according to claim 20, wherein the activity amount is calculated from a frequency of staying at a floor surface position for a predetermined period. 25. The detection of the amount of movement by the control index determining unit includes : step position information obtained from two consecutive thermal images obtained from a personal information extracting unit of the image processing device; Moveability determining means for searching for a combination of foot position information that can be moved from the maximum moving distance capable of moving in time, and nearest neighbor search means for searching for the nearest occupant from the combination information and determining the combination 25. The control device for an air conditioner according to claim 24, further comprising moving speed calculating means for calculating a moving speed from a determined combination. 26. A calendar, a sensor for detecting a person in the sensor section, an absent / absence determination section for determining presence / absence of an occupant, a occupancy control index determination section for determining a occupancy control index, and a request. Item 23. A control device for an air conditioner, comprising: a control index determining unit including the absence control index determining unit according to Item 23; and an air conditioning control unit that controls an air conditioner.