JP2013024534A - Situation recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a situation recognition device to execute a situation recognizing processing of high accuracy even at home.SOLUTION: This situation recognition device for recognizing domestic situation, includes a human movement detecting means for detecting human movement information, a situation recognition processing determining means for determining whether the situation recognizing processing is to be executed based on the detection results of the human movement detecting means, and a situation recognizing means for recognizing domestic situation when the execution of the situation recognizing processing is determined by the situation recognition processing determining means. The situation recognizing processing of high accuracy can be executed even at home by estimating the human movement information by the human movement detecting means, and performing the situation recognizing processing only when the human moves.

Description

  The present invention relates to a situation recognition technique in a home, and more particularly to an air conditioner capable of performing an optimal air conditioning operation reflecting the situation of an air conditioning space.

  By identifying the person in the air-conditioned space without learning the user at all, and automatically learning the equipment operation tendency of the person, the universal design can be realized, and the user's favorite air-conditioning can be realized. Air conditioners are known.

  For example, in Patent Document 1, when a user operates a control instruction unit, a person who operates the control instruction unit is specified, and the tendency of the person to operate the air conditioner can be learned according to the operation content. Further, as a means for identifying a person, Patent Document 2 describes that when a registrant is limited to a small number, personal identification can be made based on height and physique.

  Further, as a means for acquiring height information, Patent Document 3 detects a person's feet and the top of the head from a photographed image, and becomes a target based on a difference in height between the detected position of the foot and the position of the top of the head. A technique for estimating the height of a person is disclosed. Further, as another situation recognition technique, in Patent Document 4, by using a camera, the lowest end of a moving object is obtained as a point where the moving object and the floor surface come into contact, and by using history information of the contact information, A method for detecting obstacles is described. By detecting the position where an obstacle that obstructs the airflow exists, it is possible to control the airflow throughout the air-conditioned area.

  In order to estimate the height, it is necessary to estimate the camera parameter. However, Patent Document 5 discloses means for calculating a vanishing line that is one of the camera parameters from a captured image.

JP 2010-091228 A JP 2010-107114 A JP 2001-56853 A JP 2010-190432 A JP 2007-212216 A

  However, the conventional situation recognition technology has been developed for use as a surveillance camera and is difficult to use as it is in the home.

  This is because, for example, in the outdoors where surveillance cameras are widely used, it is normal for a person to move around, and a person staying there is irregular. On the other hand, at home, a person is often sitting or sleeping, and is often stationary. Patent Document 3 discloses a method for estimating height from the distance between the position of the top of the head and the position of the foot, but a person often sits in the home. In this case, the position of the top of the head and the position of the foot The height could not be calculated accurately from the distance.

In Patent Document 3, the head position and the foot position are obtained from the difference information. However, when the person is stationary, the whole body of the person cannot be accurately detected by many difference processes. It is difficult to accurately acquire the part position and the foot position. Therefore, when the person is stationary, the height cannot be calculated accurately.

  In Patent Document 4, the lowest end of the moving object is obtained as a point where the moving object contacts the floor surface. However, when the lower body of the person is shielded, it is difficult to obtain the foot position from the image. . Also, for example, as shown in FIG. 22, when there is a person 321 who wipes the table from a direction away from the image sensor unit 24, the position of the person's arm is erroneously determined as the foot position, There is a high possibility that the area where the obstacle exists, that is, the table, is determined as the area where the obstacle does not exist.

  In order to achieve the above object, the situation recognition apparatus of the present invention is configured to detect whether or not a person movement detection unit detects the movement information of a person in the home and whether or not to perform the situation recognition process based on the detection result of the person movement detection unit. And a situation recognition unit for recognizing a situation in the home when it is determined by the situation recognition process judgment unit that the situation recognition process is to be performed. The object is to realize highly accurate situation recognition processing by executing situation recognition processing only when a person is moving.

  According to the present invention, it is possible to realize a highly accurate situation recognition technique in the home by using person movement information. Furthermore, by utilizing the situation recognition technology of the present invention, it is possible to recognize the situation of the air conditioning space and perform an optimal air conditioning operation.

1 is a functional block diagram showing a configuration of a situation recognition device according to a first embodiment of the present invention. The functional block diagram which shows the structure in the case of performing the height estimation process as a situation recognition process in the situation recognition apparatus in the 1st Embodiment of this invention The flowchart which shows the flow of the height estimation process in the 1st Embodiment of this invention The flowchart which shows the flow of the detection process of a person's presence / absence and position information of the person movement detection process of this invention The schematic diagram for demonstrating the background difference process in the person movement detection means of this invention Schematic diagram for explaining processing for creating a background image in background difference processing Schematic diagram for explaining processing for creating a background image in background difference processing Schematic diagram for explaining processing for creating a background image in background difference processing The schematic diagram for demonstrating the area division | segmentation process in the person movement detection process of this invention The flowchart which shows the flow of the height estimation process of this invention Schematic diagram for explaining three coordinate systems used in the present invention Schematic diagram for explaining the three-dimensional position of the foot position in the camera coordinate system Schematic diagram for explaining the three-dimensional position of the head position in the camera coordinate system Functional block diagram showing the configuration of the situation recognition apparatus according to the second embodiment of the present invention, which performs an obstacle detection process as the situation recognition process The flowchart which shows the flow of the obstruction detection process in the 2nd Embodiment of this invention. The flowchart which shows the flow of the obstacle map update process of the obstacle detection means in the 2nd Embodiment of this invention. Schematic diagram of home to explain obstacle map and short-term obstacle map FIG. 17 is a schematic diagram of an image taken by the image sensor unit in the home shown in FIG. Schematic diagram for explaining a method of acquiring an obstacle map as a probability that no obstacle exists by storing a person's movement history as an obstacle map Schematic diagram showing an example of a short-term obstacle map Schematic diagram showing an example of an obstacle map Schematic diagram of an example in which foot position estimation fails in the prior art Schematic diagram showing a short-term obstacle map and another area division of the obstacle map Functional block diagram showing the configuration of the situation recognition apparatus according to the third embodiment of the present invention, which performs camera parameter estimation processing as situation recognition processing The flowchart which shows the flow of the camera parameter estimation process in the 3rd Embodiment of this invention. Schematic diagram for explaining vanishing lines The functional block diagram which shows the structure of the air conditioner carrying the condition recognition apparatus in the 1st Embodiment of this invention. The front view of the indoor unit of the air conditioner which concerns on this invention 28 is a longitudinal sectional view of the indoor unit of FIG. 28 is a longitudinal sectional view of the indoor unit in FIG. 28 in a state where the movable front panel opens the front opening and the upper and lower blades open the outlet. 28 is a longitudinal sectional view of the indoor unit of FIG. 28 in a state where the lower blades constituting the upper and lower blades are set downward. The flowchart which shows the flow of a process of the air conditioner carrying the height estimation apparatus in the 1st Embodiment of this invention. The functional block diagram which shows the structure of the air conditioner carrying the condition recognition apparatus in the 2nd Embodiment of this invention. The flowchart which shows the flow of a process of the air conditioner carrying the obstruction detection apparatus in the 2nd Embodiment of this invention. Schematic showing the human position area detected by the imaging sensor unit Schematic showing the definition of the wind direction at each position of the left and right blades constituting the left and right blades The functional block diagram which shows another structure of the air conditioner carrying the condition recognition apparatus of this invention The flowchart which shows the flow of another process of the air conditioner carrying the height estimation apparatus in the 1st Embodiment of this invention.

  A first invention is a situation recognition apparatus for recognizing a situation in a home, and a person movement detection means for detecting movement information of a person and whether the situation recognition processing is performed based on the detection result of the person movement detection means. The situation recognition processing judgment means for judging whether or not and the situation recognition means for recognizing the situation in the home when the situation recognition processing judgment means judges that the situation recognition processing is to be performed.

  With this configuration, the person movement detection means estimates the person movement information and performs the situation recognition process only in a situation where the situation recognition process can be performed with high accuracy. Can be realized.

  According to a second aspect of the present invention, the situation recognition process is performed by the person movement detection means when the person is moving, and the situation recognition process is not performed otherwise. Recognition processing can be realized.

According to a third aspect of the present invention, when the person is moving in the orthogonal direction with respect to the vector connecting the head and the foot, the person movement detecting means performs height estimation and moves in the vector direction. In some cases, the situation recognition process can be realized even in the home by not performing the situation recognition process.

  According to the fourth aspect of the present invention, a high-precision height estimation process can be realized even in the home by using a height estimation unit that detects the height of a person as the situation recognition unit.

  According to a fifth aspect of the present invention, as the height estimating means, a foot position detecting means for detecting the foot position of a person, a head position detecting means for detecting the head position of the person, and foot position information obtained by the foot position detecting means. And height estimation means for detecting the height of a person using head position information obtained by the head position detection means.

  With this configuration, it is possible to estimate the movement information of a person by the person movement detection means, and perform the height estimation process only in a situation where the height estimation process can be performed with high accuracy, thereby enabling high-precision height estimation processing even at home. Can be realized.

  According to the sixth aspect of the present invention, by using an obstacle detection means for detecting an obstacle in the home as the situation recognition means, a highly accurate obstacle detection process can be realized even in the home.

  According to the seventh aspect of the present invention, it is possible to realize highly accurate obstacle detection processing even in the home by detecting an obstacle using the movement history of the person as the obstacle detection means.

  According to the eighth aspect of the present invention, it is possible to realize highly accurate obstacle detection processing even in the home by detecting an obstacle using the history of the person's foot position as the obstacle detection means.

  According to the ninth aspect of the present invention, high-precision camera parameter estimation processing can be realized even in the home by using camera parameter estimation means for performing camera parameter estimation processing as the situation recognition means.

  According to the tenth aspect of the present invention, high-precision camera parameter estimation can be realized even in the home by estimating a vanishing line as a camera parameter as camera parameter estimation means.

  An eleventh aspect of the present invention is an air conditioner equipped with the situation recognition apparatus of the present invention, and has an air conditioning control unit that realizes air conditioning control based on the identification result identified by the person identifying means.

  With this configuration, the situation can be recognized with high accuracy even in the air-conditioned space, so that an optimal air-conditioning operation can be realized.

  A twelfth aspect of the present invention is an air conditioner equipped with the situation recognition apparatus of the present invention, and further includes a recognition result display means for displaying a recognition result recognized by the situation recognition means.

  With this configuration, the user can recognize whether or not the situation identification device is performing the identification, and thus can be used with confidence.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a functional block diagram showing the configuration of the situation recognition apparatus according to the first embodiment of the present invention. Furthermore, FIG. 2 is a functional block diagram showing a configuration in the case of performing a height estimation process as the situation recognition process. That is, the situation recognition unit 103 includes a foot position detection unit 104, a head position detection unit 105, and a height estimation unit 106. FIG. 3 is a flowchart showing the flow of the height estimation process in the present embodiment. In FIG. 3, the person movement detection means 101 uses the image sensor unit 24 to capture an image of the home and obtain a frame image (step S101).

  The person movement detecting means 101 detects the presence / absence and position information of the person using the acquired frame image (step S102). If a person exists, whether the person is still staying, Movement information such as how it is moving is detected (step S103). Based on the detection result of the person movement detection unit 101, the situation recognition process determination unit 102 determines whether to perform the height estimation process on the current frame image (step S104).

  That is, when the situation recognition process determining unit 102 determines that the person is moving (Yes in step S104), the foot position detecting unit 104 estimates the foot position of the person (step S105). Further, the head position detecting means 105 estimates the head position of the person (step S106). The height estimating means 106 detects the height of the person using the foot position information obtained by the foot position detecting means 104 and the head position information obtained from the head position (step S107).

  On the other hand, when the situation recognition processing determination unit 102 determines that no person exists or has moved (No in step S104), the height estimation unit 106 estimates the height of the current frame image. Without performing the process, the previous height information is held as the current height information (step S108).

<Configuration of person movement detection means>
The image sensor unit 24 includes a circuit board, a lens attached to the circuit board, and an image sensor mounted inside the lens. Also, the person movement detection unit 101 determines the presence / absence of a person using a circuit board based on, for example, a difference process described later, and further estimates the position of the person.

  That is, the circuit board acts as the person movement detection unit 101. Of course, the situation recognition processing determination means 102 and the situation recognition means 103, which will be described later, are also realized by a circuit board, but detailed description thereof is omitted.

<Detection of person movement by image sensor unit>
In order to detect person movement by the image sensor unit 24, a difference method which is a known technique is used. This is to perform a difference process between a background image that is an image in which no person is present and an image captured by the image sensor unit 24, and to estimate that a person is present in a region where the difference is generated.

FIG. 4 is a flowchart showing the flow of the process of detecting the presence / absence of a person and position information in step S102 of the person movement detecting means in the present embodiment. In step S201, a background difference process is used to detect a pixel that has a difference in the frame image acquired in step S101. Background difference processing is a background image captured under specific conditions,
By comparing the background image and the captured image captured under the same imaging conditions such as the field of view, viewpoint, and focal length of the image sensor unit, objects that are not present in the background image but present in the captured image are detected. It is a technique to do. In order to detect a person, an image without a person is created as a background image.

  FIG. 5 is a schematic diagram for explaining the background difference processing. FIG. 5A shows a background image. In this figure, 301 indicates a window existing in the home, and 302 indicates a door. FIG. 5B shows the frame image captured in step S <b> 101 by the image sensor unit 24. Here, the field of view, viewpoint, focal length, and the like of the image sensor unit are the same as the background image of FIG.

  Reference numeral 303 denotes a person existing in the home. In the background difference process, a person is detected by creating a difference image shown in FIGS. 5 (a) and 5 (b). FIG. 5C shows a difference image. White pixels indicate pixels where no difference exists, and black pixels indicate pixels where a difference occurs. It can be seen that the area of the person 303 that is not present in the background image but is present in the captured frame image is detected as the area 304 in which the difference occurs. That is, it is possible to detect a person area by extracting an area where a difference occurs from the difference image, and to detect the presence or absence of a person.

  Further, the above-described background image can be created by using inter-frame difference processing. 6 to 8 are schematic diagrams for explaining this processing. FIGS. 6A to 6C are schematic views illustrating three consecutive frames captured by the image sensor unit 24 in a scene in which the person 303 is moving from right to left in front of the window 301. is there. FIG. 6B shows an image of the next frame of FIG. 6A, and FIG. 6C shows an image of the next frame of FIG. 6B.

  FIGS. 7A to 7C show inter-frame difference images obtained by performing inter-frame difference processing using the image of FIG. White pixels indicate pixels where no difference exists, and black pixels 305 indicate pixels where a difference occurs. Here, if the only object moving within the field of view is a person, it is considered that no person exists in a region where no difference occurs in the inter-frame difference image. Therefore, in a region where no inter-frame difference occurs, the background image is replaced with the current frame image. By repeating this process, a background image can be automatically created.

  FIGS. 8A to 8C are diagrams schematically showing the update of the background image in each frame of FIGS. 6A to 6C. A hatched area 306 indicates an area where the background image has been updated, a black area 307 indicates an area where a background image has not yet been created, and a white area 308 indicates an area where the background image has not been updated. That is, the total area of the black area 307 and the white area 308 in FIG. 8 is equal to the black area 305 in FIG. As shown in the figure, when a person is moving, it can be seen that the black region 307 gradually decreases and a background image is automatically created.

  Next, in step S202, the obtained difference area is divided into areas, and if there are a plurality of persons, the difference areas are divided into a plurality of difference areas. The position information of the person is acquired as the position information of each difference area thus obtained. In addition, in order to perform such region division, a known image clustering method may be used. For example, “a pixel in which a difference is generated and a pixel in which a difference existing in the vicinity of 8 is the same are the same. The difference image may be divided into regions in accordance with the rule of “region of

FIG. 9 is a schematic diagram in which this area division processing is executed. FIG. 9A shows a difference image calculated by difference processing, and the black pixels 309 and 310 are pixels in which a difference occurs. FIG. 9B shows that when FIG. 9A is obtained as a difference image, the “pixel in which the difference is generated and the pixel in which the difference existing in the vicinity of the 8 is generated are the same region. The result of area division according to the rule “is shown.

  Here, it is determined that the horizontal stripe region 311 and the vertical stripe region 312 are different regions. At this time, noise removal processing such as morphological processing widely used in image processing may be performed.

  Next, in step S103, movement information such as whether the person is staying or how the person is moving is detected from the amount of change in each area. Here, as the movement information, it is determined whether or not “a plurality of persons are not overlapped and one person is moving in a direction substantially orthogonal to a vector passing through the head and feet”. This is realized by performing spatiotemporal filtering on the position information of the detected area. That is, it is confirmed whether or not the following four conditions are satisfied.

1. Does the position change within a certain range occur over a plurality of frames, and whether there is a difference area whose magnitude change is within the certain range?
2. Is position information larger than a certain area detected?
3. Is the aspect ratio of the difference area greater than a certain value?
4). Is the detected moving direction of the person orthogonal to the vector passing through the person's head and feet?
If the above four conditions are satisfied, it is determined that there is a person at that position and that the person is moving.

  First, in condition 1, temporal filtering processing is performed. That is, when the difference area is a person, the moving speed is not faster than the walking speed (position change within a certain range), and the size does not change greatly (the magnitude change is within a certain range). ). On the other hand, when the difference area is caused by a change in the state of light, the size and position vary greatly. Therefore, by using Condition 1, noise other than a person can be removed. Such a condition can be realized, for example, by continuously detecting an area in which the difference in the center of gravity u coordinate value is within 10 pixels and the difference in the width of the difference area is 2 pixels or less continuously in 5 frames or more.

  In condition 2, filtering by size is performed, and noise such as calendars moved by wind is removed. This can be realized, for example, by setting the number of pixels in the horizontal direction of the difference area to 4 or more.

  Condition 3 performs filtering by shape on the assumption that a person is walking. By such filtering, for example, a pet moving around in the space, a state in which height estimation is difficult, and a plurality of persons overlapped can be removed from the difference region. Such a condition can be realized, for example, by setting the height of the difference area to 1.5 times or more the horizontal width of the detection frame. Here, the area where a plurality of persons overlap is removed as noise because the moving direction of the person cannot be accurately obtained in such a situation.

  In condition 4, the movement is limited to that the person is moving in a direction (hereinafter referred to as the horizontal direction) orthogonal to a vector passing through the person's head and feet (hereinafter referred to as the vertical direction). This is a condition for performing detection. When the person is moving in the vertical direction, the appearance on the image is difficult to distinguish from the case where the person sitting on the chair or sofa facing the imaging sensor unit 24 stands up. When a person sitting facing the imaging sensor unit 24 stands up, only the upper body changes and the lower body hardly changes on the image.

  Therefore, when a difference image is created, only the upper body is detected, and it is difficult to accurately estimate the height. Therefore, in the height estimation device according to the present embodiment, the height estimation accuracy can be increased by performing the height estimation process only for the person moving in the horizontal direction. Such a condition is, for example, that the difference in the uppermost v-coordinate value of the difference area is 2 pixels or less in the preceding and following frames, and the difference in the centroid u coordinate value of the difference area in the preceding and following frames is 2 pixels or more and 10 pixels. It can be realized by filtering that is within.

  The person movement detection unit 101 of this embodiment determines that the difference area is a moving person when all of the above four conditions are satisfied. On the other hand, if any one of the four conditions is not satisfied, it is determined that the difference area is not a moving person.

  The situation recognition processing determination unit 102 determines whether to perform height estimation based on the detection result of the person movement detection unit. Here, when the person movement detection means 101 determines that the difference area is a moving person, the situation recognition process determination means 102 determines that the current frame image is suitable for height estimation. , Execute the height estimation process.

  On the other hand, when the person movement detection unit 101 determines that the difference area is not a moving person, the situation recognition processing determination unit 102 determines that the current frame image is not suitable for height estimation, The height estimation process is not performed, the height estimation process is not performed on the current frame image, and the previous height information is held as the current height information.

<Height estimation means>
When the situation recognition process determination unit 102 determines that the person is moving (Yes in step S104), the foot position detection unit 104 estimates the foot position of the person (step S105). Further, the head position detecting means 105 estimates the head position of the person (step S106). The height estimating means 106 detects the height of the person using the foot position information obtained by the foot position detecting means 104 and the head position information obtained from the head position (step S107). These processes will be described in detail.

  FIG. 10 is a flowchart showing the flow of the height estimation process.

  First, the foot position detecting means 104 estimates the foot position from the difference area on the image (step S203). Here, the foot position on the image is obtained as the barycentric coordinates of the pixel in which the background difference occurs in the pixels existing in the lower end area of the difference area. Of course, the bottom end of the difference area may be selected when obtaining the foot position on the image.

  Next, the foot position in the real space is calculated using the foot position on the obtained image (step S204). In order to detect the foot position in the world coordinate system from the foot position on the image, perspective projection transformation may be used.

  In the following description, three coordinate systems are used. First, an image coordinate system and a camera coordinate system will be described in order to explain perspective projection transformation. FIG. 11A is a schematic diagram for explaining these two coordinate systems. First, consider the image coordinate system. This is a two-dimensional coordinate system in the captured image, where the upper left pixel of the image is the origin, u is rightward, and v is downward. Next, consider a camera coordinate system, which is a three-dimensional coordinate system based on the image sensor unit 24. In the camera coordinate system, the focal position of the image sensor unit is the origin, the optical axis direction of the image sensor unit 24 is Zc, the image sensor unit upward is Yc, and the image sensor unit left direction is Xc. At this time, the following relationship is established by perspective projection conversion.

  Here, f is the focal length [mm], (u0, v0) is the image center [Pixel] on the image coordinates, and (dpx, dpy) is the size [mm / Pixel] of one pixel of the image sensor. . Here, paying attention to the fact that Xc, Yc, and Zc are unknown numbers, when the coordinates (u, v) on the image are known, [Equation 1] is the actual three-dimensional position corresponding to the coordinates: It can be seen that it exists on a certain straight line passing through the origin of the camera coordinate system.

  Here, it is assumed that the foot position on the image is (uf, vf). Further, as shown in FIGS. 12A and 12B, the three-dimensional position in the camera coordinate system of the foot position indicated by the point 315 is defined as (Xfc, Yfc, Zfc). FIG. 12A is a schematic view of the air-conditioned space viewed from the side, and FIG. 12B is a schematic view of the air-conditioned space viewed from above. Further, it is assumed that the height at which the image sensor unit is installed is H and the Xc direction is equal to the horizontal direction, and that the foot position is on the floor surface, that is, on the plane having the height H from the image sensor unit. Assume. The three-dimensional position (Xfc, Yfc, Zfc) of the foot position in the camera coordinate system can be calculated by the following equation from the perspective projection transformation of [Equation 1].

  [Expression 2] is rewritten as follows.

  If f, u0, v0, dpx, and dpy are already known by existing calibration (camera parameter estimation) processing, β and φ in [Equation 2] and [Equation 3] are uniquely determined by (uf, vf). It is determined. Therefore, the unknown in [Equation 3] is only Zfc.

  Now consider another coordinate system, the world coordinate system. FIG. 11B is a schematic diagram for explaining the world coordinate system. Here, it is assumed that the Xc direction of the image sensor unit 24 is a horizontal direction. In the world coordinate system, the focal position of the image sensor unit is the origin, the image sensor unit left direction (Xc axis direction) is Xw, the Xw-Yw plane is Yw, and the vertical downward direction is Zw. At this time, the camera coordinate system (Xc, Yc, Zc) and the world coordinate system (Xw, Yw, Zw) satisfy the following relationship.

Here, the matrix R is a 3 × 3 matrix transformation matrix. Here, when the installation direction of the image sensor unit 24 is known, the matrix R is known. That is, when (Xc, Yc, Zc) in the camera coordinate system is known, the world coordinate system (Xw, Yw, Zw) is also uniquely obtained by [Equation 4].

  Therefore, from [Equation 3] and [Equation 4], the foot position (Xfw, Yfw, Zfw) in the world coordinate system is obtained from the following equation.

  Here, considering that the image sensor unit is normally installed at a height of H = about 2 m, the following equation holds.

  Therefore, by solving [Equation 5] and [Equation 6], the foot position (Xfw, Yfw, Zfw) in the world coordinate system can be uniquely obtained.

  Next, a process in which the head position detecting unit 105 estimates the head position of a person from the image will be described. First, the head position detecting means 105 estimates the head position from the difference area on the image (step S205). Here, the head position on the image is obtained as the barycentric coordinates of the pixel in which the background difference occurs in the pixel existing in the upper end area of the difference area. Of course, when obtaining the head position on the image, the top end of the difference area may be selected.

  Next, the head position in the world coordinate system is calculated using the head position on the image obtained in step S205 and the foot position in the world coordinate system obtained by the foot position detection means 104 in step S204. (Step S206). In order to detect the head position in the world coordinate system from the head position on the image, it is sufficient to use perspective projection transformation, but since the height information of the person is unknown, the height information is different from the foot position. It is not possible to uniquely determine the head position in the world coordinate system. Therefore, the head position is obtained by assuming that the person stands vertically, that is, the head position exists immediately above the foot position.

  This method will be described. First, similarly to the foot position detection means 104, the head position (Xhw, Yhw, Zhw) in the world coordinate system is expressed by the following equation using the unknown variable Zhc.

Here, the head position on the image is (uh, vh). Further, as shown in FIGS. 13A and 13B, the three-dimensional position of the head position in the camera coordinate system is (Xhc, Yhc, Zhc). FIG. 13A is a schematic view of the air-conditioned space viewed from the side, and FIG. 13B is a schematic view of the air-conditioned space viewed from above. In this figure, the point 315 indicates the three-dimensional position of the foot position, and the point 316 indicates the three-dimensional position of the three-dimensional position of the eastern position. By the way, assuming that the person is standing vertically, that is, the head position 316 exists immediately above the foot position 315, the head position (Xhw, Yhw) and the foot position (Xfw, Yfw) on the horizontal plane of the world coordinates. ) Should be equal. Therefore, the foot position (Xhw, Yhw, Zhw) in the world coordinate system can be obtained by obtaining Zhc that minimizes the evaluation function E (Zhc) of [Equation 8].

  Here, the evaluation function E (Zhc) is the square of the distance indicated by 317 in FIG. In order to obtain Zhc that minimizes the evaluation function E (Zhc) of [Equation 8], when [Equation 8] is differentiated by Zhc, the following is obtained.

  From [Equation 10] and [Equation 7], the head position detecting means 105 estimates the head position of the person in the world coordinate system.

  Next, the height estimation means 106 detects the height of the person using the foot position information obtained by the foot position detection means 104 and the head position information obtained by the head position detection means 105 (step S207). ). This method will be described. If it is assumed that the person is standing vertically, that is, the head position is directly above the foot position, the height L of the person is obtained by the following equation.

  Here, ZfW is the foot height in world coordinates obtained by the foot position detecting means 104, and ZhW is the head height in world coordinates obtained by the head position detecting means 105.

  Through the above processing, the height information of the person can be estimated using the frame image captured by the image sensor unit 24. By the way, in order to accurately estimate the height information by the above processing, it is necessary to satisfy the following condition A.

  Condition A: The person stands vertically, that is, the head position is directly above the foot position.

In the outdoors where surveillance cameras are widely used, people are usually moving around, and people who are staying are irregular. On the other hand, in the home, people are often sitting and sleeping, and rarely move around. Therefore, it is not necessary to consider the condition A so much in order to estimate the height with the surveillance camera. However, in the home, selecting and processing only the image satisfying the condition A improves the estimation accuracy. It is very important in making it happen. In the height estimation apparatus of this embodiment, the person movement detection unit 101 detects whether the difference area is a moving person, and the situation recognition processing determination unit 102 estimates the height only when the person is moving. Therefore, high-precision height estimation can be realized.

  Also, in the home, unlike outdoor or public spaces, people often sit or lie down. In this case, since the condition A is not satisfied, the accuracy of the height estimation is greatly deteriorated. On the other hand, in the height estimation device of this embodiment, height estimation is performed only when a person is moving, but when a person is moving, the person is sleeping or sitting down. Almost impossible. For this reason, the height estimation apparatus of the present embodiment does not perform height estimation in a scene where a person is sleeping or sitting, and thus can achieve high-precision height estimation.

  In the above description, instead of using the imaging sensor unit 24 as the person movement detection unit 101, a human sensor or an infrared sensor, which is a known technique, may be used.

  As described above, the height estimation apparatus of the present embodiment estimates person movement information by the person movement detection unit 101, and the situation recognition processing determination unit 102 performs the height estimation unit 106 only when the person is moving. By performing the height estimation process, a highly accurate height estimation process can be realized even in the home.

(Embodiment 2)
FIG. 14 is a functional block diagram showing the configuration of the situation recognition apparatus according to the second embodiment of the present invention, which performs an obstacle detection process as the situation recognition process. In FIG. 14, the same components as those in FIGS. 1 and 2 are denoted by the same reference numerals, and the description thereof is omitted. FIG. 15 is a flowchart showing the flow of obstacle detection processing in the present embodiment. In FIG. 15, the same components as those in FIG. 3 are denoted by the same reference numerals, and the description thereof is omitted.

  The obstacle detection means 107 according to the present embodiment assumes that “there is no obstacle in the foot area of the person imaged by the imaging sensor unit 24”, and the movement history of the person moving around in the home is recorded as the obstacle map. To detect obstacles in the home.

  In FIG. 15, the person movement detection unit 101 uses the image sensor unit 24 to capture an image of the home and obtain a frame image (step S <b> 101). Furthermore, the person movement detecting means 101 detects the presence / absence and position information of the person using the acquired frame image (step S102), and if there is a person, is the person staying further? The movement information such as how it moves is detected (step S103). Based on the detection result of the person movement detection unit 101, the situation recognition process determination unit 102 determines whether to perform an obstacle detection process on the current frame image (step S104).

  That is, when the situation recognition processing determination unit 102 determines that the person is moving (Yes in step S104), the obstacle detection unit 107 updates the obstacle map (step S109) and uses the obstacle map. Then, the obstacle information in the home is acquired (step S110). On the other hand, when the situation recognition processing determination unit 102 determines that the person does not exist or has not moved (No in step S104), the obstacle detection unit 107 does not update the obstacle map. The obstacle information in the home is acquired using the obstacle map (step S110).

  FIG. 16 is a flowchart showing the flow of obstacle map update processing in the obstacle detection means 107.

  First, the obstacle detection means 107 confirms whether an obstacle map already exists (step S301). If no obstacle map exists (No in step S301), an initialized obstacle map is created (step S302).

  On the other hand, if an obstacle map exists (Yes in step S301), the existing obstacle map is used without being initialized. Next, the obstacle detection means 107 confirms whether a short-term obstacle map already exists (step S303). If no short-term obstacle map exists (No in step S303), an initialized short-term obstacle map is created (step S304).

  On the other hand, if a short-term obstacle map exists (Yes in step S303), the existing short-term obstacle map is used without being initialized. Next, the foot position on the image is estimated using the above-described method for the frame image captured by the image sensor unit 24 (step S203). The obstacle detection means 107 votes a fixed value on the short-term obstacle map corresponding to the obtained foot position coordinates (step S305). Next, it is confirmed whether a sufficient number of votes have been cast for the short-term obstacle map (step S306).

  Here, when a sufficient number of votes are performed (Yes in step S306), the obstacle map is updated according to the short-term obstacle map (step S307), and the short-term obstacle map is discarded (step S308). On the other hand, when the sufficient number of votes has not been performed yet (No in step S306), the obstacle detection means 107 does not update the obstacle map, and does not discard the short-term obstacle map and holds it as it is. .

  The obstacle map and the short-term obstacle map will be described. The obstacle map is map information indicating the probability that no obstacle exists in each area by dividing the home into several areas. The short-term obstacle map indicates a probability that there is no obstacle in a shorter period of time compared to the obstacle map, and an obstacle map is created using the short-term obstacle map. 17 and 18 are schematic diagrams for explaining the obstacle map and the short-term obstacle map.

  FIG. 17 is a schematic diagram illustrating a state in the home, and FIG. 18A is a schematic diagram of an image captured by the image sensor unit 24 in the home illustrated in FIG. The obstacle map and the short-term obstacle map are given as the respective areas in FIG. 18B, which are obtained by dividing FIG.

  In this figure, each area indicated by 318 is one element of an obstacle map and a short-term obstacle map. Such area division is realized by, for example, dividing a photographed image into about 80 × 60 areas. The obstacle map and the short-term obstacle map are different map information having the same area.

  Next, the detailed flowchart of FIG. 16 will be described. In step S302, an initialized obstacle map is created. As described above, this is done by creating map information divided into 80 × 60 areas for the captured image and writing 0 in all areas.

  In step S304, a short-term obstacle map is initialized. This is done by writing 0 in each area of the short-term obstacle map as in the obstacle map.

  In step S305, a fixed value is voted for the short-term obstacle map corresponding to the foot position coordinates obtained in step S203. The obstacle detection means 107 according to the present embodiment assumes that “there is no obstacle in the area where the foot of the person is imaged by the imaging sensor unit 24”, and the movement history of the person moving around in the home is shown as an obstacle. By storing it as a map, it detects obstacles in the home.

  That is, by storing the foot position history obtained in step S203 as an obstacle map, the obstacle map is acquired as a probability that no obstacle exists.

  FIG. 19 is a schematic diagram for explaining this process. FIG. 19A is a schematic diagram showing a frame image in which the person 303 exists in the home shown in FIG. Here, the area 319 is a difference area.

  At this time, a certain value is added to the area of the obstacle map corresponding to the foot position detected by the foot position detecting means 104. Of course, a constant value may be added to the area of the short-term obstacle map corresponding to the area included in the lower 20% of the difference area (area 320 in FIG. 19B). Alternatively, a certain value may be added to the area of the obstacle map corresponding to the certain range of the foot position detected in step. This is because there is a high possibility that a person's feet exist in these areas.

  In step S306, the obstacle detection means 107 confirms whether a sufficient number of votes have been cast for the short-term obstacle map. For example, it may be confirmed whether data for 150 frames has been voted.

  If a sufficient number of votes are cast for the short-term obstacle map in step S306 (Yes in step S306), the obstacle map is updated according to the short-term obstacle map in step S307. This is realized as follows.

  Here, Obst_Map_s [i] is the value of the short-term obstacle map in the region i, Obst_Map [i] is the value of the obstacle map in the region i, TH_OBST_S is a threshold value for removing noise in the short-term obstacle map, and Q_OBST Is a value added to the obstacle map. Here, for example, TH_OBST_S = 2 and Q_OBST = 255 may be set.

  FIG. 20A is a schematic diagram showing an example of the short-term obstacle map created in this way. In each region, the value is larger as white is 0 and closer to black. FIG. 20B is a schematic diagram showing a region where the value of Obst_Map_s [i] is larger than TH_OBST_S in black and a region other than that in white according to [Equation 12]. Q_OBST is added to each area of the obstacle map corresponding to the black area, and 1 is subtracted from each area of the obstacle map corresponding to the white area.

  FIG. 21A is a schematic diagram showing an example of an obstacle map created by repeating such processing. In each region, the value is larger as white is 0 and closer to black. Although noise remains on the right sofa and part of the left stairs, it can be seen that the obstacle map can be acquired as a probability that no obstacle exists by accumulating the foot position history.

  Thereafter, in step S308, the short-term obstacle map is discarded. Thus, temporal filtering processing can be performed by using the short-term obstacle map instead of directly operating the obstacle map using the frame image. This is effective for irregular movement such as removal of a difference area other than a person who is a noise, or overlapping of a plurality of persons.

  On the other hand, in step S306, if a sufficient number of short-term obstacle maps have not been voted, the obstacle map and short-term obstacle map update processing is not performed.

  Thus, by voting the foot positions in a plurality of frames, the movement history of the person's foot positions is stored in the short-term obstacle map. The difference between the frame images captured by the image sensor unit 24 means that there is nothing between the image sensor unit 24 and the region where the difference occurs, that is, there is no obstacle. means. Furthermore, by using only the foot position, the presence or absence of an obstacle at the foot position can be stored as an obstacle map.

  Through the above processing, the obstacle detection unit 107 updates the obstacle map and obtains the probability that no obstacle exists as the obstacle map.

  Next, a process in which the obstacle detection unit 107 acquires obstacle information in the home using the obstacle map in step S110 will be described. This can be realized by executing threshold processing on the obstacle map Obst_Map [i] as follows.

  Here, TH_OBST is a threshold value for removing noise in the short-term obstacle map, and for example, TH_OBST = 1 may be set. FIG. 21B is a schematic diagram showing a region where the value of Obst_Map [i] is larger than TH_OBST in black and a region other than that in white according to [Equation 13]. In FIG. 21 (a), noise remained on the right sofa and part of the left staircase and table. However, by executing threshold processing, the noise can be removed and obstacles can be extracted accurately. I understand that.

  However, in order to obtain the foot position of a person by the above method, the following conditions must be satisfied.

  Condition: The estimated foot position is equal to the foot position existing on the floor.

  In a conventional obstacle detection device, for example, when the lower body of a person is shielded, it is difficult to obtain the foot position from an image. For this reason, the estimation of the foot position fails. Also, for example, as shown in FIG. 22, when there is a person 321 who wipes the table from a direction away from the image sensor unit 24, the position of the person's arm is erroneously determined as the foot position, An area where an obstacle exists, called a table, is determined as having no obstacle.

On the other hand, in the obstacle detection apparatus according to the present embodiment, the movement of the person is determined using the person movement detection unit 101, and the obstacle detection unit 107 is used only when the person is moving by the situation recognition processing determination unit 102. Therefore, the obstacle detection process is not performed when the foot of the person is shielded or when the table is covered, so that the obstacle can be detected with high accuracy.

  Specifically, when the lower body of the person is shielded, the above-described condition 1. , Condition 2. And condition 3. Therefore, it is determined that the person is not a moving person. When the table is being wiped, the above-mentioned condition 4. Therefore, it is determined that the person is not a moving person. Therefore, it is possible to detect an obstacle with high accuracy.

  Note that the area division of the obstacle map and the short-term obstacle map may be performed by dividing the size of each area instead of dividing the image equally as shown in FIG. Performing such region division has the following effects. That is, the person is photographed as a large area in the vicinity of the image sensor unit 24, but the person is photographed small in the distance.

  For this reason, voting is performed on a wide area for a nearby person, but only on a relatively small area for a distant person. Since the obstacle map is obtained by thresholding the number of votes for each area, it is considered that an obstacle in the distance is difficult to detect. Therefore, this problem can be solved by dividing the area of the obstacle map or the short-term obstacle map so that the area of the area decreases as the distance from the image sensor unit 24 increases.

  FIG. 23 is a schematic diagram showing the short-term obstacle map and the area division of the obstacle map created as described above. Here, the regions are divided so that the areas in the world coordinate system when all regions are projected onto the floor surface are all equal. This can be realized by using perspective projection transformation.

  In the above description, the obstacle map is initialized in step S301 when there is no obstacle map, that is, only at the start of the process. You can do this. For example, a reset button may be provided in the obstacle detection device, and processing such as discarding the obstacle map may be performed when the user presses the reset button.

  Further, the obstacle map initialization process may be performed using the image processing result. This is because the background difference processing as described above is performed, and when there is a difference in most areas of the screen, it is determined that the pattern has been changed, and processing such as discarding the obstacle map is performed. Good.

  As described above, the obstacle detection apparatus according to the present embodiment estimates the person movement information by the person movement detection unit 101, and the situation recognition process determination unit 102 determines the obstacle detection unit only when the person is moving. By performing the obstacle detection process by 107, a highly accurate obstacle detection process can be realized even in the home.

(Embodiment 3)
FIG. 24 is a functional block diagram showing a configuration of a situation recognition apparatus according to the third embodiment of the present invention that performs camera parameter estimation processing as situation recognition processing. In FIG. 24, the same components as those in FIGS. 1 and 2 are denoted by the same reference numerals, and the description thereof is omitted. FIG. 25 is a flowchart showing the flow of camera parameter estimation processing in the present embodiment. In FIG. 25, the same components as those in FIG. 3 are denoted by the same reference numerals, and the description thereof is omitted.

  As camera parameter estimation means 108 of the present embodiment, it is assumed that the height of a person is constant, and vanishing line information, which is a camera parameter, is estimated from the movement history of a person moving around in the home.

  In FIG. 25, first, the person movement detection unit 101 uses the image sensor unit 24 to capture an image of the home and obtain a frame image (step S101). Furthermore, the person movement detecting means 101 detects the presence / absence and position information of the person using the acquired frame image (step S102), and if there is a person, is the person staying further? The movement information such as how it moves is detected (step S103).

  The situation recognition processing determination unit 102 determines whether to perform camera parameter estimation processing for the current frame image based on the detection result of the person movement detection unit 101 (step S104). That is, when the situation recognition process determining unit 102 determines that the person is moving (Yes in step S104), the foot position detecting unit 104 estimates the foot position of the person (step S105).

  Further, the head position detecting means 105 estimates the head position of the person (step S106). The camera parameter estimation means 108 confirms whether a sufficient number of data has been acquired by the foot position detection means 104 (step S111).

  Here, when a sufficient number of pieces of data are acquired for estimating the camera parameters (Yes in step S111), the camera parameter estimation means 108 determines the foot position information obtained by the foot position detection means 104 and the head position. The vanishing line that is a camera parameter is estimated using the head position information obtained from the part position (step S112).

  On the other hand, if a sufficient number of pieces of data for estimating the camera parameters have not been acquired (No in step S111), the camera parameter estimation unit 108 does not perform the camera parameter estimation process on the current frame image. The previous camera parameter information is held as current camera parameter information (step S113).

  When the situation recognition processing determination unit 102 determines that no person is present or has moved (No in step S104), the camera parameter estimation unit 108 determines whether the current frame image is a camera. The parameter estimation process is not performed, and the previous camera parameter information is held as the current camera parameter information (step S113).

  Next, the camera parameter estimation unit 108 will be described.

  In step S <b> 111, the camera parameter estimation unit 108 confirms whether a sufficient number of pieces of data have been acquired by the foot position detection unit 104. For example, when the foot position and the head position are acquired in 10 or more frame images, it is determined that a sufficient number of data has been acquired. As will be described later, in order to obtain the vanishing line, it is sufficient to have two pieces of data. However, in order to remove noise, the vanishing line is estimated using more data. Of course, instead of using ten pieces of data, two or more pieces of data may be used.

In step S112, the camera parameter estimation unit 108 estimates a vanishing line that is a camera parameter using the foot position information obtained by the foot position detection unit 104 and the head position information obtained from the head position. As described in Patent Document 5, when imaging a pair of objects having the same height and parallel to each other with the imaging sensor unit 24 having a fixed position, straight lines connecting the upper ends of the objects, The straight line connecting the lower ends of the object intersects at one point. This intersection is called the vanishing point. Further, the vanishing points calculated with respect to a set of parallel objects having various heights and the same height are positioned on one straight line. This straight line is called the vanishing line.

  FIG. 26 is a schematic diagram for explaining the vanishing line. Here, it is assumed that the same person is photographed at three places 322, 323, and 324. If a person stands upright on a plane, a vector passing through the foot position and head position at 322, a vector passing through the foot position and head position at 323, and a foot position and head position at 324 All passing vectors are parallel. Further, since the distance between the foot position and the head position at each position is equal to the height, focusing on the fact that they are all equal, a straight line 325 passing through the foot position at 322 and the foot position at 323, and the head position at 322 And an intersection 327 of a straight line 326 passing through the head position at 323 is a vanishing point.

  Similarly, an intersection 330 of a straight line 328 passing through the foot position at 322 and the foot position at 324 and a straight line 329 passing through the head position at 322 and the head position at 324 is also a vanishing point. Therefore, the vanishing line can be obtained by obtaining a straight line 331 passing through the vanishing point 327 and the vanishing point 330.

  As described above, in order to accurately estimate the camera parameters, the following conditions must be satisfied.

  Condition A: The person stands vertically, that is, the head position is directly above the foot position.

  In the outdoors where surveillance cameras are widely used, people are usually moving around, and people who are staying are irregular. On the other hand, in the home, people are often sitting and sleeping, and rarely move around.

  Therefore, in order to perform camera parameter estimation by the surveillance camera, it is not necessary to consider the condition A so much, but selecting an image satisfying the condition A in the home improves the estimation accuracy. Very important. In the camera parameter estimation apparatus according to the present embodiment, the person movement detection unit 101 detects whether the difference area is a moving person, and the situation recognition processing determination unit 102 detects the camera only when the person is moving. Since parameter estimation is performed, highly accurate camera parameter estimation processing can be realized.

  Also, in the home, unlike outdoor or public spaces, people often sit or lie down. In this case, since the condition A is not satisfied, the accuracy of camera parameter estimation is greatly degraded. On the other hand, in the camera parameter estimation apparatus of the present embodiment, camera parameter estimation is performed only when a person is moving, but when the person is moving, the person is sleeping or sitting. There is hardly anything.

  Therefore, the camera parameter estimation apparatus according to the present embodiment can achieve highly accurate camera parameter estimation without performing camera parameter estimation in a scene where a person is sleeping or sitting.

  As described above, the camera parameter estimation apparatus according to the present embodiment estimates the person movement information by the person movement detection unit 101, and the situation recognition processing determination unit 102 performs the camera parameter estimation unit only when the person is moving. By performing the camera parameter estimation process by 108, a highly accurate camera parameter estimation process can be realized even at home.

Of course, instead of estimating the vanishing line as the camera parameter, the direction of the image sensor unit 24 or the like may be estimated.

<Air conditioner equipped with the situation recognition device of the present invention>
The situation recognition apparatus of the present invention is very effective for controlling an air conditioner. An air conditioner equipped with the situation recognition device of the present invention will be described.

  FIG. 27 is a functional block diagram showing a configuration of an air conditioner equipped with the situation recognition device according to the first embodiment of the present invention. In FIG. 27, the same components as those in FIGS. 1 and 2 are denoted by the same reference numerals, and the description thereof is omitted.

  An air conditioner used in a general home is composed of an outdoor unit, an indoor unit, and a remote controller 30 (not shown) that are usually connected to each other through refrigerant pipes. FIGS. 28 to 31 show the air according to the present invention. It shows the indoor unit of a harmony machine.

  The indoor unit has a main body 2 and a movable front panel (hereinafter simply referred to as a front panel) 4 that can freely open and close the front opening 2a of the main body 2, and the front panel 4 is the main body 2 when the air conditioner is stopped. While the front opening 2a is closed in close contact with the front, the front panel 4 moves in a direction away from the main body 2 to open the front opening 2a during operation of the air conditioner. 28 and 29 show a state where the front panel 4 closes the front opening 2a, and FIGS. 30 and 31 show a state where the front panel 4 opens the front opening 2a.

  As shown in FIGS. 28 to 31, inside the main body 2, the heat exchanger 6 and the indoor air taken in from the front opening 2 a and the top opening 2 b are heat-exchanged by the heat exchanger 6, An indoor fan 8 for blowing air, an upper and lower blades 12 for opening and closing a blower outlet 10 for blowing heat-exchanged air into the room and changing the air blowing direction up and down, and left and right blades 14 for changing the air blowing direction left and right Between the front opening 2a and the upper surface opening 2b and the heat exchanger 6 for removing dust contained in the indoor air taken in from the front opening 2a and the upper surface opening 2b. A filter 16 is provided.

  Further, the upper part of the front panel 4 is connected to the upper part of the main body 2 via two arms 18 and 20 provided at both ends thereof, and a drive motor (not shown) connected to the arm 18 is driven and controlled. Thus, during operation of the air conditioner, the front panel 4 moves forward and obliquely upward from the position when the air conditioner is stopped (closed position of the front opening 2a).

  Further, the upper and lower blades 12 are composed of an upper blade 12a and a lower blade 12b, and are respectively attached to the lower portion of the main body 2 so as to be swingable. The upper blade 12a and the lower blade 12b are connected to separate driving sources (for example, stepping motors), and are independently angle-controlled by a control device (for example, a microcomputer) built in the indoor unit. As is clear from FIGS. 30 and 31, the changeable angle range of the lower blade 12b is set larger than the changeable angle range of the upper blade 12a.

  In addition, the upper and lower blades 12 can be composed of three or more upper and lower blades. In this case, at least two (particularly, the uppermost blade and the lowermost blade) can be independently angle-controlled. Is preferred.

  In addition, the left and right blades 14 are configured by a total of 10 blades arranged five by left and right from the center of the indoor unit, and are respectively swingably attached to the lower portion of the main body 2. The left and right five blades are connected to separate drive sources (for example, stepping motors) as a unit, and the left and right five blades are independently angle-controlled by a control device built in the indoor unit. . A method for driving the left and right blades 14 will also be described later.

  As shown in FIG. 28, an imaging sensor unit 24 is attached as an imaging device to the upper part of the front panel 4, and the imaging sensor unit 24 is held by a sensor holder.

  In addition, the situation recognition apparatus of the present invention is realized by a memory (not shown), a microcomputer (not shown), or the like on a circuit board on which an image taken by the imaging sensor unit 24 is mounted.

  The air conditioner performs air conditioning control based on the result estimated by the height estimation device of the present invention. The height estimation apparatus of the present invention performs person identification using the estimated height information, and performs air-conditioning control according to the preset temperature of the air conditioner set by the remote controller 30 in advance. In the air conditioner equipped with the height estimation device of the present invention, it is possible to perform air conditioning in accordance with the temperature previously set by the person using the height information of the person.

  This is because, for example, children with short stature are not directly exposed to the wind from the air conditioner, and air conditioning is performed with a weak air flow. It is possible to perform direct air conditioning and air conditioning with a strong air flow.

  FIG. 32 is a flowchart showing the flow of processing of the air conditioner equipped with the height estimation device according to the first embodiment of the present invention. In FIG. 32, the same components as those in FIG. 3 are denoted by the same reference numerals, and the description thereof is omitted.

  In FIG. 32, the air-conditioning control means 109 uses the height information estimated by the height estimation means 105, and is associated with the height information, which is set in advance as “set temperature of the air conditioner” “air conditioning”. Air conditioning control is realized in accordance with air conditioning control information such as “air direction of the machine” and “air volume of the air conditioner” (step S114). Such air conditioning control information is set in advance using the remote controller 30.

  For example, in the case of a person having a height of 140 cm or less, “air conditioner set temperature = 28 degrees”, “air conditioner wind direction = wind protection”, “air conditioner air volume = weak”, and in the case of height 140 cm or more, “air condition Setting temperature of the machine = 26 degrees, “wind direction of the air conditioner = winding”, “air volume of the air conditioner = strong”, etc. That is, by performing the height estimation process of the present invention, air conditioning control corresponding to a person existing in the air conditioned space can be realized.

  Therefore, the air conditioner equipped with the height estimation device according to the first embodiment of the present invention can perform air conditioning control corresponding to a person existing in the air conditioned space.

  Of course, the air conditioner equipped with the situation recognition apparatus according to the first embodiment of the present invention performs personal identification using height information as described in Patent Document 2, and the identified registrant You may make it control an air conditioner by the setting driving | operation linked | related previously.

  Unlike homes and public spaces, people often sit or lie in homes where air conditioners are often installed. For this reason, in the conventional situation recognition apparatus, since the recognition accuracy cannot be increased, it is impossible to realize an optimal air conditioning operation.

On the other hand, the situation recognition apparatus according to the present embodiment estimates person movement information by the person movement detection unit 101 as described above, and the situation recognition processing determination unit 102 performs the height estimation process only when the person is moving. By performing the above, it is possible to realize high-precision height estimation processing even in the home. Therefore, the situation can be recognized with high accuracy even in the air-conditioned space, and the optimum air-conditioning operation can be realized.

  FIG. 33 is a functional block diagram showing the configuration of the air conditioner equipped with the situation recognition apparatus in the second embodiment of the present invention. In FIG. 33, the same components as those in FIGS. 14 and 27 are denoted by the same reference numerals, and the description thereof is omitted. FIG. 34 is a flowchart showing a process flow of the air conditioner equipped with the obstacle detection apparatus according to the second embodiment of the present invention. In FIG. 34, the same components as those in FIGS. 15 and 32 are denoted by the same reference numerals, and the description thereof is omitted.

  In FIG. 34, the air conditioning control means 109 uses the obstacle information detected by the obstacle detection means 107 to realize air conditioning control that avoids the obstacle (step S114). This obstacle avoidance control will be described.

<Obstacle avoidance control>
Based on the obstacle information, the upper and lower blades 12 and the left and right blades 14 as the wind direction changing means are controlled as follows during heating.

  In the following description, the terms “position”, “block”, “field”, “short distance”, “medium distance”, and “far distance” will be used. These terms will be described first.

  FIGS. 35A and 35B are schematic diagrams showing a human position area that is a difference area detected by the imaging sensor unit 24 that is configured, and a person is present in each of the areas A to G on the image. In which area in the air-conditioned space a person is present. The area where the person exists is obtained by the person movement detecting means 101 in step S102. Further, the floor surface of the living space is subdivided by the obstacle detection means 107 as shown in FIG. 35 (c), and each of these areas is defined as an obstacle position determination area or “position”. Determine if an obstacle exists.

  Here, “position” includes a plurality of regions of the obstacle map. Note that all positions shown in FIG. 35 (c) substantially coincide with all areas of the human position determination area shown in FIG. 35 (b), and the area boundary in FIG. 35 (b) is shown in FIG. 35 (c). By substantially matching the position boundary and corresponding the area and position as follows, the air conditioning control described later can be easily performed, and the memory to be stored is reduced as much as possible.

Area A: Position A1 + A2 + A3, Area B: Position B1 + B2, Area C: Position C1 + C2, Area D: Position D1 + D2, Area E: Position E1 + E2, Area F: Position F1 + F2, Area G: Position G1 + G2
In addition, the regions A to G shown in FIG. 35B belong to the next block, respectively.

Block N: Area A, Block R: Area B, E, Block C: Area C, F, Block L: Area D, G
Regions A to G belong to the following fields, respectively.

Field 1: Area A, Field 2: Areas B and D, Field 3: Area C, Field 4: Areas E and G, Field 5: Area F
Furthermore, the distance from the indoor unit is defined as follows.

  Short distance: Region A, Medium distance: Regions B, C, D, Long distance: Regions E, F, G

  Table 1 shows the target setting angles at the positions of the five left blades and the five right blades constituting the left and right blades 14, and the symbols attached to the numbers (angles) are as shown in FIG. The case where the left or right blade is directed inward is defined as a positive (+, no sign in Table 1) direction, and the case where it is directed outward is defined as a negative (−) direction.

  The “heating B area” in Table 1 is a heating area in which obstacle avoidance control is performed, and “normal automatic wind direction control” is wind direction control in which obstacle avoidance control is not performed. Here, the determination of whether or not to perform the obstacle avoidance control is based on the temperature of the indoor heat exchanger 6. When the temperature is low, the wind direction control is not applied to the occupant, and when the temperature is too high, the maximum air volume position is determined. In the case of wind direction control and moderate temperature, the wind direction control to the heating B area is performed. In addition, “temperature is low”, “too high”, “wind direction control that does not apply wind to the occupant”, and “wind direction control at the maximum airflow position” have the following meanings.

-Low temperature: The temperature of the indoor heat exchanger 6 is set to the skin temperature (33 to 34 ° C) as the optimum temperature, and a temperature that can be lower than this temperature (for example, 32 ° C).
-Too high temperature: for example, 56 ° C or higher-Wind direction control that does not direct wind to the occupant: Wind direction control that causes the wind to flow along the ceiling by controlling the angle of the upper and lower blades 12 so as not to send the wind to the living space -Wind direction control at the maximum airflow position: When the air conditioner bends the airflow with the upper and lower blades 12 and the left and right blades 14, resistance (loss) is always generated, so the maximum airflow position is the wind direction where the loss is close to zero. Control (in the case of the left and right blades 14, it is a position facing directly in front, and in the case of the upper and lower blades 12, it is a position facing downward 35 degrees from the horizontal)

  Table 2 shows target setting angles in the fields of the upper and lower blades 12 when performing obstacle avoidance control. In Table 2, the upper blade angle (γ1) and the lower blade angle (γ2) are angles (elevation angles) measured upward from the vertical line.

  Next, the obstacle avoidance control according to the position of the obstacle will be described in detail. The terms “swing operation”, “position stop operation”, and “block stop operation” used in the obstacle avoidance control will be described first.

  The swinging motion is a swinging motion of the left and right blades 14, and basically swinging with a predetermined left-right angle width around one target position and having no fixed time at both ends of the swing. is there.

In addition, the position stop operation means correction of Table 3 with respect to a target setting angle (an angle in Table 1) of a certain position, which is a left end and a right end, respectively. As the operation, the left end and the right end each have a wind direction fixing time (time for fixing the left and right blades 14). For example, when the wind direction fixing time elapses at the left end, it moves to the right end and the wind direction fixing time elapses at the right end. The wind direction at the right end is maintained, and after the fixed time of the wind direction has passed, it moves to the left end and repeats it. The wind direction fixing time is set to 60 seconds, for example.

  In other words, if there is an obstacle at a certain position and the target setting angle of that position is used as it is, the hot air always hits the obstacle. It is possible to reach the position where there is.

  Further, in the block stop operation, the set angles of the left and right blades 14 corresponding to the left end and the right end of each block are determined based on, for example, Table 4. The operation has a fixed wind direction at the left and right ends of each block.For example, when the fixed wind direction has elapsed at the left end, it moves to the right end and maintains the right wind direction until the fixed wind direction has elapsed at the right end. Then, after the elapse of the wind direction fixing time, it moves to the left end and repeats it. The wind direction fixing time is set to 60 seconds, for example, similarly to the position stop operation. Since the left end and the right end of each block coincide with the left end and the right end of the person position area belonging to the block, the block stop operation can be said to be a stop operation of the person position area.

  In addition, position stop operation and block stop operation are properly used according to the size of the obstacle. When the obstacle in front is small, the position stopping operation is mainly performed at the position where there is an obstacle. When there is an obstacle, the air is blown over a wide range by performing a block stop operation.

  In the present embodiment, the swing operation, the position stop operation, and the block stop operation are collectively referred to as the swing operation of the left and right blades 14.

  Hereinafter, a specific example of control of the upper and lower blades 12 or the left and right blades 14 will be described. When the person movement detection unit 101 determines that a person is only in a single area, the person movement detection unit 101 has a person. When the obstacle detection means 107 determines that there is an obstacle in front of the difference area determined to be, the upper and lower blades 12 are controlled to perform airflow control that avoids the obstacle from above.

Further, when the obstacle detection unit 107 determines that there is an obstacle in the position belonging to the person position area determined by the person movement detection unit 101, the person movement detection unit 101 belongs to the person position area determined to have a person. First air flow control in which the left and right blades 14 are swung at at least one position and the left and right blades 14 are not fixed at both ends of the swing range, and a person position area determined to have a person or adjacent to the area The left and right blades 14 are swung at at least one position belonging to the person position area, and one of the second airflow controls in which the left and right blades 14 are fixed at both ends of the swing range is selected.

  In the following description, the control of the upper and lower blades 12 and the control of the left and right blades 14 are separated, but the control of the upper and lower blades 12 and the control of the left and right blades 14 are appropriately combined depending on the position of the person and the obstacle. Is called.

A. Upper and lower blade control (1) When there is a person in any of the regions B to G and there are obstacles at positions A1 to A3 in front of the region where the person is present, the set angle of the upper and lower blades 12 is set to normal field wind direction control (Table 2). ) Is corrected as shown in Table 5, and airflow control is performed with the upper and lower blades 12 set upward.

(2) When there is a person in any of the areas B to G and there is no obstacle in the area A in front of the area in which the person is present (other than (1) above)
Usually performs automatic wind direction control.

B. Left and right blade control B1. When there is a person in the area A (short distance) (1) When there is one position without an obstacle in the area A, the first airflow control is performed by swinging left and right around the target setting angle of the position without the obstacle. . For example, when there are obstacles at positions A1 and A3 and there are no obstacles at position A2, the position A2 is basically air-conditioned by swinging left and right around the target setting angle of position A2. Since there is no guarantee that there are no people at positions A1 and A3, an air flow is distributed to positions A1 and A3 by adding a swing motion.

  More specifically, since the target setting angle and the correction angle (swing angle during the swing operation) of the position A2 are determined based on Tables 1 and 3, both the left blade and the right blade have 10 degrees. It continues to swing (swing) without stopping in the center at an angle range of ± 10 degrees. However, the timing of swinging the left and right blades to the left and right is set to be the same, and the swinging motions of the left and right blades are linked.

(2) When there are two obstacle-free positions in the area A and they are adjacent (A1 and A2 or A2 and A3)
The first airflow control is performed by swinging the target setting angles of the two positions with no obstacles at both ends to basically air-condition the positions without the obstacles.

(3) When there are two obstacle-free positions in area A (A1 and A3
)
The second airflow control is performed by operating the block stop with the target set angles of two positions having no obstacles at both ends.

(4) When there are obstacles at all positions in the region A Since it is unclear where to aim, the block N is operated in a block stop and the second airflow control is performed. This is because the block stop operation is more directional and can reach far away than the entire area, and there is a high possibility of avoiding obstacles. That is, even when obstacles are scattered in the area A, there is usually a gap between the obstacles, and the air can be blown through the gap between the obstacles.

(5) When there are no obstacles in all positions in the area A The normal automatic wind direction control in the area A is performed.

B2. When there is a person in any of the areas B, C, D (medium distance) (1) When there is an obstacle only in one of the two positions belonging to the person's area The first airflow control is performed by swinging left and right. For example, when there is a person in the region D and there is an obstacle only at the position D2, the swing operation is performed to the left and right around the target setting angle of the position D1.

(2) When there are obstacles in both of the two positions belonging to the area where the person is present The block including the area where the person is present is operated to stop the block and the second air flow control is performed. For example, when there is a person in the area D and there are obstacles in both the positions D1 and D2, the block L is operated while being stopped.

(3) When there are no obstacles in an area where people are present Normal normal wind direction control is performed in areas where people are present.

B3. When there is a person in any of the areas E, F, G (far distance) (1) When there is an obstacle only in one of the two positions belonging to the middle distance area in front of the area where the person is present (example: area E There is an obstacle at position B2 and there is no obstacle at position B1)
(1.1) When there are no obstacles on both sides of a position with obstacles (eg, there are no obstacles at positions B1 and C1)
(1.1.1) When there is no obstacle behind the position where there is an obstacle (eg, there is no obstacle at position E2)
The second airflow control is performed by stopping the position around the position where the obstacle is located. For example, if there is a person in the area E and there is an obstacle at the position B2, and there are no obstacles on either side of the obstacle, the air current can be sent to the area E while avoiding the obstacle at the position B2 from the side. .

(1.1.2) When there is an obstacle behind the position where there is an obstacle (eg, there is an obstacle at position E2)
The first airflow control is performed by performing a swing operation around the target setting angle in a position where there is no obstacle in the middle distance region. For example, if there is a person in the area E and there is an obstacle at the position B2 and there are no obstacles on both sides, but there are obstacles behind it, it is advantageous to send airflow from the position B1 where there is no obstacle. .

(1.2) When there is an obstacle on both sides of the position where there is an obstacle and there is no obstacle on the other side, the first airflow control is performed by swinging around the target setting angle of the position where there is no obstacle . For example, if there is a person in the area F, there is an obstacle in position C2, there is an obstacle in position D1 of both sides of position C2, and there is no obstacle in C1, the obstacles from position C1 to position C2 where there is no obstacle Airflow can be sent to area F while avoiding objects.

(2) When there is an obstacle in both of the two positions belonging to the middle distance area in front of the area where the person is present The block including the area where the person is present is operated to stop the block and the second airflow control is performed. For example, when there is a person in the area F and there are obstacles in both the positions C1 and C2, the block C is operated in a block stop state. In this case, since there is an obstacle ahead of the person and there is no way to avoid the obstacle, the block stop operation is performed regardless of whether there is an obstacle in the block adjacent to the block C.

(3) When there are no obstacles in both of the two positions belonging to the middle distance area in front of the area where the person is present (example: there is a person in the area F and there are no obstacles in the positions C1 and C2)
(3.1) When there is an obstacle only in one of the two positions belonging to the area where the person is present The first airflow control is performed by swinging around the target set angle of the other position where there is no obstacle. For example, if there is a person in the area F, there are no obstacles in the positions C1, C2, and F1, and there is an obstacle in the position F2, the front of the area F in which the person is present is open. Considering this, air conditioning is performed around the far-off position F1 without an obstacle.

(3.2) When there are obstacles in both of the two positions belonging to the area where the person is present The block including the area where the person is present is operated in a block stop to perform the second air flow control. For example, if there is a person in the area G, there are no obstacles in the positions D1 and D2, and there are obstacles in both the positions G1 and G2, the front of the area G in which the person is present is open. Since there is an obstacle and it is unclear where to aim, block L is put into block stop operation.

(3.3) When there are no obstacles in both of the two positions belonging to the area where the person is present Normal normal wind direction control is performed in the area where the person is present.

  Unlike homes and public spaces, people often sit or lie in homes where air conditioners are often installed. For this reason, in the conventional situation recognition apparatus, since the recognition accuracy cannot be increased, it is impossible to realize an optimal air conditioning operation. On the other hand, as described above, the situation recognition apparatus according to the present embodiment estimates person movement information by the person movement detection unit 101, and the situation recognition processing determination unit 102 detects an obstacle only when the person is moving. By performing processing, it is possible to realize highly accurate obstacle detection processing even in the home, so it is possible to recognize the status of the air-conditioned space with high accuracy and realize energy-saving operation while realizing a comfortable air-conditioned space The optimum air conditioning operation can be realized.

  Further, even in an air conditioner equipped with the camera parameter estimation device according to the third embodiment of the present invention, camera parameter estimation at home, which has been difficult in the past, can be performed with high accuracy. Accuracy can be improved and optimal air conditioning operation can be realized.

  In addition, although the air conditioner which performs air-conditioning control was demonstrated based on the result recognized with the condition recognition apparatus of this invention, it is made to have the recognition result display means 110 for displaying an recognition result. It doesn't matter.

FIG. 37 is a functional block diagram showing the configuration of an air conditioner equipped with the situation recognition device of the present invention. In FIG. 37, the same components as those in FIGS. 1 and 27 are denoted by the same reference numerals, and description thereof is omitted.

  FIG. 38 is a flowchart showing the flow of processing of the air conditioner equipped with the height estimation device according to the first embodiment of the present invention. In FIG. 38, the same components as those in FIGS. 3 and 32 are denoted by the same reference numerals, and the description thereof is omitted.

  In FIG. 38, the recognition result display means 110 displays the estimation result of the height estimation means 105 to a person in the air-conditioned space (step S115). As a display method, for example, when it is determined that the person is moving by the situation recognition process determination unit 102 and the height estimation process is performed, a light source such as an LED installed in the indoor unit of the air conditioner is used for a certain period of time. It can be flashed or lit. Of course, when the height estimation process is performed, a light source such as an LED installed in the indoor unit of the air conditioner is turned on for a certain period of time, while when the height estimation process is not performed, the air conditioner A light source such as an LED installed in the indoor unit may be blinked for a certain period of time.

  Thus, by displaying the situation identification result to a person in the home, the user can recognize whether or not the situation identification apparatus has performed the identification, so that the user can use it with peace of mind. In addition, by using a light source such as an LED installed in an indoor unit of an air conditioner, it is possible to recognize from various places in the home, and the height estimation device without disturbing the user Status can be displayed.

  Of course, as a display method, instead of using a light source such as an LED, for example, a sound may be emitted from a speaker installed in an indoor unit of an air conditioner. By producing sound from a speaker installed in an indoor unit of an air conditioner, the status of the height estimation device can be recognized from various places in the home.

  Of course, as a display method, the movable front panel 4, the upper and lower blades 12, and the left and right blades 14, which are operating parts of the air conditioner, may be operated. By doing in this way, the situation of the height estimating device can be displayed without getting in the way of the user.

  As described above, the situation recognition apparatus according to the present invention can practically and accurately realize situation recognition processing even in the home. Furthermore, the air conditioner equipped with the situation recognition device according to the present invention can recognize the situation in the air-conditioned space with high accuracy even in the home, and the control accuracy can be improved. It can be widely applied to the use of all devices that give operation instructions.

DESCRIPTION OF SYMBOLS 2 Indoor unit main body 2a Front opening part 2b Upper surface opening part 4 Movable front panel 6 Heat exchanger 8 Indoor fan 10 Outlet 12 Upper and lower blades 14 Left and right blades 16 Filter 18, 20 Front panel arm 24 Image sensor unit 30 Remote control 101 Person movement Detection means 102 Situation recognition processing determination means 103 Situation recognition means 104 Foot position detection means 105 Head position detection means 106 Height estimation means 107 Obstacle detection means 108 Camera parameter estimation means 109 Air conditioning control means 110 Recognition result display means

Claims (16)

A situation recognition device for recognizing a situation in the home,
A person movement detecting means for detecting movement information of the person;
A situation recognition process judging means for judging whether or not to implement a situation recognition process based on a detection result of the person movement detection means;
A situation recognition apparatus comprising situation recognition means for recognizing a situation in the home when the situation recognition process judgment means judges that the situation recognition process is to be performed. The situation recognition process determination means includes
The situation recognition apparatus according to claim 1, wherein when the person is moving, the situation recognition process is performed by the person movement detection unit, and otherwise, the situation recognition process is not performed. The situation recognition processing determination means performs height estimation when the person is moving in a direction orthogonal to a vector connecting the head and the foot by the person movement detection means, and moves in the vector direction. The situation recognition apparatus according to claim 2, wherein the situation recognition processing is not performed when the situation is detected. The situation recognition means includes
The situation recognition apparatus according to any one of claims 1 to 3, wherein the situation recognition unit is a height estimation unit that detects the height of a person. The height estimation means which is the situation recognition means,
A foot position detecting means for detecting a foot position of the person;
A head position detecting means for detecting a head position of the person;
5. The height estimating means for detecting the height of a person using the foot position information obtained by the foot position detecting means and the head position information obtained by the head position detecting means. Situation recognition device. The situation recognition means includes
The situation recognition apparatus according to any one of claims 1 to 3, wherein the situation recognition unit is an obstacle detection unit that detects an obstacle in a home. The obstacle detection means which is the situation recognition means,
The situation recognition apparatus according to claim 6, wherein an obstacle is detected using a movement history of a person. The obstacle detection means which is the situation recognition means,
The situation recognition apparatus according to claim 7, wherein an obstacle is detected using a history of a person's foot position. The situation recognition means includes
The situation recognition apparatus according to claim 1, wherein the situation recognition apparatus is a camera parameter estimation unit that performs a camera parameter estimation process. The camera parameter estimation means in the situation recognition means,
The situation recognition apparatus according to claim 9, wherein a vanishing line that is a camera parameter is estimated. An air conditioner equipped with the situation recognition device according to any one of claims 1 to 3,
An air conditioner comprising an air conditioning control unit that realizes air conditioning control based on a situation recognition result identified by the situation recognition means. An air conditioner equipped with the situation recognition device according to claim 11,
The air conditioner further comprises recognition result display means for displaying a recognition result recognized by the situation recognition means. A situation recognition method for recognizing the situation in the home,
A person movement detection step for detecting movement information of the person;
A situation recognition process determination step for determining whether or not to implement a situation recognition process based on the detection result of the person movement detection step;
A situation recognition method comprising a situation recognition step of recognizing a situation in a home when it is judged in the situation recognition process judgment step that the situation recognition process is performed. The situation recognition step includes
The situation recognition method according to claim 13, which is a height estimation step of detecting the height of a person. The situation recognition step includes
The situation recognition method according to claim 13, which is an obstacle detection step of detecting an obstacle in the home. The situation recognition step includes
The situation recognition method according to claim 13, wherein the camera parameter estimation step performs camera parameter estimation processing.