JP2016171526A - Image sensor, person detection method, control system, control method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image sensor, a person detection method, a control system, a control method, and a computer program capable of accurately detecting a person from an image obtained by imaging a wide range.SOLUTION: The image sensor comprises an imaging part 12, a dictionary information storage part 13, and a person information acquisition part 14. The imaging part 12 performs wide angle imaging of a target space of person detection. The dictionary information storage part 13 stores plural pieces of dictionary information indicating the characteristic of a person. The person information acquisition part 14 detects the person from an image on the basis of an image imaged by the imaging part 12 and the pieces of dictionary information.SELECTED DRAWING: Figure 3

Description

  Embodiments described herein relate generally to an image sensor, a human detection method, a control system, a control method, and a computer program.

One common method for detecting a moving object from an image is an inter-frame difference method for detecting a moving object based on a change in luminance of the image. The inter-frame difference method is a method of acquiring a difference between images acquired continuously and detecting a subject in a region where the change has occurred as a moving object. A technique for detecting a person based on the result of such moving object detection has been proposed.
However, when a camera is installed in an office or the like, the luminance of an image may change due to changes in the brightness of the space caused by external light or illumination as well as human movement. Therefore, there is a possibility that a person is erroneously detected even though no person actually exists. In order to solve this problem, a technique has been proposed in which representative behavior and behavior of a person are learned and whether or not a moving object is a person is identified.

  However, such a conventional technique is based on the premise that a person is imaged with a sufficient size. Under such a premise, many cameras are required depending on the size of the space to be detected. On the other hand, there are cameras with fish-eye lenses and super-wide-angle lenses that can capture a wide range of images. Images captured over a wide range vary greatly in how the subject is captured depending on lens distortion and the distance from the subject. For this reason, it has been difficult to accurately detect a person from an image in which a wide range has been captured.

Japanese Patent No. 4432181 Japanese Patent No. 4852159 Japanese Patent No. 5349622

  The problem to be solved by the present invention is to provide an image sensor, a person detection method, a control system, a control method, and a computer program that can accurately detect a person from an image captured over a wide range.

  The image sensor according to the embodiment includes an imaging unit, a storage unit, and a detection unit. An imaging part images the space of the object which detects a person. The storage unit stores a plurality of dictionary information indicating human characteristics. The detection unit detects a person from the image based on the image captured by the imaging unit and the plurality of dictionary information.

The figure which shows the usage example of the image sensor 1 of embodiment. 1 is a system configuration diagram showing a configuration of a device control system using an image sensor 1 of an embodiment. The functional block diagram which shows the function structure of the image sensor 1 of embodiment. The flowchart which shows the flow of the process in which the image sensor 1 acquires person information. The figure which shows the specific example of the image which the image sensor 1 acquires. The figure which shows the specific example of the image which the image sensor 1 acquires. The figure which shows the specific example of the image divided | segmented according to distortion of a lens. FIG. 3 is a functional block diagram showing a functional configuration of the device control apparatus 5. The figure which shows the specific example of the method the control information generation part 56 produces | generates control information. The functional block diagram which shows the function structure of the terminal 7 for adjustment.

  Hereinafter, an image sensor, a human detection method, a control system, a control method, and a computer program according to embodiments will be described with reference to the drawings.

(Embodiment)
FIG. 1 is a diagram illustrating a usage example of the image sensor 1 of the embodiment.
FIG. 1 shows an example in which the image sensor 1 is used to control equipment on an office floor. In the case of the example in FIG. 1, the lighting device 20 and the air conditioning device 30 are devices to be controlled. The image sensor 1 is installed on the ceiling 10 of the office floor and images the office floor. The image sensor 1 acquires environment information and person information from the captured image. The environment information is information regarding the environment of the imaging target space (hereinafter referred to as “target space”). For example, the environmental information is information indicating the illuminance or temperature of the office. The person information is information about a person in the target space. For example, the person information is information indicating the presence or absence of a person, the number of people, the behavior of the person, the amount of activity of the person, and the like. In order to acquire person information, the image sensor 1 analyzes the acquired image and detects a person. The image sensor 1 transmits the acquired environmental information and person information to a control device that controls the lighting device 20 and the air conditioning device 30. The control device generates control information for controlling the lighting device 20 and the air conditioning device 30 based on the environmental information and the person information acquired by the image sensor 1. The control device transmits the generated control information to the lighting device 20 and the air conditioning device 30. The lighting device 20 and the air conditioning device 30 operate the own device based on the control information transmitted from the control device. By such control, device control based on environmental information and personal information is realized.

  For example, when information indicating the presence or absence of a person is acquired as person information, the control device turns on air conditioning and lighting in an area where there is a person, and turns off air conditioning and lighting in an area where there is no person, or Air conditioning and lighting can be controlled to reduce output. In addition, when there is a person, control that turns on task lighting is also conceivable. In addition, control such as changing the dimming rate of the ambient lighting according to a place where a person is detected, such as a desk or a passage, can be considered.

  Also, for example, when information indicating the amount of activity of a person is acquired as personal information, the control device determines that there are many people who are doing desk work when the amount of activity is small, and reduces the output of air conditioning. When the amount is large, it can be determined that the person is actively active and the air conditioning can be controlled to increase the output of the air conditioning. Further, for example, when information indicating a person distribution is acquired as the person information, the lighting can be controlled so as to change the dimming rate of the lighting in the floor according to the person distribution.

  In recent years, efforts have been made to save energy of devices using sensing results of various sensors including the above-described image sensors. Among them, the image sensor can acquire a wide variety of information as compared with the human sensor and the light sensor. Furthermore, when a fish-eye lens, a super-wide-angle lens, or the like is provided, a single image sensor can acquire a wide range of information. As described above, if information about a person can be acquired with high accuracy from an image captured over a wide range, fine control that achieves both energy saving and comfort is possible. The image sensor 1 according to the embodiment acquires information about a person with higher accuracy by learning representative actions and behaviors of the person.

FIG. 2 is a system configuration diagram illustrating a configuration of a device control system using the image sensor 1 of the embodiment.
The device control system of FIG. 2 includes a plurality of image sensors 1-1 to 1-3. The image sensors 1-1 to 1-3 transmit personal information and environment information (hereinafter referred to as “detection information”) acquired from the images to the gateway server 3 via the network hub 2. The gateway server 3 transmits the person information and environment information transmitted from each image sensor 1 to a BEMS (Building Energy Management System) 4, a device control device 5, a visualization device 6, an adjustment terminal 7, and the like.

  The BEMS 4 acquires information indicating the energy used in the building and the indoor environment from a sensor or the like, and performs analysis and diagnosis to make use of this information for energy saving.

  The device control device 5 generates control information for optimally controlling the device to be controlled based on the detection information. The device control apparatus 5 controls the device to be controlled by transmitting the generated control information. A device to be controlled by the device control device 5 is a device such as an air conditioner, a lighting device, and an elevator, for example.

  The visualization device 6 is a device that visualizes information of the device control system. For example, the visualization device 6 obtains the results of analysis and diagnosis by the BEMS 4 and visualizes the energy used in the building, the indoor environment, and the like.

  The adjustment terminal 7 is a terminal that communicates with the image sensor 1 and in which an adjustment program for performing various settings and adjustments of the image sensor 1 is installed. For example, the adjustment terminal 7 is an information terminal such as a PC (Personal Computer), a smartphone, or a tablet. The adjustment program may be installed in a device other than the adjustment terminal 7 as long as it can communicate with the image sensor 1.

  The network hub 2 and the adjustment terminal 7 are connected to the device network 40, and the BEMS 4, the device control device 5, and the visualization device 6 are connected to the control network 50. For example, the control network 50 is a network such as Ethernet (registered trademark), and the control network 50 is a BACnetIP network. Communication between the device connected to the device network 40 and the device connected to the control network 50 is relayed by protocol conversion of the gateway server 3.

  The device network 40 and the control network 50 may be different networks as long as communication between the networks is possible. For example, the device network 40 and the control network 50 may be a network such as industrial Ethernet (registered trademark) standardized for a specific application, or may be a network of a unique protocol. The device network 40 and the control network 50 may be the same network.

FIG. 3 is a functional block diagram illustrating a functional configuration of the image sensor 1 according to the embodiment.
The image sensor 1 includes a CPU (Central Processing Unit), a memory, an auxiliary storage device, and the like connected by a bus, and executes an image sensor program. The image sensor 1 functions as an apparatus including the communication unit 11, the imaging unit 12, the dictionary information storage unit 13, and the person information acquisition unit 14 by executing the image sensor program. Note that all or part of the functions of the image sensor 1 may be realized using hardware such as an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA). The image sensor program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in the computer system. The image sensor program may be transmitted via a telecommunication line.

  The communication unit 11 is configured using a communication interface for connecting the own device to the network hub 2. The communication unit 11 communicates with various devices connected to the device network 40 and the control network 50 via the network hub 2.

  The imaging unit 12 is configured using an imaging device such as a camera. The imaging unit 12 is installed at a position and orientation at which detection information about the device to be controlled can be acquired. The imaging unit 12 captures a space corresponding to the installation position and orientation, and acquires image data. The imaging unit 12 outputs the acquired image data to the person information acquisition unit 14.

  The dictionary information storage unit 13 stores a plurality of dictionary information used for acquiring personal information. The dictionary information is information indicating correspondence between the feature amount extracted from the image and various features of the image. The dictionary information is generated based on a pre-captured image (hereinafter referred to as “pre-image”) and stored in the dictionary information storage unit 13 in advance. Hereinafter, this generation and storage of dictionary information is referred to as learning.

  The person information acquisition unit 14 (detection unit and person information acquisition unit) acquires image data from the imaging unit 12. The person information acquisition unit 14 acquires person information from the acquired image data based on various dictionary information. The person information acquisition unit 14 indicates, for example, a person's action such as “walking”, “speaking”, and “PC work”, and a person's posture such as “face orientation”, “standing position”, and “sitting position”. Information is acquired as person information. The person information acquisition unit 14 transmits the acquired person information to the BEMS 4, the device control device 5, and the visualization device 6.

FIG. 4 is a flowchart showing a flow of processing in which the image sensor 1 acquires person information.
First, the imaging unit 12 captures an image of a space where a person is to be detected, and acquires image data (step S101). The imaging unit 12 outputs the acquired image data to the person information acquisition unit 14.

  The person information acquisition unit 14 acquires image data from the imaging unit 12. The person information acquisition unit 14 detects a moving object from the image based on the acquired image data. For example, the person information acquisition unit 14 detects a moving object from an image by an interframe difference method. The person information acquisition unit 14 extracts the feature amount of the image of the detected moving object region (step S102). The feature amount extracted here is a feature amount related to a luminance change such as a cumulative difference described in Patent Document 2 or a feature related to a luminance distribution such as CoHOG (Co-occurrence Histograms of Oriented Gradients) described in Patent Document 3. An amount may be used.

  The person information acquisition unit 14 acquires dictionary information from the dictionary information storage unit 13. The person information acquisition unit 14 determines whether or not the moving object is a person based on the feature amount of the moving object region and the dictionary information, and identifies the detected movement and posture of the person (step S103). Neural networks, SVM (Support Vector Machine), k-neighbor discriminators, Bayesian classification, or the like may be used as a method for identifying a person's movement or posture. The person information acquisition unit 14 transmits the determination result as person information to the BEMS 4, the device control apparatus 5, and the visualization apparatus 6 (step S104).

  Note that the dictionary information used for the above identification is generated in advance based on a plurality of images in which an event to be identified is captured. The image used for generating the dictionary information may be an image acquired by the image sensor 1 or an image captured by another imaging device. Further, the image used for generating the dictionary information may be an image captured at the installation position of the image sensor 1 or an image captured at another location.

  Further, the detection performance of the image sensor 1 depends on the quality of dictionary information. And in order to improve the quality of dictionary information, a large amount of prior images are required. However, the work of acquiring a large amount of preliminary images is a work that requires a great deal of labor. Therefore, instead of acquiring a new preliminary image by imaging, a new preliminary image may be generated in a pseudo manner by processing a previously acquired preliminary image. The dictionary information may be information generated based on the image generated in a pseudo manner in this way. By generating the dictionary information by such a method, the quality of the dictionary information can be improved with little effort. The pre-image processing is, for example, operations such as image rotation, resolution change, extraction, and extraction position change.

  In addition, the person information acquisition unit 14 may use a plurality of dictionary information according to the operation and posture to be identified. In addition, the person information acquisition unit 14 may change the type and combination of dictionary information used for identification according to the operation and posture to be identified. Moreover, each identification process based on several dictionary information may be processed in parallel, and may be processed in series. The feature amount extraction and identification processing may be performed in units of pixels, or may be performed in units of regions extracted from the image. In this case, feature amount extraction and identification processing may be performed in units of regions including the extracted regions, such as circumscribed rectangles. Different processing units may be used according to a plurality of dictionary information.

5 and 6 are diagrams illustrating specific examples of images acquired by the image sensor 1.
The subject in the image is imaged with distortion due to the characteristics of the lens. Image distortion appears more prominently in an image captured using a lens capable of capturing a wide range. For example, FIG. 5 is an example of an image captured using a fisheye lens. The image in FIG. 5 is an image obtained by capturing an image of the room as the camera installed on the ceiling of the room looks down on the floor. When an image is captured using a fisheye lens, the image of the imaging region is circular as shown in FIG. From the image of FIG. 5, it can be seen that the distortion near the center of the image is small, and the distortion of the image increases as the distance from the center of the image increases.

  Also, the subject is imaged with different sizes depending on the distance from the camera. A subject closer to the camera is imaged larger, and a subject farther from the camera is imaged smaller. Further, as in the example of FIG. 6, even in the same subject, the way the image is reflected varies greatly depending on the position at the time of imaging. For example, an image 100-1 in FIG. 6 is an image obtained by capturing a person from the front. An image 100-2 is an image obtained by imaging a person from directly above. The image 100-3 is an image obtained by capturing an image of a human side from an obliquely upper side. An image 100-4 is an image obtained by capturing the front of a person from an oblique front. The image 100-5 is an image obtained by imaging the back of a person from obliquely behind.

  The dictionary information storage unit 13 stores a plurality of dictionary information in order to identify such differences in image distortion, size, and appearance. The feature information acquired from the prior image is associated with various features corresponding to these differences, so that the person information acquisition unit 14 identifies these differences regarding the detected person based on a plurality of dictionary information. can do.

  Hereinafter, a specific example in which person information is acquired based on a plurality of dictionary information will be described.

[First specific example]
The first specific example is a method of dividing an image into a shape corresponding to the distortion of the lens and acquiring person information based on dictionary information for each divided area. For example, in the case of the image shown in FIG. 5, the image is divided as shown in FIG.

FIG. 7 is a diagram illustrating a specific example of an image divided according to lens distortion.
In the case of an image captured using a fisheye lens, the image is divided into regions according to the distance from the center of the imaging region. In the case of the example in FIG. 7, the image is divided into divided areas 201, 202, and 203, and dictionary information for each divided area is stored in the dictionary information storage unit 13. Here, the dictionary information corresponding to the divided areas 201, 202, and 203 is described as dictionary information 1, dictionary information 2, and dictionary information 3.

  The dictionary information 1 is generated based on a person's prior image captured in the same manner as the divided area 201. By using the dictionary information 1 generated in this way, the person information acquisition unit 14 acquires person information from an image in which a person's head is large and the whole body is hardly reflected, as in the example of FIG. be able to.

  The dictionary information 2 is generated based on a person's prior image captured in the same manner as the divided area 202. By using the dictionary information 2 generated in this way, the person information acquisition unit 14 can acquire person information from an image in which the whole body of a person is captured obliquely from above, as in the example of FIG.

  The dictionary information 3 is generated based on a person's previous image captured in the same manner as the divided area 203. By using the dictionary information 3 generated in this way, the person information acquisition unit 14 can acquire person information from an image in which the front of a person far from the camera is captured as in the example of FIG. .

  Note that, as described above, the farther away from the camera, the smaller the image is captured due to distortion of the lens, so the divided areas may be arbitrarily set according to the characteristics of the lens. When a person is detected near the boundary between the divided areas, the person information acquisition unit 14 may acquire person information using dictionary information corresponding to two adjacent divided areas. In this case, the person information acquired based on one of the dictionary information based on a predetermined priority order may be adopted, or acquired based on both dictionary information based on a predetermined weight. You may employ | adopt combining personal information. The priorities and weights described above may be determined according to, for example, the area ratio of the area where the person is detected. In this case, the area of the region where the person is detected may be corrected in consideration of image distortion, and the corrected area ratio may be used.

[Second specific example]
A problem in acquiring person information from an image is that a part of a person is captured while being hidden by another object. In such a case, there is a possibility that the person information cannot be obtained with high accuracy only by using dictionary information (hereinafter referred to as “whole body dictionary”) generated based on an image of the whole body of a person. Therefore, in the second specific example, the person information is acquired by combining the whole body dictionary and the dictionary information generated according to the body part.

  For example, there is a high possibility that a person's whole body is imaged in a space with good visibility such as an elevator hall, a hallway, and a meeting space with few desks. On the other hand, in an office or the like, there is a case where an image is hidden behind an office device such as a desk, a locker, a chair, or a monitor. Therefore, in such a case, for example, dictionary information generated based on an image in which the upper body of a person is captured in a whole body dictionary (hereinafter referred to as “upper body dictionary”) or an image in which the head of a person is captured. The person information can be obtained with high accuracy by combining the dictionary information generated according to the body part, such as the dictionary information generated based on (hereinafter referred to as “head dictionary”).

  In this case, the dictionary information is generated by associating a region corresponding to each part of the body (whole body, upper body, or head) with a plurality of images obtained by imaging events to be identified such as human movements and postures. . The association between the image and the body part may be performed manually or automatically. When the association is performed automatically, for example, the upper body and the head may be automatically extracted from the region corresponding to the whole body by setting the lens center in the imaging region and the region corresponding to the whole body. .

  In this way, when combining a plurality of dictionary information generated according to a body part, which person is based on the priority order set for each part, the similarity of person information acquired by each dictionary information, etc. It may be determined whether to adopt information. Similarly to the first specific example, the personal information to be finally adopted may be generated by combining the personal information based on the weight of the dictionary information.

  In addition, when acquiring person information based on a plurality of dictionary information, processing related to each dictionary information may be executed in parallel, and finally the person information to be finally adopted may be comprehensively determined based on each result. Then, the processing related to each dictionary information may be executed in series to obtain more detailed person information. For example, after a person is detected based on the whole body dictionary, actions such as “standing position”, “walking”, and “running” may be identified. Further, for example, “face orientation” may be identified after a human head is detected based on a head dictionary. By appropriately setting combinations, priorities, weights, etc. of a plurality of dictionaries, it is possible to reduce the processing amount and improve the performance.

  The image sensor 1 according to the embodiment configured as described above can detect a person with high accuracy from an image in which a wide range is captured by using a plurality of dictionary information indicating characteristics of the person for detecting the person from the image. Can do. In addition, in the detection of a person, by using a plurality of dictionary information indicating a person's state such as an action and a posture, person information indicating the detected person's state can be acquired. Further, in the detection of a person, by using a plurality of dictionary information corresponding to different reflection methods depending on the position of the person, it is possible to accurately detect a person from an image in which a wide range is imaged with distortion.

  Hereinafter, a specific example of a device control system using the image sensor 1 will be described.

  For example, the device control system includes an image sensor 1, a device control device 5, and an adjustment terminal 7.

FIG. 8 is a functional block diagram showing a functional configuration of the device control device 5.
The device control device 5 includes a CPU, a memory, an auxiliary storage device, and the like connected by a bus, and executes a device control program. The device control device 5 functions as a device including a communication unit 51, a person information acquisition unit 52, an illuminance information acquisition unit 53, a floor information acquisition unit 54, a storage unit 55, and a control information generation unit 56 by executing the device control program. Note that all or part of the functions of the device control device 5 may be realized using hardware such as an ASIC, PLD, or FPGA. The device control program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in the computer system. The device control program may be transmitted via a telecommunication line.

  The communication unit 51 is configured using a communication interface for connecting the own apparatus to the control network 50. The communication unit 51 communicates with various devices connected to the device network 40 and the control network 50 via the network hub 2 or the gateway server 3.

  The person information acquisition unit 52 acquires person information from the image sensor 1. The person information acquisition unit 31 stores the acquired person information in the storage unit 55. The person information may be acquired from another device that acquires the person information from the image sensor 1. For example, the person information acquisition unit 52 may acquire person information from the BEMS 4.

The illuminance information acquisition unit 53 (spatial information acquisition unit) acquires illuminance information (spatial information) indicating the illuminance of the space captured by the image sensor 1. The illuminance information acquisition unit 53 stores the acquired illuminance information in the storage unit 55.
The floor information acquisition unit 54 acquires floor information of a space imaged by the image sensor 1. The floor information is information indicating a layout such as an office floor. The floor information acquisition unit 54 stores the acquired floor information in the storage unit 55. For example, the floor information acquisition unit 54 acquires floor information from the BEMS 4.

  The storage unit 55 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 55 stores person information, illuminance information, and floor information.

  The control information generation unit 56 (control unit) generates control information for controlling devices related to the space captured by the image sensor 1 based on the person information, the illuminance information, and the floor information. The control information generation unit 56 transmits the generated control information to the control target device. The device to be controlled operates based on the control information generated by the control information generation unit 56.

FIG. 9 is a diagram illustrating a specific example of a method by which the control information generation unit 56 generates control information.
For example, in the control of lighting equipment, it is better to turn on the front lighting rather than the back of the person in order to ensure the comfort of the person working in the office. In this case, as shown in FIG. 9A, the control information generation unit 56 determines the presence or absence of a person, the detected face orientation of the person, and the like based on the person information acquired from the image sensor 1. . The control information generation unit 56 generates control information that turns on the illumination in front of the person based on the orientation of the person's face.

  In addition, for example, at night, it is difficult to work by turning on only one illumination. In this case, as shown in FIG. 9B, the control information generation unit 56 may generate control information that turns on the illumination around the detected person. In addition, the control information generation unit 56 determines the brightness of the space based on the illuminance information, and generates control information that reduces the dimming rate of the illumination when the brightness is excessive due to external light. May be.

  In addition, for example, the control information generation unit 56 determines the status of the entire floor based on the person information and the floor information acquired from the plurality of image sensors 1, and the lighting equipment and air conditioning equipment in the floor are optimal for the entire floor. Control information may be generated such that For example, the control information generation unit 56 determines the distribution of people in the entire floor based on the person information, and generates control information so as to change the lighting equipment to be lit, the airflow direction of the air conditioning, the set temperature of the air conditioning equipment, etc. Also good.

  Thus, in the device control system, by combining the image sensor 1 and the device control device 5, flexible device control according to the state of the person is possible. The device control device 5 is not limited to the illuminance information acquired by the illuminance information acquisition unit 53, and may generate control information based on any information as long as it is information related to determination of device control. In this case, the illuminance information acquisition unit 53 functions as a position information acquisition unit that acquires position information. For example, temperature information indicating the temperature of the space or position information may be acquired from a terminal device held by a person to be detected. The device control device 5 may acquire the person information of the person existing at the position indicated by the position information from the image sensor 1.

FIG. 10 is a functional block diagram showing a functional configuration of the adjustment terminal 7.
The adjustment terminal 7 includes a CPU, a memory, an auxiliary storage device, and the like connected by a bus and executes an adjustment program. The adjustment terminal functions as an apparatus including the communication unit 71, the image collection unit 72, the storage unit 73, the dictionary information generation unit 74, and the parameter adjustment unit 75 by executing the adjustment program. Note that all or part of the functions of the adjustment terminal 7 may be realized using hardware such as an ASIC, PLD, or FPGA. The adjustment program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in the computer system. The adjustment program may be transmitted via a telecommunication line.

  The communication unit 71 is configured using a communication interface for connecting the own apparatus to the device network 40. The communication unit 71 communicates with various devices connected to the device network 40 and the control network 50 via the network hub 2 or the gateway server 3.

  The image collection unit 72 acquires a pre-image necessary for creating dictionary information. The image collection unit 72 may acquire a pre-image from the image sensor 1 or may acquire pre-information from another imaging device different from the image sensor 1. The image collection unit 72 may be configured to acquire a pre-image from a storage medium such as a CD-ROM or a flash memory. In this case, the image collection unit 72 may be configured as an information reading device that reads information from these storage media or an interface that connects the information reading device to the own device. The image collection unit 72 stores the acquired image data in the storage unit 73.

  The storage unit 73 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 73 stores the image data acquired by the image collection unit 72 and the dictionary information generated by the dictionary information generation unit 74.

  The dictionary information generation unit 74 generates dictionary information used by the image sensor 1 to acquire person information based on the collected prior images. In order for the image sensor 1 to acquire person information with sufficient accuracy, the image sensor 1 needs to learn high-quality dictionary information. Therefore, the dictionary information generation unit 74 needs to generate dictionary information using a large amount of pre-images. Since the work of acquiring a large number of pre-images requires a lot of labor, the dictionary information generation unit 74 may have a function of automating the acquisition of pre-images by communicating with the image sensor 1.

  For example, the dictionary information generation unit 74 may have a function as an input unit that accepts user input, and may acquire various setting information related to generation of dictionary information. For example, the dictionary information generation unit 74 may acquire setting information related to the timing at which the image sensor 1 executes acquisition of a prior image. In this case, the dictionary information generation unit 74 may communicate with the image sensor 1 at the timing indicated by the setting information, and instruct the image sensor 1 to acquire a prior image.

  For example, the dictionary information generation unit 74 may acquire setting information related to the timing at which the image collection unit 72 collects the pre-images. In this case, the dictionary information generation unit 74 may instruct the image collection unit 72 to collect a pre-image at the timing indicated by the setting information.

  Further, for example, the dictionary information generation unit 74 may be configured to acquire the above-described user input through communication with another device. When the dictionary information generation unit 74 has such a function, the user can remotely control work related to learning of dictionary information of the image sensor 1. For example, the user can control the learning operation of the image sensor 1 while visually checking each image sensor 1.

  Since the dictionary information generation unit 74 has the above-described function, it is possible to reduce the labor required for acquiring the prior image.

  The parameter adjustment unit 75 determines a recommended value such as a combination of a plurality of dictionary information, a priority order, and a weight in the process of acquiring the person information of the image sensor 1. The parameter adjustment unit 75 transmits the determined recommended value to the image sensor 1.

  A specific example of how the adjustment terminal 7 determines the recommended value is shown below. For example, the adjustment terminal 7 acquires position information from the device control device 5 and acquires information indicating the operation state of each device from the BEMS 4. In this case, the parameter adjustment unit 75 may determine the office usage status of the day based on the position information and the driving information, and may determine the priority order of the dictionary information according to the office usage status.

  For example, when the device is frequently turned on or off, it may be determined that the day when the person goes in and out frequently, and the weight of the dictionary information for identifying the person “walking” may be increased. Further, the distribution of people may be determined based on the position information, and the priority order and determination criteria of the dictionary information may be changed according to the distance from the area where the people are distributed.

  For example, the parameter adjustment unit 75 may determine the recommended value according to the state of the space in which the image sensor 1 is installed. For example, if the number of desks installed in the office is large or the desks are in close contact with each other, the parameter adjustment unit 75 determines a recommended value so as to raise the priority of the head dictionary over the whole body dictionary. May be. On the other hand, if the meeting space has a small number of desks, the parameter adjustment unit 75 may determine the recommended value so as to raise the priority of the whole body dictionary. Alternatively, when the coordinator visually determines the main use of the office and there are many people sitting, the parameter adjustment unit 75 changes the weight in the posture identification, and the priority order of the dictionary information regarding the identification of “sitting position” The recommended value may be determined so as to increase the value. Further, the parameter adjustment unit 75 may digitize the degree of occurrence of overexposure or underexposure due to the performance limit of the camera, and may change a parameter related to feature amount extraction. The person information generated by the image sensor 1 may be used as the information about the person necessary for the analysis regarding the determination of the recommended value.

  By determining priorities and weights in this way, for example, it is possible to change the use of dictionary information according to differences in work styles and events that occur depending on the type of business or department, and the detection performance of the image sensor 1 can be improved. Can be improved.

  In this manner, by combining the image sensor 1 and the adjustment terminal 7, it is possible to improve the accuracy of human detection of the image sensor 1.

  Note that the dictionary information generated by the adjustment terminal 7 may be generated based on image data obtained by imaging another space having similar characteristics. Here, the dictionary information generated in this way is referred to as prior dictionary information. On the other hand, dictionary information generated based on an image of a detection target space is referred to as local dictionary information. In this case, by generating the dictionary information by combining the pre-dictionary information and the local dictionary information, it is possible to reduce the quality of the dictionary information while reducing the labor for generating the dictionary information.

  For example, in a building or the like, it is generally assumed that collection of learning images is performed by a coordinator during adjustment before a person enters the building. If the dictionary information is generated by the method as described above, the operation can be started using the dictionary information generated before moving in, and the quality of the dictionary information can be improved with the image captured during the operation.

  According to at least one embodiment described above, an imaging unit that captures a space of a target for detecting a person, a storage unit that stores a plurality of dictionary information indicating human characteristics, an image captured by the imaging unit, A person information acquisition unit that detects a person from an image based on a plurality of dictionary information and acquires person information indicating the state of the detected person, so that a person can be accurately detected from an image captured over a wide area. Can be detected.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1, 1-1, 1-2, 1-3 ... Image sensor, 11 ... Communication part, 12 ... Imaging part, 13 ... Dictionary information storage part, 14 ... Person information acquisition part, 2 ... Network hub, 3 ... Gateway server , 4 ... BEMS (Building Energy Management System), 5 ... Device control device, 51 ... Communication unit, 52 ... Person information acquisition unit, 53 ... Illuminance information acquisition unit, 54 ... Floor information acquisition unit, 55 ... Storage unit, 56 ... Control information generation unit, 6 ... visualization device, 7 ... adjustment terminal, 71 ... communication unit, 72 ... image collection unit, 73 ... storage unit, 74 ... dictionary information generation unit, 75 ... parameter adjustment unit, 10 ... ceiling, 20 ... Lighting equipment, 30 ... Air conditioning equipment, 40 ... Equipment network, 50 ... Control network, 100-1, 100-2, 100-3, 100-4, 100-5 ... Image, 201, 202, 203 ... Divided area

Claims (15)

An imaging unit that images a space of a target for detecting a person;
A storage unit for storing a plurality of dictionary information indicating human characteristics;
A detection unit for detecting a person from the image based on the image captured by the imaging unit and the plurality of dictionary information;
An image sensor comprising: The storage unit stores, for each of a plurality of regions into which the image is divided, a plurality of dictionary information indicating human characteristics according to the divided regions,
The detection unit detects a person for each of the plurality of divided regions based on the image captured by the imaging unit and the plurality of dictionary information.
The image sensor according to claim 1. The storage unit stores a plurality of dictionary information indicating a person's state,
A personal information acquisition unit that acquires personal information indicating the state of the person detected by the detection unit based on the plurality of dictionary information;
The image sensor according to claim 1 or 2. The storage unit stores a plurality of dictionary information indicating characteristics of a person's whole body or body part,
The person information acquisition unit acquires the person information from an image obtained by capturing the whole body or part of a person based on the plurality of dictionary information.
The image sensor according to claim 2. The person information acquisition unit acquires a plurality of person information in parallel based on the plurality of dictionary information, and a person to be adopted from the plurality of person information according to a priority or weight of the plurality of dictionary information Determine information,
The image sensor according to any one of claims 1 to 4. The person information acquisition unit acquires new person information based on the acquired person information and the plurality of dictionary information.
The image sensor as described in any one of Claim 1 to 5. The imaging unit images the space using a lens capable of imaging a wide range,
The storage unit stores, for each of a plurality of regions into which the image is divided according to distortion by the lens, a plurality of dictionary information indicating human characteristics according to the divided regions,
The person information acquisition unit acquires the person information from an image captured with distortion based on the plurality of dictionaries.
The image sensor according to any one of claims 2 to 6. A human detection method performed by an image sensor comprising: an imaging unit that images a space to be detected by a person; and a storage unit that stores a plurality of dictionary information indicating human characteristics,
A detection step of detecting a person from the image based on the image captured by the imaging unit and the plurality of dictionary information;
Person detection method. In a human detection method performed by an image sensor including an imaging unit that images a space of a target for detecting a person and a storage unit that stores a plurality of dictionary information indicating characteristics of the person,
A detection step of detecting a person from the image based on the image captured by the imaging unit and the plurality of dictionary information;
A computer program for causing a computer to execute. The image sensor according to any one of claims 1 to 7,
A spatial information acquisition unit that acquires spatial information indicating a state of a space captured by the image sensor;
Control information for controlling the device related to the space is acquired based on the personal information indicating the state of the person existing in the space acquired by the image sensor and the spatial information acquired by the spatial information acquisition unit. A control unit;
A control system comprising: The control unit acquires the control information based on person information acquired from a plurality of the image sensors.
The control system according to claim 10. It further communicates with a terminal device held by a person to be detected, and further comprises a position information acquisition unit that acquires position information indicating the position of the terminal device from the terminal device,
The spatial information acquisition unit acquires information indicating a state of the device related to the space as spatial information,
The image sensor acquires person information of a person existing at a position indicated by the position information in the captured image;
The control system according to claim 11. A dictionary information generating unit that acquires an image from the image sensor and generates dictionary information used by the image sensor to acquire person information based on the acquired image;
When the image sensor acquires person information using a plurality of dictionary information generated by the dictionary information generating unit, an adjustment unit that determines a combination value of the dictionary information, a priority or a recommended value of weight,
A control system further comprising: A control method performed by a control system including the image sensor according to claim 1,
A spatial information acquisition step of acquiring spatial information indicating a state of a space captured by the image sensor;
Acquire control information for controlling a device related to the space based on the personal information indicating the state of a person existing in the space acquired by the image sensor and the spatial information acquired in the spatial information acquisition step. A control step to
A control method. In the control method which a control system provided with the image sensor as described in any one of Claim 1 to 7 performs.
A spatial information acquisition step of acquiring spatial information indicating a state of a space captured by the image sensor;
Acquire control information for controlling a device related to the space based on the personal information indicating the state of a person existing in the space acquired by the image sensor and the spatial information acquired in the spatial information acquisition step. A control step to
A computer program for causing a computer to execute.

Images