JP7145693B2 - SEAT CONDITION SENSING SYSTEM, SEAT CONDITION DETERMINATION METHOD, AND IMAGE SENSOR - Google Patents

Description

Embodiments of the present invention relate to systems, methods, and image sensors for sensing the state of a seat (chair and/or table).

2. Description of the Related Art In recent years, services for guiding users to available seats in free-address offices and commercial facilities (such as food courts) have been considered. For example, people's positions can be measured by means of people flow analysis technology or existing image analysis technology. However, in order to determine whether a seat is in use or not, not only the position of the person but also the position of the chair must be acquired, and such technology is not known. Furthermore, since the chairs and tables may move, in such a case, it is not possible to correctly grasp the state of the seat (for example, seat in use/empty).

JP 2013-61832 A JP 2016-126373 A JP 2014-137190 A

In order to grasp the state of the seat, it is necessary to obtain not only the positional information of the person but also the positional information of the chair accurately, and then process this information appropriately.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a seat state sensing system, a seat state determination method, and an image sensor capable of accurately grasping the state of a seat.

According to embodiments, a seat condition sensing system comprises an image sensor having an imaging element. The image sensor generates human detection information indicating position information of a person in the target space from image data obtained by imaging the target space with the imaging element, and detects the position of the chair in the target space from the image data. a detection unit that generates chair detection information indicating position information; a determination unit that determines a seat state based on the person detection information and the chair detection information and generates seat state information that indicates the state of the seat; and a storage section storing dictionary data for human detection and a plurality of dictionary data for chair detection storing shape information of the chair, wherein the detection section stores the dictionary data for human detection. and generating the chair detection information using the dictionary data for chair detection are alternately performed. A plurality of dictionary data are sequentially switched and used .

FIG. 1 is a diagram showing an example of an indoor space. FIG. 2 is a diagram illustrating an example of a sensing system according to the embodiment; FIG. 3 is a block diagram showing an example of the image sensor 3. As shown in FIG. FIG. 4 is a diagram showing an example of information flow between the functional blocks shown in FIG. FIG. 5 is a diagram showing an example of human detection information. FIG. 6 is a diagram showing an example of chair detection information. FIG. 7 is a diagram showing an example of seat state information. FIG. 8 is a diagram showing an example of information transmission in the first embodiment. FIG. 9 is a block diagram showing another example of the image sensor 3. As shown in FIG. FIG. 10 is a diagram showing an example of information flow between functional blocks shown in FIG. FIG. 11 is a block diagram showing another example of the image sensor 3. As shown in FIG. FIG. 12 is a diagram showing an example of information flow between functional blocks shown in FIG. FIG. 13 is a block diagram showing another example of the image sensor 3. As shown in FIG. 14 is a diagram showing an example of information flow between the functional blocks shown in FIG. 13. FIG. FIG. 15 is a block diagram showing another example of the image sensor 3. As shown in FIG. 16 is a diagram showing an example of information flow between the functional blocks shown in FIG. 15. FIG. FIG. 17 is a flow chart showing an example of the processing procedure of the processor 33 in the fifth embodiment. FIG. 18 is a block diagram showing another example of the image sensor 3. As shown in FIG. 19 is a diagram showing an example of information flow between the functional blocks shown in FIG. 18. FIG. FIG. 20 is a diagram showing an example of information transmission in the sixth embodiment. FIG. 21 is a diagram showing an example of information transmission in the seventh embodiment. FIG. 22 is a functional block diagram showing an example of the gateway 200. As shown in FIG. FIG. 23 is a diagram showing an example of information flow between functional blocks shown in FIG. FIG. 24 is a block diagram showing another example of the image sensor 3. As shown in FIG. FIG. 25 is a diagram showing an example of information flow between functional blocks shown in FIG. FIG. 26 is a diagram showing an example of information transmission in the ninth embodiment. FIG. 27 is a functional block diagram showing an example of the server 600 provided in the cloud 400. As shown in FIG. FIG. 28 is a diagram showing an example of information transmission in the tenth embodiment. FIG. 29 is a diagram showing another example of the sensing system. FIG. 30 is a diagram showing another example of the sensing system. FIG. 31 is a diagram showing another example of the sensing system. FIG. 32 is a diagram showing another example of the sensing system. FIG. 33 is a diagram showing another example of the sensing system.

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram showing an example of an indoor space. As shown in FIG. 1, a lighting device 1, an air outlet of an air conditioner 2, and an image sensor 3 are arranged on, for example, the ceiling of each floor of an office building.

FIG. 2 is a diagram illustrating an example of a sensing system according to the embodiment; In FIG. 2, a gateway (GW) 200, a router 300, and a controller 500 are connected to a LAN (Local Area Network) formed in an office building. Among them, the controller 500 controls air conditioning equipment, lighting equipment, etc. in the building based on instructions from an upper BEMS (Building Energy Management System) server (not shown) or the like.

The router 300 communicably connects the intra-building LAN and the cloud computing system (cloud) 400 via a public communication line such as the Internet or a dedicated line. Thereby, the sensing system can exchange various data with the cloud 400 . Examples of communication protocols on the LAN include BACnet (registered trademark), LONWORKS (registered trademark), KNX (registered trademark), DALI, ZigBee (registered trademark), and ECHONET Lite (registered trademark), which are communication standards for intelligent building networks. ) can be applied.

The gateway 200 accommodates a plurality of image sensors 3 (3-1 to 3-n) under its own control. Image sensors 3 - 1 to 3 - n are connected to gateway 200 via sensor network 100 . The sensor network 100 is, for example, EtherCAT (registered trademark), and connects the image sensors 3-1 to 3-n and the gateway 200 so as to be able to communicate with each other through a line having a loop structure of transition wiring. The BEMS server, controller 500 and router 300 can also communicate with the image sensors 3-1 to 3-n via the gateway 200. FIG.

The image sensor 3 acquires image data by photographing the target space. The image sensor 3 processes the acquired image data and creates sensing information such as environment information and person information. The environmental information is information about the environment of the space to be controlled (target space), and includes, for example, the distribution of illuminance in the target space. Person information is information about people in the target space, such as presence or absence (presence/absence) of people, length of stay, number of people, behavior of people, amount of activity of people, and clothing information about people's clothes. can be mentioned. The sensing information is transmitted to the controller 500, the server on the cloud 400, etc. via the gateway 200. FIG. Next, a plurality of embodiments will be described based on the above configuration.

[First Embodiment]
FIG. 3 is a block diagram showing an example of the image sensor 3. As shown in FIG. The image sensor 3 includes an imaging section 31 , a storage section 32 , a processor 33 and a communication section 34 . These are connected to each other via an internal bus 35 .

The imaging unit 31 includes a fisheye lens 31a, an aperture mechanism 31b, an imaging element 31c, and a register 30. The fish-eye lens 31a captures the interior of the room, which is the target space, in a view looking down from the ceiling, for example, and forms an image on the imaging element 31c. The imaging element 31c is an image sensor represented by a CMOS (Complementary Metal Oxide Semiconductor), and generates a video signal with a frame rate of 30 frames per second, for example. This video signal is digitally encoded and output as image data. The amount of light incident on the imaging device 31c is adjusted by the diaphragm mechanism 31b.

The register 30 stores camera information 30a. The camera information 30a is information about the imaging unit 31, such as the state of the auto gain control function, gain value, and exposure time, or information about the image sensor 3 itself.

The storage unit 32 is a semiconductor memory such as DRAM (Dynamic Random Access Memory), SDRAM (Synchronous Dynamic RAM), EPROM (Erasable Programmable ROM), and stores image data 32a acquired by the imaging unit 31 and various It stores a program 32b for causing the processor 33 to execute a function, a sensor ID (IDentification) 32c for uniquely identifying an image sensor, and the like.

The processor 33 loads and executes programs stored in the storage unit 32 to implement various functions described in the embodiments. The processor 33 is, for example, a multi-core CPU (Central Processing Unit) and is an LSI (Large Scale Integration) tuned to execute image processing at high speed. The processor 15 can also be configured with an FPGA (Field Programmable Gate Array) or the like. An MPU (Micro Processing Unit) is also one of the processors.
The communication unit 34 is connectable to the sensor network 100 and mediates the transfer of data with a communication partner (controller 500 or the like).

By the way, the processor 33 includes a person detection unit 33a, a chair detection unit 33b, and a determination unit 33c as processing functions according to the embodiment. The person detection unit 33a, the chair detection unit 33b, and the determination unit 33c are processes generated in the course of the processor 33 executing arithmetic processing according to the program 32b loaded in the register of the processor 33, for example.

The person detection unit 33a analyzes the image data 32a accumulated in the storage unit 32 using a predetermined algorithm, detects a person (person) within the field of view, and generates person detection information indicating the position information of the person. The chair detection unit 33b analyzes the image data 32a with a predetermined algorithm to detect a chair within the field of view, and generates chair detection information indicating the position information of the chair. The positional information may be positional coordinates in an absolute coordinate system in the target space, or may be positional coordinates in a coordinate system set for each of the image sensors 3-1 to 3-n.

By analyzing the image data 32a, features such as histograms of oriented gradients (HOG) features, contrast, resolution, signal-to-noise ratio, and color tone can be calculated. In addition, luminance gradient direction co-occurrence histogram (Co-occurrence HOG: Co-HOG) feature amount, Haar-Like feature amount, etc. are also known. Objects within the field of view can be identified based on such feature amounts. Alternatively, it is also possible to identify the target within the field of view by a method such as pattern recognition using dictionary data that stores the shape information of the detection target (person or chair).

Based on the person detection information and the chair detection information, the determination unit 33c determines whether each seat is vacant or in use, and generates seat state information. For example, if the distance between the person and the chair calculated from the position coordinates of the person indicated by the person detection information and the position coordinates of the chair indicated by the chair detection information is equal to or less than a predetermined value, the determination unit 33c determines that the chair is in use, and the seat associated with that chair is also in use.

FIG. 4 is a diagram showing an example of information flow between the functional blocks shown in FIG. The image data acquired by the imaging unit 31 is passed to the person detection unit 33a and the chair detection unit 33b. The human detection unit 33 a analyzes the image data to generate human detection information, and passes this human detection information to the determination unit 33 c and the communication unit 34 . The human detection information is, for example, information generated by associating a detection position with a human detection ID (IDentification), as shown in FIG. A person detection ID is an identifier of an object detected as a person in image data, and is generally associated with an individual person. The detection position indicates the detection position coordinates of the object assigned the human detection ID, and is represented by, for example, numerical values in the XY coordinate system of the image sensor that is the detection source.

The chair detection unit 33b analyzes the image data to generate chair detection information, and passes this chair detection information to the determination unit 33c.
The chair detection information is, for example, information generated by associating a detection position with a chair detection ID, as shown in FIG. A chair detection ID is an identifier of an object detected as a chair in the image data. The detected position indicates the detected position coordinates of the object assigned the chair detection ID, and is represented by, for example, numerical values in the XY coordinate system of the image sensor that is the detection source.

The chair detection ID may be associated with information (detection type) indicating the shape, classification, type, characteristics, and the like of the detected chair. FIG. 6 shows that three types of chairs have been detected, distinguished by numerical values 1, 2, and 3, respectively.

The determination unit 33c that has acquired the human detection information and the chair detection information calculates the distance between the person and the chair using the Pythagorean theorem or the like based on the positional coordinates of the person and the positional coordinates of the chair. If the calculated distance is equal to or less than a predetermined threshold, the determination unit 33c determines that the chair is being used by a person near the chair, and otherwise determines that the chair is vacant. The determination unit 33c determines whether each chair (seat) is in use or vacant, and based on the result, generates seat state information indicating the state of the seat. The generated seat state information is passed to the communication unit 34 .

FIG. 7 is a diagram showing an example of seat state information. The seat state information is information in which a seat ID for distinguishing a seat is associated with its position coordinates and information indicating whether the chair is in use or vacant. The seat state information may include the detection type of the chair.

The state of the chair is not limited to being distinguished by two states (True/False, 0/1, etc.) of being in use and being vacant. For example, the probability that the chair is in use may be represented by a continuous numerical value (eg, 0 to 1). Alternatively, the probability that the chair is in use may be represented by a value between 0% and 100%. Alternatively, the probability that the chair is in use may be represented by graded ranks (integer values).
The communication unit 34 (FIG. 4), which receives the seat state information and the human detection information, transmits these information to, for example, an adjacent image sensor together with its own sensor ID.

FIG. 8 is a diagram showing an example of information transmission in the first embodiment. Each image sensor 3 sequentially generates and outputs seat state information (and human detection information), and this information is sequentially transmitted to the gateway 200 via adjacent image sensors in, for example, a bucket brigade system. Gateway 200 transmits seat state information with sensor ID and human detection information to controller 500 and/or router 300 via LAN.

As described above, in the first embodiment, the image data acquired by the image sensor 3 is image-processed to detect the positional coordinates of the person and the positional coordinates of the chair, and the positional coordinates of the person are detected based on the respective positional coordinates. and the chair. The image sensor 3 is provided with a determination unit 33c, and if the distance between the person and the chair is equal to or less than a specified value, it is determined that the chair is in use. With this arrangement, it is possible for the edge device to determine whether the chair and seat are in use or empty based on objective indicators.

None of the existing techniques refer to the correspondence between the position of a person and the position of a chair, much less the technique of determining whether a chair is in use or vacant. It is possible to treat the position of the chair as a fixed value to some extent and estimate the seat situation from the position information.

In addition, existing sensor technology can only detect the presence/absence of one or more people in a detection area of several meters, and cannot acquire detailed position information. Inability to identify multiple persons. It is also difficult to measure whether a person is sitting on a seat and using it, especially in a wide area such as a store or office.

On the other hand, in the first embodiment, the image data from the image sensor is processed to detect a chair, its position coordinates are acquired, and together with the position information of the person, the situation of the seat (i.e., whether the chair is being used / chair not in use). Therefore, even if the chair moves, the state can be correctly grasped. Furthermore, at the time of initial introduction, the position of the chair can be automated without manual input. For these reasons, according to the first embodiment, it is possible to provide a seat state sensing system, a seat state determination method, and an image sensor that enable an accurate grasp of the seat state.

[Second embodiment]
In the first embodiment, assuming that the position of the chair changes, the image sensor acquires the position information of the chair. In the second embodiment, it is assumed that the chair is fixed, such as a sofa in a restaurant or coffee shop.

FIG. 9 is a block diagram showing another example of the image sensor 3. As shown in FIG. In the second embodiment, the image sensor 3 stores chair position information 32d in the storage unit 32. FIG. The chair position information 32d is a database in which position coordinates are associated with each chair and recorded.

FIG. 10 is a diagram showing an example of information flow between functional blocks shown in FIG. The image data acquired by the imaging unit 31 is passed to the person detection unit 33a. The human detection unit 33 a analyzes the image data to generate human detection information, and passes this human detection information to the determination unit 33 c and the communication unit 34 .

The determination unit 33c that has acquired the human detection information further reads the chair position information 32d from the storage unit 32, and calculates the distance between the person and the chair based on the position coordinates of the person and the chair. If the calculated distance is equal to or less than the threshold, the determination unit 33c determines that the chair is in use, otherwise determines that the chair is vacant. The seat state information generated based on the determination result is passed to the communication unit 34 and transmitted to the communication partner together with the person detection information and the sensor ID.

In the second embodiment, assuming that the chair is fixed, the position information of the chair is set in the image sensor 3 in advance. Then, the positional information of the person is acquired by the image sensor, the positional relationship between the person and the chair is obtained using the positional information of the chair, and it is determined whether the individual chair is in use or vacant. Therefore, according to the second embodiment as well, it is possible to accurately grasp the state of the seat.

[Third Embodiment]
FIG. 11 is a block diagram showing another example of the image sensor 3. As shown in FIG. In the third embodiment, the image sensor 3 further stores chair position information 32d in the storage unit 32 in addition to the configuration shown in FIG.

FIG. 12 is a diagram showing an example of information flow between functional blocks shown in FIG. In the third embodiment, the determination unit 33c acquires chair position information in addition to person detection information and chair detection information. Then, based on these three types of information, the determination unit 33c determines the use state of the chair.
With this configuration, it is possible to accurately grasp the state of the seat also according to the third embodiment. In addition, accuracy of determination can be improved.

[Fourth embodiment]
FIG. 13 is a block diagram showing another example of the image sensor 3. As shown in FIG. In the fourth embodiment, the image sensor 3 stores dictionary data 32e in the storage unit 32. FIG.

The dictionary data 32e is data similar to so-called template data prepared according to detection items (person, object, male, female, etc.) in the image sensor 3. FIG. A chair is detected from the image data by performing template matching processing on the image data acquired by the image sensor 3 using, for example, the dictionary data 32e having the shape information of the chair prepared for the detection of "chair". be done. A variety of dictionary data 32e are prepared in advance in order to distinguish the accuracy, type of chair (office chair, sofa, legless chair, etc.) and color. In addition, it is possible to create dictionary data for detecting wheelchairs, strollers, children, carts, white cane users, loitering old people, suspicious persons, lost children, persons who failed to escape, and the like. As an example of the creation method, a machine-learning framework such as a support vector machine or a Boltzmann machine can be used.

14 is a diagram showing an example of information flow between the functional blocks shown in FIG. 13. FIG. There are various types/shapes of chairs, each of which requires a dictionary. However, even if we try to create a dictionary containing information on all existing chair types/shapes, the size of the data would be too large to handle for embedded applications (such as image sensors) with limited memory space and processing power. is difficult.

Therefore, in the fourth embodiment, dictionary data for each chair type (dictionary: chair 1 to dictionary: chair n) is created and stored in the image sensor 3 in advance. Then, the dictionary data of each chair is switched sequentially to carry out chair detection processing. The types of chairs that exist in the target space where the image sensor 3 is installed are limited to some extent, and this tendency is particularly strong in office spaces and the like. Therefore, it is possible to flexibly and accurately detect a chair by assuming in advance the types of chairs that are likely to exist and switching between a plurality of dictionaries.

As shown in FIG. 14, for example, n dictionary data relating to chairs are stored in the storage unit 32 of the image sensor 3 . The chair detection unit 33b processes image data while sequentially switching dictionary data to detect a chair. By adding information indicating which dictionary was used when a chair was detected, it is possible to distinguish which type of chair was detected. With this configuration, it is possible to accurately grasp the state of the seat also according to the fourth embodiment. In addition, accuracy of determination can be improved.

[Fifth Embodiment]
FIG. 15 is a block diagram showing another example of the image sensor 3. As shown in FIG. In the fifth embodiment, the image sensor 3 includes a detection section 33d in the processor 33 and stores dictionary data 32e in the storage section 32 . The detection unit 33d has a function of detecting objects in the object space based on dictionary data prepared in advance, and can detect various objects by switching the dictionary data. That is, in the fifth embodiment, both the person and the chair are detected by the common detection unit. As shown in FIG. 16, n different dictionaries (dictionary: chair 1 to dictionary: chair n) are used for chair detection, and a dictionary for human detection (dictionary: person) is used for human detection. Then, an object detected using any one of (dictionary: chair 1 to dictionary: chair n) is recognized as a chair, and chair detection information is generated. On the other hand, an object detected using (dictionary: person) is recognized as a person, and person detection information is generated.

By the way, unlike a chair, a person moves voluntarily. Therefore, in order to accurately grasp the movement of a person, it is necessary to make the detection cycle shorter than that of the chair. However, if human detection is included in the detection process for many types of chairs, the processing time for human detection becomes long.

In other words, if the image sensor also performs chair detection processing, the processing amount of the image processing increases, which may exceed the computational resources of the image sensor, extending the sensing interval and degrading the response. Since this causes a decrease in detection accuracy and the like, countermeasures are required.

Conversely, the chair does not move spontaneously, and when the chair moves, it can be assumed that there is human intervention. The drop is acceptably small. Based on such characteristics of the detection target, the fifth embodiment optimizes the period of human detection and the period of chair detection.

FIG. 17 is a flow chart showing an example of the processing procedure of the processor 33 in the fifth embodiment. The processor 33 first detects an object using the dictionary data of (person) (step S1). Next, the dictionary data is switched to (chair 1) to detect the object (step S2), and then the object is detected again using the dictionary data of (person) (step S3). In the next step S4, the dictionary data of (chair 2) is used, and in the next step, the dictionary data of (person) is used again. After the dictionary data of (person) is used in step S5 and the detection using the dictionary data of (chair n) is completed, the processing procedure returns to step S1 and is repeated.

In this way, the detection process using the person's dictionary data is repeatedly executed each time the plural types of chair data (1 to n) are sequentially switched. In other words, people and chairs are detected alternately, and the chair detection process is set to one type (one dictionary) for each switching, so that the human detection process and the detection process for various chairs can be performed at the expense of accuracy. can be compatible without

As described above, in the fifth embodiment, the detection unit 33d switches dictionary data to detect a person and a chair. For example, one type of dictionary data is used to detect a person, and n types of dictionary data are used to detect a chair. Then, the frequency of detecting a chair that can be regarded as not moving is decreased, and the frequency of detecting a moving person is increased. However, when detecting a chair, the dictionary data to be used is switched sequentially. By doing so, processing resources can be distributed in a well-balanced manner to detection of people and detection of a plurality of types of chairs. Of course, the method of switching dictionary data over time is not limited to the example of FIG.

Since this is done, it is possible to accurately grasp the state of the seat also according to the fifth embodiment. Moreover, it is possible to achieve both detection of a person and detection of various types of chairs without deterioration in accuracy.

[Sixth Embodiment]
FIG. 18 is a block diagram showing another example of the image sensor 3. As shown in FIG. In the sixth embodiment, the processor 33 of the image sensor 3 has an image data processing section 33e as its processing function. As shown in FIG. 19, the image data processing unit 33e performs processing such as data compression and container processing on the image data from the imaging unit 31 to generate transmission data in a form suitable for network transmission. pass to The communication unit 34 sends transmission data including image data to the cloud 400 via the gateway 200 together with the person detection information, the chair detection information, and the sensor ID. Gateway 200 and router 300 relay data sent from each image sensor 3 to cloud 400 .

FIG. 20 is a diagram showing an example of information transmission in the sixth embodiment. The cloud 400 includes an image processing unit 400a, a sensor management unit 400b, a dictionary generation unit 400c, and an image database 401 as processing functions implemented in, for example, a data center or a group of distributed server devices.

The image processing unit 400a processes image data collected from a plurality of image sensors 3-1 to 3-n, and extracts feature amounts of each image data.
The sensor management unit 400b has functions such as individually managing a plurality of image sensors 3-1 to 3-n and designating destinations of instruction messages (telegrams).

The dictionary generation unit 400c generates dictionary data related to a detection target (person, chair, etc.) based on the feature amount of the image data passed from the image processing unit 400a.

The image database 401 accumulates image data collected from the image sensors 3-1 to 3-n.

To create dictionary data, image data taken from multiple directions is required for each type of chair. In addition, the processing functions of the image sensor 3 and the gateway 200 are limited because the creation of dictionary data requires a large amount of computer resources. Therefore, in the sixth embodiment, image data is uploaded to the cloud 400 , data processing is performed on the cloud 400 side to create dictionary data, and the dictionary data is distributed to the image sensor 3 .

In FIG. 20, dictionary data is generated by, for example, the following procedure.
(1) In order to create new chair dictionary data, the sensor management unit 400b of the cloud 400 sends an image acquisition request to the image sensor 3 via the gateway 200 .
(2) Upon receiving the request, the image sensor 3 stores the image data in transmission data and transmits the data to the cloud 400 via the gateway 200 and router 300 .
(3) Cloud 400 receives image data from image sensor 3 and stores it in image database 401 . Further, the image processing unit 400a of the cloud 400 cuts out the image of the area of the chair for which the dictionary is to be created from the accumulated image data.
(4) The dictionary generation unit 400c processes the image data of the clipped region and generates dictionary data (for example, dictionary data using CoHOG as a parameter) about chair A.
(5) The image sensors 3-1 to 3-n send requests to the gateway 200 to acquire the new dictionary data generated.
(6) Gateway 200 acquires new dictionary data from cloud 400 .
(7) Gateway 200 transmits the new dictionary data to all image sensors 3 or designated image sensors 3 .
(8) The image sensor 3 that has acquired the new dictionary data updates the dictionary data stored in the storage unit 32 with the new dictionary data and stores the new dictionary data.

As described above, in the sixth embodiment, image data is uploaded to the cloud 400 , dictionary data is generated on the cloud 400 side, and the generated dictionary data is downloaded to the image sensor 3 . By doing so, it is possible to create dictionary data for detecting various and newly appearing chairs. Moreover, since the resources of the cloud 400 are used, the device on the edge side is not overloaded.

With this configuration, it is possible to accurately grasp the state of the seat also according to the sixth embodiment. In addition, the load on edge devices can be reduced.

[Seventh Embodiment]
The dictionary data does not necessarily have to be created and prepared by the system vendor. For example, it is possible to obtain shape information of chairs from a fixture maker that sells chairs, and create dictionary data based on this information. Furthermore, it is also possible to distribute the dictionary data itself from the furniture maker.

FIG. 21 is a diagram showing an example of information transmission in the seventh embodiment. In the seventh embodiment, dictionary data is distributed according to the following procedure, for example. The following procedure can be implemented, for example, in distributing the chair's dictionary data to the image sensors in the area where the chair is located.
(10) For example, dictionary data (chair A) generated in a local development environment or the like by a furniture maker is uploaded in advance to the server of the cloud 400 and stored.
(11) The image sensors 3-1 to 3-n send requests to the gateway 200 to acquire the new dictionary data generated.
(12) Gateway 200 acquires new dictionary data from cloud 400 .
(13) Gateway 200 transmits the new dictionary data to all image sensors 3 or designated image sensors 3 .
(14) The image sensor 3 that has acquired the new dictionary data updates the dictionary data stored in the storage unit 32 to the new dictionary data and stores the new dictionary data.

As described above, in the seventh embodiment, dictionary data separately prepared in advance can be distributed to target image sensors. Since this is done, it is possible to accurately grasp the state of the seat also according to the seventh embodiment. In addition, it is possible to increase the degree of freedom in system construction.

[Eighth Embodiment]
FIG. 22 is a diagram showing an example of the gateway 200 according to the eighth embodiment. The gateway 200 as a seat state determination device is a computer including a processor 250 such as a CPU or MPU, a ROM (Read Only Memory) 220 and a RAM (Random Access Memory) 230 . The gateway 200 further includes a storage unit 210 such as a hard disk drive (HDD), an optical media drive 260 and an interface unit 270 .

The ROM 220 stores basic programs such as BIOS (Basic Input Output System) and UEFI (Unified Extensible Firmware Interface), various setting data, and the like. RAM 230 temporarily stores programs and data loaded from storage unit 210 .

Optical media drive 260 reads digital data recorded on a recording medium such as CD-ROM 280 . Various programs executed by the gateway 200 are recorded on, for example, a CD-ROM 280 and distributed. The program stored in this CD-ROM 280 is read by the optical media drive 260 and installed in the storage section 210 .

The interface unit 270 includes an internal communication unit 270a and an external communication unit 270b. The internal communication unit 270a is connected to the sensor network 100 and controls communication with the image sensors 3-1 to 3-n. The external communication unit 270 b is connected to the intra-building LAN and controls communication with the router 300 , the controller 500 and the cloud 400 . Various programs executed by the gateway 200 can also be downloaded from the cloud 400 via the interface unit 270 and installed in the storage unit 210 . It is also possible to download the latest program from the cloud server via the interface unit 270 and update the installed program.
The storage unit 210 stores person detection information 210a and chair detection information 210b.

The processor 250 executes an OS (Operating System) and various programs. The processor 250 also includes a person detection unit 250a, a chair detection unit 250b, and a determination unit 250c as processing functions according to the eighth embodiment. The person detection unit 250a, the chair detection unit 250b, and the determination unit 250c are processes generated in the course of the processor 250 executing arithmetic processing according to a program loaded in the register of the processor 250, for example. That is, in the eighth embodiment, the determination unit mounted on the image sensor 3 is mounted on the gateway 200 .

FIG. 23 is a diagram showing an example of information flow between functional blocks shown in FIG. In FIG. 23, the internal communication unit 270a of the gateway 200 acquires the chair detection information and the person detection information transmitted from the image sensor 3, and transfers them to the determination unit 250c.

The determination unit 250c determines whether each seat is vacant or in use based on the person detection information and the chair detection information, and generates seat state information. For example, if the distance between the person and the chair calculated from the position coordinates of the person indicated by the person detection information and the position coordinates of the chair indicated by the chair detection information is equal to or less than a predetermined value, the determination unit 250c determines that the chair is in use, and the seat associated with that chair is also in use. The generated seat state information is sent to, for example, the cloud 400 via the external communication unit 270b.

As described above, in the eighth embodiment, the gateway 200 is provided with the determination unit 250c, and the gateway 200 generates the seat state information based on the chair detection information and the person detection information from the image sensor.

With this configuration, it is possible to accurately grasp the state of the seat also according to the eighth embodiment. Additionally, the processing load on the image sensor 3 can be reduced.

[Ninth Embodiment]
FIG. 24 is a block diagram showing another example of the image sensor 3. As shown in FIG. In the ninth embodiment, the image sensor 3 is provided with a person detection unit 33a and a chair detection unit 33b, and the image sensor 3 calculates the person detection information and the chair detection information and transmits them to the gateway 200. FIG. Then, the determination unit is mounted in the gateway 200 so that the gateway 200 side determines the seat state.

FIG. 25 is a diagram showing an example of information flow between functional blocks shown in FIG. In the image sensor 3, the person detection information generated by the person detection unit 33a and the chair detection information generated by the chair detection unit 33b are transmitted by the communication unit 34 to the gateway 200 together with the sensor ID. As shown in FIG. 26, the person detection information and the chair detection information are acquired by the gateway 200 and the seat state information is generated in the gateway 200 . The seat state information is transmitted to, for example, the controller 500 together with the sensor ID, and used to control the lighting equipment and air conditioning equipment in the building.

As described above, in the ninth embodiment as well, the gateway 200 is provided with the determination unit, so the seat state can be accurately grasped and the load on the image sensor 3 can be reduced.

[Tenth embodiment]
FIG. 27 is a functional block diagram showing an example of the server 600 provided in the cloud 400, for example. The server 600 includes a communication processing unit 600b that acquires human detection information and chair detection information from the image sensor 3, and a determination unit 600a that generates seat state information based on the acquired human detection information and chair detection information.

The determination unit 600a determines whether each seat is vacant or in use based on the person detection information and the chair detection information, and generates seat state information. For example, if the distance between the person and the chair calculated from the position coordinates of the person indicated by the person detection information and the position coordinates of the chair indicated by the chair detection information is equal to or less than a predetermined value, the determination unit 250c determines that the chair is in use, and the seat associated with that chair is also in use. The generated seat state information is sent to, for example, an external system via communication processing section 600b.

FIG. 28 is a diagram showing an example of information transmission in the tenth embodiment. 28, the determination unit 600a can be understood as a function of the cloud 400. FIG. 28, the gateway 200 transmits the chair detection information and the person detection information transmitted from the image sensor 3 to the server 600 via the router 300 together with the sensor ID.

The determination unit 400d of the server 600 determines whether each seat is vacant or in use based on the person detection information and the chair detection information, and generates seat state information. The generated seat state information is sent to, for example, an external system together with the sensor ID.

As described above, in the tenth embodiment, the server 600 of the cloud 400 is provided with the determination unit 600a, and the seat state information is generated in the cloud 400 based on the chair detection information and the person detection information from the image sensor. did. Since this is done, it is possible to accurately grasp the state of the seat also according to the tenth embodiment. In addition, processing loads on the image sensor 3 and gateway 200 can be reduced.

[Other embodiments]
It should be noted that the present invention is not limited to the above-described embodiments, and can take various forms.

FIG. 29 is a diagram showing another example of the sensing system. In FIG. 29, the communication device 700 may provide a function for the gateway 200 to directly communicate with the cloud 400 so that the cloud 400 can directly communicate with each image sensor 3 via the gateway 200 . That is, the gateway 200 may be provided with a communication function for wide area network connection.

FIG. 30 is a diagram showing another example of the sensing system. As shown in FIG. 30, as the sensor network 101, it is also possible to employ a multi-drop type topology instead of transition wiring. FIG. 30 shows a form in which the gateway 200 has a communication function for wide area network connection, but as shown in FIG. It is possible.

Alternatively, as shown in FIG. 32, a plurality of image sensors 3-1 to 3-n may be directly connected to the intra-building LAN. Furthermore, as shown in FIG. 33, the image sensor 3 may be provided with a communication device 700, and the image sensor 3 may be provided with a communication function for direct wide area network connection.

While several embodiments of the invention have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.

Reference Signs List 1 Lighting equipment 2 Air conditioning equipment 3 Image sensor 3-1 to 3-n Image sensor 15 Processor 30 Register 30a Camera information 31 Imaging unit 31a Fisheye lens 31b Aperture mechanism 31c... Imaging device 32... Storage unit 32a... Image data 32b... Program 32d... Chair position information 32e... Dictionary data 33... Processor 33a... Person detection unit 33b... Chair detection unit 33c Determination unit 33d Detection unit 33e Image data processing unit 34 Communication unit 35 Internal bus 100 Sensor network 101 Sensor network 200 Gateway 210 Storage unit 210a Person detection information , 210b...Chair detection information, 220...ROM, 230...RAM, 250...Processor, 250a...Human detection unit, 250b...Chair detection unit, 250c...Judgment unit, 260...Optical media drive, 270...Interface unit, 270a...Inside Communication unit 270b External communication unit 300 Router 400 Cloud (cloud computing system) 400a Image processing unit 400b Sensor management unit 400c Dictionary generation unit 400d Determination unit 401 Image database , 500... Controller, 600... Server, 600a... Determination unit, 600b... Communication processing unit, 700... Communication device.

Claims (11)

Equipped with an image sensor having an imaging element,
The image sensor is
Human detection information indicating the position information of a person in the target space is generated from the image data obtained by imaging the target space with the imaging device, and the chair indicating the position information of the chair in the target space from the image data. a detector that generates detection information;
a determination unit that determines a seat state based on the person detection information and the chair detection information and generates seat state information indicating the state of the seat ;
a storage unit in which dictionary data for human detection and a plurality of dictionary data for chair detection storing shape information of the chair are recorded ;
The detecting unit alternately generates the human detection information using the dictionary data for human detection and generates the chair detection information using the dictionary data for chair detection. A seat state sensing system that sequentially switches and uses a plurality of dictionary data for chair detection when generating chair detection information . Equipped with an image sensor having an imaging element and a server capable of communicating with the image sensor,
The image sensor is
Human detection information indicating the position information of a person in the target space is generated from the image data obtained by imaging the target space with the imaging device, and the chair indicating the position information of the chair in the target space from the image data. a detection unit that generates detection information;
a storage unit in which dictionary data for human detection and a plurality of dictionary data for chair detection storing shape information of the chair are recorded ;
The detecting unit alternately generates the human detection information using the dictionary data for human detection and generates the chair detection information using the dictionary data for chair detection. when generating chair detection information, sequentially switching and using a plurality of dictionary data for chair detection,
The server is
A seat state sensing system comprising a determination unit that determines a state of a seat based on the person detection information and the chair detection information and generates seat state information indicating the state of the seat. comprising an image sensor having an imaging element and a gateway device that communicably connects the image sensor to a network;
The image sensor is
Human detection information indicating the position information of a person in the target space is generated from the image data obtained by imaging the target space with the imaging device, and the chair indicating the position information of the chair in the target space from the image data. a detection unit that generates detection information;
a storage unit in which dictionary data for human detection and a plurality of dictionary data for chair detection storing shape information of the chair are recorded ;
The detecting unit alternately generates the human detection information using the dictionary data for human detection and generates the chair detection information using the dictionary data for chair detection. when generating chair detection information, sequentially switching and using a plurality of dictionary data for chair detection,
The gateway device
A seat state sensing system comprising a determination unit that determines a state of a seat based on the person detection information and the chair detection information and generates seat state information indicating the state of the seat. The determination unit
4. The chair is determined to be in use if the distance calculated from the coordinates indicating the position information of the person and the coordinates indicating the position information of the chair is equal to or less than a predetermined value. 2. The seat state sensing system according to item 1. 4. The seat state sensing system according to any one of claims 1 to 3, wherein in the seat state information, the determination unit indicates whether or not the chair is in use by two states, occupied/empty seat. . 4. The seat state sensing system according to any one of claims 1 to 3, wherein, in the seat state information, the determination unit expresses the probability that the chair is in use by a continuous numerical value. 4. The seat state sensing system according to any one of claims 1 to 3, wherein said determination unit expresses the degree of certainty that said chair is in use in said seat state information by graded ranks. A seat state determination method applicable to a sensing system equipped with an image sensor having an imaging element,
the image sensor acquires image data by imaging the target space with the imaging device;
The image sensor generates human detection information indicating position information of a person in the target space from the image data using dictionary data for human detection, and stores shape information of a chair from the image data. generating chair detection information indicating the position information of the chair in the target space by using any one of a plurality of dictionary data for chair detection; and generating the chair detection information when generating the chair detection information. use multiple dictionary data for
A seat state determination method, wherein the image sensor determines the state of the seat based on the person detection information and the chair detection information. A seat state determination method applicable to a sensing system comprising an image sensor having an imaging element and a server capable of communicating with the image sensor,
the image sensor acquires image data by imaging the target space with the imaging device;
The image sensor generates human detection information indicating position information of a person in the target space from the image data using dictionary data for human detection, and stores shape information of a chair from the image data. generating chair detection information indicating the position information of the chair in the target space by using any one of a plurality of dictionary data for chair detection; and generating the chair detection information when generating the chair detection information. use multiple dictionary data for
the server
A seat state determination method, wherein a seat state is determined based on the person detection information and the chair detection information. A seat state determination method applicable to a sensing system comprising an image sensor having an imaging device and a gateway device communicably connecting the image sensor to a network,
the image sensor acquires image data by imaging the target space with the imaging device;
The image sensor generates human detection information indicating position information of a person in the target space from the image data using dictionary data for human detection, and stores shape information of a chair from the image data. generating chair detection information indicating the position information of the chair in the target space by using any one of a plurality of dictionary data for chair detection; and generating the chair detection information when generating the chair detection information. use multiple dictionary data for
A seat state determination method, wherein the gateway device determines a seat state based on the person detection information and the chair detection information. an imaging device that captures an image of a target space and acquires image data;
a detection unit that generates person detection information indicating position information of a person in the target space from the image data, and generates chair detection information indicating position information of a chair in the target space from the image data;
a determination unit that determines a seat state based on the person detection information and the chair detection information and generates seat state information indicating the state of the seat ;
a storage unit in which dictionary data for human detection and a plurality of dictionary data for chair detection storing shape information of the chair are recorded ;
The detecting unit alternately generates the human detection information using the dictionary data for human detection and generates the chair detection information using the dictionary data for chair detection. An image sensor that sequentially switches and uses a plurality of dictionary data for chair detection when generating chair detection information .

Images