JP7555044B2 - Image sensor, image sensor system, image sensor output method, and program - Google Patents

Description

The present disclosure generally relates to an image sensor, an image sensor system, an image sensor output method, and a program. More specifically, the present disclosure relates to an image sensor, an image sensor system, an image sensor output method, and a program that output captured images.

For example, as disclosed in Patent Document 1, there is a conventional image sensor that can determine the presence or absence of a person based on captured image data, transmit a detection signal indicating the presence or absence of a person via a transmission line, and also transmit the captured image data via the transmission line.

JP 2019-29301 A

In conventional image sensors such as those described in Patent Document 1, there was a possibility that the transmission capacity of image data (image information) would be insufficient when a detection signal (determination information) was output.

The present disclosure has been made in consideration of the above problems, and aims to provide an image sensor, an image sensor system, an image sensor output method, and a program that can maintain the transmission capacity of image information even when outputting judgment information.

An image sensor according to one aspect of the present disclosure includes an imaging unit, a determination unit, a first communication unit, and a second communication unit. The imaging unit captures an image of a predetermined space and generates data of the captured image. The determination unit determines the state of a person in the space based on the captured image. The first communication unit outputs determination information based on the determination result of the determination unit. The second communication unit outputs image information including the captured image and communicates with the first communication unit using a communication method different from that of the first communication unit . The output destinations of the first communication unit and the second communication unit, as well as the communication path between the first communication unit and the output destination of the first communication unit and the communication path between the second communication unit and the output destination of the second communication unit, are different. The first communication unit and the second communication unit can output at least the determination information and the image information simultaneously.

An image sensor system according to one aspect of the present disclosure includes the image sensor described above and a receiving device that receives at least one of the determination information and the image information.

An output method of an image sensor according to one aspect of the present disclosure includes an imaging step, a determination step, a first communication step, and a second communication step. The imaging step involves capturing an image of a predetermined space and generating captured image data. The determination step involves determining the state of a person in the space based on the captured image. The first communication step involves the first communication unit outputting determination information based on the determination result of the determination step. The second communication step involves the second communication unit outputting image information including the captured image and communicating using a communication method different from that of the first communication step . The output destinations of the first communication unit and the second communication unit, as well as the communication path between the first communication unit and the output destination of the first communication unit and the communication path between the second communication unit and the output destination of the second communication unit, are different. The output method of an image sensor allows the first communication step and the second communication step to be performed simultaneously.

A program according to one aspect of the present disclosure causes a computer to execute the image sensor output method described above.

This disclosure has the advantage that the transmission capacity of image information can be maintained even when judgment information is output.

FIG. 1 is a block diagram showing a configuration of a management system including an image sensor according to an embodiment. FIG. 2 is a plan view showing a management space of the management system. FIG. 3 is a block diagram showing the configuration of the image sensor of the above embodiment. FIG. 4 is a schematic diagram showing an image captured by the image sensor of the above embodiment. FIG. 5 is a flowchart showing an output method of the image sensor.

One embodiment of the present invention will be described below. The embodiment described below is merely one of the various embodiments of the present invention. In addition, the embodiment described below can be modified in various ways depending on the design, etc., as long as the object of the present invention can be achieved.

(1) Overview of Management System First, an overview of a management system 10 according to the present embodiment will be described with reference to FIG.

The management system 10 comprises a center device 1, a lighting controller 21, an entrance/exit controller 22, an air conditioning controller 23, a communication interface 3, a hub (concentrator) 4, an image sensor 5, and a transmission adapter 6. The management system 10 manages or monitors the people in the managed space R1, as well as each environment of the managed space R1, such as the lighting and air conditioning. The managed space R1 is, for example, an office, a factory, or a store. The center device 1 and the image sensor 5 constitute an image sensor system 20.

The center device 1 corresponds to the receiving device of the present disclosure, and is configured to be able to communicate with the lighting controller 21, the entry/exit controller 22, and the air conditioning controller 23 via the communication line W1. In this embodiment, the center device 1 is connected to the lighting controller 21, the entry/exit controller 22, and the air conditioning controller 23 via the communication line W1. The center device 1 communicates with the lighting controller 21, the entry/exit controller 22, and the air conditioning controller 23 in accordance with a communication protocol for a building network, such as BACnet (Building Automation and Control Network). Furthermore, the center device 1 is also able to communicate via the communication line W5 and the hub 4.

The lighting controller 21 is connected to the communication interface 3 via the communication line W2, and the communication interface 3 is connected to the transmission adapter 6 via the communication line W3. The transmission adapter 6 is configured to be able to communicate with at least one lighting fixture L1. The transmission adapter 6 is a communication adapter that communicates with the lighting fixture L1 by wired communication via the communication line W4 or wireless communication using wireless signals, and performs protocol conversion of signals between the upper level (signal of the communication line W3) and the lower level (signal of the communication line W4 and wireless signal). The wired communication is, for example, wired communication via a twisted pair cable, a dedicated communication line, or a LAN (Local Area Network) cable. The wireless communication is, for example, wireless communication conforming to standards such as Wi-Fi (registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), or low-power wireless (specific low-power wireless) that does not require a license, or wireless communication such as infrared communication.

The hub 4 is connected to the center device 1, the communication interface 3, and the image sensor 5 via a communication line W5. The communication line W5 is a LAN (Local Area Network) cable, and signals are transmitted and received between the center device 1, the communication interface 3, the hub 4, and the image sensor 5 in accordance with EtherCAT (registered trademark) or Ethernet (registered trademark), etc. The hub 4 is preferably a switching hub with a PoE (Power over Ethernet) power supply function.

The image sensor 5 is connected to the communication interface 3 via the communication line W6, and signals are transmitted and received between the image sensor 5 and the communication interface 3 by a time division multiplexing transmission method. As a communication protocol for the time division multiplexing transmission method, for example, the NMAST (registered trademark) communication standard can be adopted. The image sensor 5 and the communication interface 3 send a bipolar (±24V) time division multiplexing signal as a transmission signal to the communication line W6, and data is transmitted by pulse width modulation.

The image sensor 5 is further connected to a communication line W5, and transmits and receives signals between it and the hub 4 in accordance with EtherCat or Ethernet. In other words, the center device 1 and the image sensor 5 are configured to be able to communicate with each other via the hub 4. Furthermore, if importance is placed on real-time communication, it is preferable to use EtherCat.

In this embodiment, multiple image sensors 5 and multiple lighting fixtures L1 are arranged on the ceiling of the management space R1. As shown in FIG. 2, the management space R1 is divided into multiple spaces Ra1, Ra2, ... when viewed from above. Note that when there is no need to distinguish between the spaces Ra1, Ra2, ..., they are referred to as spaces Ra.

Each of the multiple image sensors 5 captures an image of a corresponding space Ra among the multiple spaces Ra1, Ra2, ... and generates data of the captured image. Then, the image sensor 5 judges the state of the person in the space Ra based on the captured image, and outputs judgment information based on the judgment result. The image sensor 5 also outputs image information including the captured image.

Each of the multiple lighting fixtures L1 illuminates a corresponding space Ra among the multiple spaces Ra1, Ra2, .... The lighting controller 21 controls the lighting for each space Ra by controlling the state (on, off, and dimming level) of each of the multiple lighting fixtures L1 via the transmission adapter 6. Note that one or more lighting fixtures L1 may correspond to one space Ra.

The entry/exit controller 22 monitors and controls the entry/exit device L2. The entry/exit device L2 includes a card reader disposed at the entrance/exit of the managed space R1 (see FIG. 2) and a door drive device disposed at the entrance/exit of the managed space R1. The card reader reads the identification information of the person entering/exiting the managed space R1, and the entry/exit controller 22 determines whether or not to allow access to the managed space R1 based on the read identification information. The person entering/exiting the managed space R1 carries an IC card that stores their own identification information, and holds the IC card over the card reader when entering and leaving the managed space R1. The card reader reads the identification information from the IC card and passes the read identification information to the entry/exit controller 22. The entry/exit controller 22 has a database that stores in advance the identification information of people permitted to access the managed space R1, and checks the read identification information against the database to determine whether or not to allow entry and exit. If the entry and exit controller 22 permits entry and exit, it unlocks the electric lock on the door, and if the door is an automatic door, it controls the automatic door to open. If the entry and exit controller 22 does not permit entry and exit, it keeps the electric lock in the locked state. The entry and exit controller 22 also outputs entry and exit information to the center device 1 that indicates the reading status of the identification information of people entering and exiting the managed space R1.

The air conditioning controller 23 adjusts the air conditioning environment of the managed space R1 by controlling an air conditioning device (not shown) installed in the managed space R1.

The center device 1 receives image information and determination information from the multiple image sensors 5, and manages people in the managed space R1 based on the image information and determination information. The center device 1 communicates with the lighting controller 21, grasps the status of the lighting fixtures L1, and manages the lighting environment of the managed space R1. The center device 1 communicates with the air conditioning controller 23, grasps the status of the air conditioning equipment, and manages the air conditioning environment in the managed space R1. Note that other controllers, such as an elevator controller that controls the elevation and descent of an elevator, may be connected to the communication line W1, and the center device 1 may manage other environments of the managed space R1 by communicating with the other controllers.

(2) Image Sensor FIG. 3 shows the configuration of the image sensor 5. The image sensor 5 includes an imaging unit 5a, a determination unit 5c, a first communication unit 5d, and a second communication unit 5e. The image sensor 5 preferably further includes a pre-processing unit 5b. The pre-processing unit 5b preferably includes a first processing unit 51 and a second processing unit 52. The image sensor 5 preferably further includes at least one of an output setting unit 5f, a third communication unit 5g, a processing area setting unit 5h, a determination area setting unit 5i, an image area setting unit 5j, and an alarm unit 5k. In this embodiment, the image sensor 5 includes all of the pre-processing unit 5b, the output setting unit 5f, the third communication unit 5g, the processing area setting unit 5h, the determination area setting unit 5i, the image area setting unit 5j, and the alarm unit 5k.

The image sensor 5 has, for example, a microcomputer having a processor and a memory. The processor executes a program stored in the memory, causing the microcomputer to function as a preprocessing unit 5b, a judgment unit 5c, a first communication unit 5d, a second communication unit 5e, an output setting unit 5f, a third communication unit 5g, a processing area setting unit 5h, a judgment area setting unit 5i, and an image area setting unit 5j. The program executed by the processor is pre-recorded in the memory of the microcomputer here, but may be provided by being recorded on a non-transitory recording medium such as a memory card, or may be provided via a telecommunications line such as the Internet.

The imaging unit 5a has a two-dimensional image sensor such as a CCD (Charge Coupled Devices) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. The imaging unit 5a uses a lens to form an image of light from a subject on the imaging surface (light receiving surface) of the imaging unit 5a and converts the light from the subject into an electrical signal (imaging signal). Data of the image (captured image) captured by the imaging unit 5a is output from the imaging unit 5a as an imaging signal. The imaging unit 5a is configured to generate a digital imaging signal, for example in YUV format, and output the digital imaging signal.

The pre-processing unit 5b (first processing unit 51 and second processing unit 52) receives the imaging signal and performs image processing on the captured image.

The first processing unit 51 performs a first image processing on the captured image that is suitable for the determination processing by the determination unit 5c described below, and passes the captured image that has been subjected to the first image processing to the determination unit 5c. As the first image processing, the first processing unit 51 performs, for example, edge detection processing and filtering processing, and performs edge enhancement and noise removal on the captured image.

The second processing unit 52 performs second image processing on the captured image so that the captured image is easy to view for the user, and passes the captured image after the second image processing to the second communication unit 5e. As the second image processing, the second processing unit 52 performs, for example, filtering, smoothing, and gray-scale conversion processing, and performs noise removal, smoothing, brightness correction, contrast correction, and color correction on the captured image.

The determination unit 5c determines the state of people in the space Ra based on the captured image. In this embodiment, the determination unit 5c uses the captured image that has been subjected to the first image processing by the first processing unit 51. The state of people determined by the determination unit 5c includes the presence or absence of people in the space Ra, the number of people (number of people) present in the space Ra, and the position coordinates of people present in the space Ra.

For example, the determination unit 5c can determine whether an object in the captured image is a person (the presence or absence of a person in the space Ra) by performing pattern recognition or template matching on the captured image and extracting the characteristics of the object in the captured image. If a person is present in the space Ra, the determination unit 5c counts the number of people (the number of objects recognized as people). In addition, the determination unit 5c determines the position of the person (the position of the object recognized as a person) in a planar view looking downward from the ceiling in two-dimensional coordinates as the position coordinates of the person present in the space Ra.

The first communication unit 5d outputs judgment information based on the judgment result of the judgment unit 5c. The output port of the first communication unit 5d is connected to the communication line W6. The judgment information is transmitted to the lighting controller 21 via the communication line W6 and the communication interface 3. The judgment information is also transmitted to the center device 1 via the communication line W6, the communication interface 3, the communication line W5, the hub 4, and the communication line W5. The judgment information includes data such as the presence or absence of people in the space Ra, the number of people present in the space Ra, and the position coordinates of people present in the space Ra.

The second communication unit 5e outputs the captured image data that has been subjected to the second image processing by the second processing unit 52 as image information. The output port of the second communication unit 5e is connected to the communication line W5. The image information is transmitted to the center device 1 via the communication line W5, the hub 4, and the communication line W5.

In this embodiment, the first communication unit 5d sends judgment information to the communication line W6. Meanwhile, the second communication unit 5e sends image information to the communication line W5. Then, when the timing at which the first communication unit 5d outputs the judgment information overlaps with the timing at which the second communication unit 5e outputs the image information, the first communication unit 5d and the second communication unit 5e output the judgment information and the image information simultaneously, respectively. In this embodiment, since the transmission path of the judgment information output by the first communication unit 5d and the transmission path of the image information output by the second communication unit 5e are different from each other, even if the judgment information and the image information are output simultaneously, the decrease in the transmission capacity of the judgment information and the transmission capacity of the image information is suppressed. Therefore, the image sensor 5 can maintain the transmission capacity of the image information even when outputting the judgment information.

The output setting unit 5f sets whether or not the second communication unit 5e is to output image information. Specifically, the output setting unit 5f outputs an output control signal to the second communication unit 5e to notify the second communication unit 5e of whether or not the image information is to be output. The second communication unit 5e can recognize whether or not the output of image information is permitted by the output control signal. If the output of image information is permitted, the second communication unit 5e outputs the image information. If the output of image information is not permitted, the second communication unit 5e does not output the image information.

The output setting unit 5f may be, for example, an operation switch provided on the image sensor 5. In this case, the user operates the operation switch to set whether or not to output image information.

The output setting unit 5f may set whether or not to output image information based on output setting information received by the third communication unit 5g. The output setting information is information related to the settings of the output setting unit 5f, and is transmitted from an external setting device to the third communication unit 5g by wireless or wired signals. The setting device is at least one of the center device 1, a personal computer, a smartphone, a tablet terminal, and a dedicated terminal. The wireless signal is, for example, a wireless signal conforming to standards such as Wi-Fi, Bluetooth, ZigBee, or a specific low-power radio, or an infrared signal.

The image sensor 5 can therefore set whether or not to output image information, taking into consideration the privacy of people in the space Ra. In other words, the image sensor 5 can control the output of image information so that the center device 1 outputs image information only when necessary, and does not output image information when the privacy of people in the space Ra is to be respected.

In addition, one of the first communication unit 5d and the second communication unit 5e may also serve as the function of the third communication unit 5g. In this case, it is preferable that the first communication unit 5d also serves as the function of the third communication unit 5g. By having the first communication unit 5d also serve as the function of the third communication unit 5g, the communication unit that outputs image information (the second communication unit 5e) and the communication unit that receives output setting information can be configured separately. A configuration in which the first communication unit 5d also serves as the function of the third communication unit 5g is preferable from the perspective of improving security compared to a configuration in which the second communication unit 5e also serves as the function of the third communication unit 5g.

The processing area setting unit 5h divides the captured image into multiple processing areas. The pre-processing unit 5b (first processing unit 51 and second processing unit 52) performs image processing (first image processing and second image processing) on each of the multiple processing areas.

Figure 4 shows a captured image G1 captured in a space Ra that is part of an office. The processing area setting unit 5h divides the captured image G1 into three processing areas G11 to G13. The two rectangular processing areas G11 and G12 are areas that show desks and the areas around the desks. The processing area G13 is an area that shows a passageway. The processing area setting unit 5h sets the processing areas G11 to G13 based on processing area setting information received by the third communication unit 5g, for example. The processing area setting information is information related to the setting of the processing areas G11 to G13, and is transmitted from an external setting device to the third communication unit 5g by wireless or wired signals. The processing area setting unit 5h may also perform pattern recognition or template matching on the captured image G1 to recognize the desks and passageways shown in the captured image G1 and automatically set the processing areas G11 to G13.

It is preferable that the processing area setting unit 5h has a function of setting the contents of the image processing to be applied to each of the multiple processing areas G11 to G13. The pre-processing unit 5b (first processing unit 51 and second processing unit 52) applies the image processing (first image processing and second image processing) set by the processing area setting unit 5h to each of the processing areas G11 to G13. For example, the processing area setting unit 5h sets the contents of the second image processing to be applied by the second processing unit 52 to the processing areas G11 to G13 individually. For the processing areas G11 and G12 in which the desk and its surroundings are reflected, the processing area setting unit 5h sets the contents of the second image processing, such as smoothing processing and grayscale conversion processing, so that the processing areas G11 and G12 are blurred and the desk and its surroundings are not clearly reflected. The processing area setting unit 5h sets the contents of the second image processing, such as filter processing and gray-scale conversion processing, for the processing area G13 in which the passage is shown so that the processing area G13 is shown clearly. The processing area setting unit 5h can also set the contents of the first image processing applied by the first processing unit 51 individually for each of the processing areas G11 to G13.

As described above, the image sensor 5 can set the image processing contents individually for each of the processing areas G11 to G13 of the captured image G1. Therefore, the image sensor 5 can process the captured image so that processing areas showing confidential information, such as the area around a desk (such as a personal computer screen and documents) or processing areas showing personal information, are not clearly visible. In other words, the image sensor 5 can generate image information and judgment information that take into account people's privacy for each of the processing areas G11 to G13.

It is preferable that the processing area setting unit 5h has a function of setting whether or not image processing is performed for each of the multiple processing areas G11 to G13 (whether or not image processing is possible). The pre-processing unit 5b (first processing unit 51 and second processing unit 52) performs image processing (first image processing and second image processing) on the processing areas set by the processing area setting unit 5h to perform image processing among the processing areas G11 to G13. For example, the processing area setting unit 5h sets whether or not second image processing by the second processing unit 52 is possible for each of the processing areas G11 to G13 individually. The processing area setting unit 5h sets "second image processing: possible" for the processing areas G11 and G12, which show the desk and its surroundings, so that the processing areas G11 and G12 are blurred and the desk and its surroundings are not clearly shown. The processing area setting unit 5h sets "second image processing: no" for the processing area G13 in which the passage is reflected so that the processing area G13 is clearly reflected. The processing area setting unit 5h can also set whether or not the first image processing performed by the first processing unit 51 is to be performed for each of the processing areas G11 to G13.

As described above, the image sensor 5 can individually set whether or not to process images for each of the processing areas G11 to G13 of the captured image G1. Therefore, the image sensor 5 can process the captured image so that processing areas showing confidential information, such as the area around a desk (such as a personal computer screen and documents) or processing areas showing personal information, are not clearly visible. In other words, the image sensor 5 can generate image information and judgment information that take into account people's privacy for each of the processing areas G11 to G13.

It is preferable that the second processing unit 52 reflects the result of the first image processing on the captured image subjected to the second image processing. The first processing unit 51 performs, for example, edge detection processing as the first image processing, and performs edge enhancement on the captured image. Therefore, the second processing unit 52 superimposes the edge generated by the first image processing on the captured image subjected to the second image processing. In addition, when the first processing unit 51 generates a heat map representing the temperature distribution of the captured image as the first image processing, the second processing unit 52 may superimpose the heat map on the captured image subjected to the second image processing. Therefore, the image sensor 5 can include the result of the first image processing in the image information.

It is preferable that the judgment area setting unit 5i sets a part of the captured image to be judged by the judgment unit 5c as the judgment area. The judgment unit 5c receives the captured image to which the first image processing has been applied from the first processing unit 51. The judgment area setting unit 5i sets a part of the captured image received by the judgment unit 5c as the judgment area. The judgment unit 5c judges the state of a person only for the judgment area out of the entire region of the captured image. For example, the judgment unit 5c judges whether an object in the judgment area is a person (presence or absence of a person in the judgment area), the number of people in the judgment area, and the position of the person in the judgment area. The judgment area setting unit 5i sets the judgment area based on, for example, the judgment area setting information received by the third communication unit 5g. The judgment area setting information is information regarding the setting of the judgment area, and is transmitted from an external setting device to the third communication unit 5g by a wireless signal or a wired signal. Therefore, the image sensor 5 can judge the state of a person only for the necessary area in the space Ra. In addition, the judgment area setting unit 5i may set a part of the captured image included in the judgment information as a judgment area, and the first communication unit 5d may output the judgment information corresponding to the judgment area.

It is preferable that the image area setting unit 5j sets the area of the captured image included in the image information as the image area. The second communication unit 5e receives the captured image that has been subjected to image processing from the second processing unit 52. The image area setting unit 5j sets a part of the captured image received by the second processing unit 52 as the image area. The second communication unit 5e generates image information including only the determination area of the captured image. The image area setting unit 5j sets the image area based on, for example, the image area setting information received by the third communication unit 5g. The image area setting information is information related to the setting of the image area, and is transmitted from an external setting device to the third communication unit 5g by a wireless signal or a wired signal. Therefore, the image sensor 5 can output image information only for the necessary area of the space Ra.

At least one of the output of the judgment information by the first communication unit 5d and the output of the image information by the second communication unit 5e is preferably executed when a predetermined event occurs outside. The predetermined event is an entry/exit event in which a person enters or leaves the managed space R1, or a log update event in which the center device 1 updates the management log of the managed space R1. When either event occurs, an event signal is transmitted from an external setting device to the third communication unit 5g. When the third communication unit 5g receives the event signal, the first communication unit 5d outputs judgment information based on the captured image. Also, when the third communication unit 5g receives the event signal, the second communication unit 5e outputs image information for a predetermined period of time. Therefore, the image sensor 5 can control the output of each piece of information so as not to output at least one of the image information and the judgment information when respecting the privacy of people in the space Ra.

The notification unit 5k preferably notifies information on at least one of the settings of the determination unit 5c, the first communication unit 5d, and the second communication unit 5e. The notification unit 5k has a light-emitting element such as an LED, and a drive circuit for driving the light-emitting element. The notification unit 5k notifies, for example, at least one of the setting information of the determination unit 5c, the first communication unit 5d, and the second communication unit 5e by turning on, blinking, or changing the light-emitting color of the light-emitting element. In this case, the setting status of the determination unit 5c is the setting information of the determination area set by the determination area setting unit 5i. The setting status of the first communication unit 5d is the setting information of the determination area set by the determination area setting unit 5i. The setting status of the second communication unit 5e is the setting information of the image area set by the image area setting unit 5j. Furthermore, the notification unit 5k may notify information on the setting of the pre-processing unit 5b. In this case, the information on the pre-processing unit 5b is the setting information of the processing area set by the processing area setting unit 5h. Therefore, the image sensor 5 can notify the user of the setting information.

The notification unit 5k may also have a display device (such as a liquid crystal display or an organic EL display). In this case, the notification unit 5k causes the display device to display the above-mentioned setting information represented by characters, symbols, signs, etc.

The output method of the image sensor 5 of this embodiment is shown in the flowchart of FIG. 5. The output method includes an imaging step S1, a determination step S2, and a communication step S3. In the imaging step S1, the imaging unit 5a captures an image of the space Ra and generates data of the captured image. In the determination step S2, the determination unit 5c determines the state of the person in the space Ra based on the captured image and outputs determination information based on the determination result. In the communication step S3, a first communication step S31 in which the first communication unit 5d outputs determination information and a second communication step S32 in which the first communication unit 5d outputs image information including the captured image are performed. In the present disclosure, the first communication step S31 and the second communication step S32 can be performed simultaneously.

(3) Center Device The center device 1 receives the above-mentioned determination information and image information from the multiple image sensors 5 in the managed space R1. The center device 1 also receives entry/exit information indicating the reading status of the identification information of people entering and leaving the managed space R1 from the entry/exit controller 22. The center device 1 can grasp the presence/absence, number of people, and position of people in the managed space R1 based on each determination information received from the multiple image sensors 5. The center device 1 also tracks the movement of people in the managed space R1 based on each image information and entry/exit information received from the multiple image sensors 5. Therefore, the center device 1 can identify people in the managed space R1 and manage or monitor the movement of people. Furthermore, the center device 1 can set various parameters related to control for the lighting controller 21, the air conditioning controller 23, and other controllers.

(4) Controller The lighting controller 21 receives the above-mentioned determination information from the multiple image sensors 5 in the managed space R1. The lighting controller 21 can detect the presence or absence of people, the number of people, and the positions of people in the managed space R1 based on each determination information received from the multiple image sensors 5. Therefore, the lighting controller 21 can individually control the multiple lighting fixtures L1 in the managed space R1 based on the presence or absence of people, the number of people, and the positions of people in the managed space R1. For example, the lighting controller 21 turns on only the lighting fixtures L1 in the space Ra in which people exist in the managed space R1. Furthermore, the lighting controller 21 increases the light output of the lighting fixtures L1 as the number of people in the space Ra increases. Furthermore, the lighting controller 21 determines the lighting fixtures L1 to be turned on in the space Ra according to the positions of people in the space Ra. Therefore, the lighting controller 21 can achieve both the efficiency of lighting control of the managed space R1 and the comfort of people.

Other controllers, such as the air conditioning controller 23, like the lighting controller 21, can also individually control multiple devices in the managed space R1 based on the presence or absence, number of people, and positions of people in the managed space R1.

(Modification)
The above embodiment is merely one of various embodiments of the present disclosure. The above embodiment can be modified in various ways depending on the design, etc., as long as the object of the present disclosure can be achieved. In addition, a function similar to that of the image sensor 5 may be embodied in an output method of an image sensor, a computer program, or a non-transitory recording medium on which a program is recorded, etc.

As a modified example, the second communication unit 5e may output other information, such as control information, other than image information.

The image sensor 5 or the image sensor output method in the present disclosure includes a computer system. The computer system has a processor and a memory as hardware. The processor executes a program recorded in the memory of the computer system to realize the function of the image sensor 5 or the image sensor output method in the present disclosure. The program may be pre-recorded in the memory of the computer system, or may be provided through an electric communication line. The program may also be recorded and provided in a non-transitory recording medium such as a memory card, an optical disk, or a hard disk drive that can be read by the computer system. The processor of the computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large scale integrated circuit (LSI). The multiple electronic circuits may be integrated in one chip or distributed across multiple chips. The multiple chips may be integrated in one device or distributed across multiple devices.

The computer system may be a system composed of one or more computers. For example, at least some of the functions of the image sensor 5 may be realized by the cloud (cloud computing).

(summary)
As described above, the image sensor (5) of the first aspect includes an imaging unit (5a), a determination unit (5c), a first communication unit (5d), and a second communication unit (5e). The imaging unit (5a) captures an image of a predetermined space (Ra) and generates data of the captured image. The determination unit (5c) determines the state of a person in the space (Ra) based on the captured image. The first communication unit (5d) outputs determination information based on the determination result of the determination unit (5c). The second communication unit (5e) outputs image information including the captured image. The first communication unit (5d) and the second communication unit (5e) can simultaneously output at least the determination information and the image information.

The image sensor (5) described above can maintain the transmission capacity of image information even when outputting judgment information.

In the first embodiment, the image sensor (5) of the second aspect preferably further includes an output setting unit (5f) that sets whether or not image information is to be output by the second communication unit (5e).

The image sensor (5) described above can be set to output or not output image information, taking into consideration the privacy of people in the space (Ra).

In the third aspect, the image sensor (5) in the second aspect preferably further includes a third communication unit (5g) that receives output setting information related to the settings of the output setting unit (5f).

The image sensor (5) described above can be set to output or not output image information, taking into consideration the privacy of people in the space (Ra).

The image sensor (5) of the fourth aspect, in any one of the first to third aspects, preferably further includes a pre-processing unit (5b) that applies image processing to the captured image.

The image sensor (5) described above can improve the accuracy of the determination process by the determination unit (5c) and clarify the image information.

In the image sensor (5) of the fifth aspect, in the fourth aspect, it is preferable that the pre-processing unit (5b) has a first processing unit (51) and a second processing unit (52). The first processing unit (51) passes the captured image that has been subjected to the first image processing as the image processing to the determination unit (5c). The second processing unit (52) passes the captured image that has been subjected to the second image processing as the image processing to the second communication unit (5e).

The image sensor (5) described above can improve the accuracy of the determination process by the determination unit (5c) and clarify the image information.

In the sixth aspect of the image sensor (5), in the fifth aspect, it is preferable that the second processing unit (52) reflects the results of the first image processing in the captured image that has been subjected to the second image processing.

The image sensor (5) described above can include the results of the first image processing in the image information.

The seventh aspect of the image sensor (5) is preferably any one of the fourth to sixth aspects, further comprising a processing area setting unit (5h) that divides the captured image (G1) into a plurality of processing areas (G11 to G13). The pre-processing unit (5b) performs image processing on each of the plurality of processing areas (G11 to G13).

The image sensor (5) described above can generate image information and judgment information that takes into account people's privacy for each processing area (G11 to G13).

In the image sensor (5) of the eighth aspect, in the seventh aspect, it is preferable that the processing area setting unit (5h) has a function of setting the content of the image processing to be performed on each of the multiple processing areas (G11 to G13).

The image sensor (5) described above can generate image information and judgment information that takes into account people's privacy for each processing area (G11 to G13).

In the image sensor (5) of the ninth aspect, in the seventh or eighth aspect, it is preferable that the processing area setting unit (5h) has a function of setting whether or not to perform image processing for each of the multiple processing areas (G11 to G13).

The image sensor (5) described above can generate image information and judgment information that takes into account people's privacy for each processing area (G11 to G13).

It is preferable that the image sensor (5) of the tenth aspect further includes a judgment area setting unit (5i) that sets at least one of a part of the captured image to be judged by the judgment unit (5c) and a part of the captured image included in the judgment information as a judgment area in any one of the first to ninth aspects.

The image sensor (5) described above can determine the state of people only in the necessary areas of the space (Ra).

It is preferable that the image sensor (5) of the eleventh aspect further includes an image area setting unit (5j) that sets a part of the captured image included in the image information as an image area in any one of the first to tenth aspects.

The image sensor (5) described above can output image information only for the necessary areas of the space (Ra).

In the image sensor (5) of the 12th aspect, in any one of the first to 11th aspects, it is preferable that at least one of the output of the determination information by the first communication unit (5d) and the output of the image information by the second communication unit (5e) is executed when a specified event occurs externally.

The image sensor (5) described above can control the output of each piece of information so as not to output at least one of the image information and the judgment information when respecting the privacy of people in the space (Ra).

It is preferable that the image sensor (5) of the thirteenth aspect further includes a notification unit (5k) that notifies information regarding at least one setting of the determination unit (5c), the first communication unit (5d), and the second communication unit (5e) in any one of the first to twelfth aspects.

The image sensor (5) mentioned above can notify the user of information regarding the settings.

The image sensor system (20) of the 14th aspect includes an image sensor (5) of any one of the first to 13th aspects, and a receiving device (1) that receives at least one of the determination information and the image information.

The image sensor system (20) described above can maintain the transmission capacity of image information even when outputting judgment information.

The image sensor output method of the fifteenth aspect includes an imaging step (S1), a determination step (S2), a first communication step (S31), and a second communication step (S32). In the imaging step (S1), a predetermined space (Ra) is imaged to generate data of the captured image. In the determination step (S2), the state of a person in the space (Ra) is determined based on the captured image. In the first communication step (S31), determination information based on the determination result of the determination step (S2) is output. In the second communication step (S32), image information including the captured image is output. And, the first communication step (S31) and the second communication step (S32) can be performed simultaneously.

The image sensor output method described above can maintain the transmission capacity of image information even when outputting judgment information.

The program of the 16th aspect causes a computer to execute the image sensor output method of the 15th aspect.

The above-mentioned program can maintain the transmission capacity of image information even when outputting judgment information.

1. Center device (receiving device)
5 Image sensor 5a Imaging section 5b Pre-processing section 5c Determination section 5d First communication section 5e Second communication section 5f Output setting section 5g Third communication section 5h Processing area setting section 5i Determination area setting section 5j Image area setting section 5k Notification section 51 First processing section 52 Second processing section G1 Captured image G11 to G13 Processing area Ra Space S1 Imaging step S2 Determination step S31 First communication step S32 Second communication step

Claims (16)

an imaging unit that captures an image of a predetermined space and generates captured image data;
A determination unit that determines a state of a person in the space based on the captured image;
a first communication unit that outputs determination information based on a determination result of the determination unit;
a second communication unit that outputs image information including the captured image and communicates with the first communication unit using a communication method different from that of the first communication unit ,
an output destination of the first communication unit and an output destination of the second communication unit, a communication path between the first communication unit and the output destination of the first communication unit, and a communication path between the second communication unit and the output destination of the second communication unit are different from each other;
The image sensor, wherein the first communication unit and the second communication unit are capable of simultaneously outputting at least the determination information and the image information. The image sensor of claim 1 , further comprising an output setting unit that sets whether or not the second communication unit is to output the image information. The image sensor of claim 2 , further comprising a third communication unit that receives output setting information related to settings of the output setting unit. The image sensor according to claim 1 , further comprising a pre-processing unit that performs image processing on the captured image. The pre-treatment unit includes:
a first processing unit that transfers the captured image that has been subjected to a first image processing as the image processing to the determination unit;
The image sensor according to claim 4 , further comprising: a second processing unit configured to transfer the captured image having undergone second image processing as the image processing to the second communication unit. The image sensor according to claim 5 , wherein the second processing unit reflects a result of the first image processing in the captured image on which the second image processing has been performed. A processing area setting unit that divides the captured image into a plurality of processing areas,
The image sensor according to claim 4 , wherein the pre-processing unit performs the image processing on each of the plurality of processing areas. The image sensor according to claim 7 , wherein the processing area setting unit has a function of setting the content of the image processing to be performed on each of the plurality of processing areas. The image sensor according to claim 7 or 8, wherein the processing area setting unit has a function of setting whether or not the image processing is to be performed for each of the plurality of processing areas. The image sensor according to claim 1 , further comprising a determination area setting unit that sets at least one of a part of the captured image to be determined by the determination unit and a part of the captured image included in the determination information as a determination area. The image sensor according to claim 1 , further comprising an image area setting unit that sets a part of the captured image included in the image information as an image area. The image sensor according to claim 1 , wherein at least one of the output of the determination information by the first communication unit and the output of the image information by the second communication unit is executed when a predetermined event occurs externally. The image sensor according to claim 1 , further comprising: a notification unit configured to notify information related to at least one setting of the determination unit, the first communication unit, and the second communication unit. An image sensor according to any one of claims 1 to 13;
a receiving device for receiving at least one of the determination information and the image information. An imaging step of imaging a predetermined space and generating data of the captured image;
A determination step of determining a state of a person in the space based on the captured image;
a first communication step in which a first communication unit outputs determination information based on a determination result of the determination step;
a second communication step in which a second communication unit outputs image information including the captured image and communicates using a communication method different from that in the first communication step ;
an output destination of the first communication unit and an output destination of the second communication unit, a communication path between the first communication unit and the output destination of the first communication unit, and a communication path between the second communication unit and the output destination of the second communication unit are different from each other;
The image sensor output method, wherein the first communication step and the second communication step can be performed simultaneously. A program for causing a computer to execute the image sensor output method described in claim 15.

Images