WO2023276605A1

WO2023276605A1 - Lighting control system, lighting control method, and program

Info

Publication number: WO2023276605A1
Application number: PCT/JP2022/023366
Authority: WO
Inventors: 千人浦; 仁吉澤; 達雄古賀
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2021-06-29
Filing date: 2022-06-09
Publication date: 2023-01-05

Abstract

A lighting control system (10) is provided with an acquisition unit (34) for acquiring image data of a space where a person is present, an identification unit (35) for identifying skeletal coordinates of the person on the basis of the acquired image data, an estimation unit (36) for estimating movement of the person on the basis of time-series data of the identified skeletal coordinates, and a control unit (38) for controlling lighting in the space.

Description

LIGHTING CONTROL SYSTEM, LIGHTING CONTROL METHOD AND PROGRAM

The present invention relates to a lighting control system, lighting control method and program.

Various technologies have been proposed for controlling lighting in a space. Patent Literature 1 discloses a technique for controlling lighting based on human movement.

JP 2008-016289 A

The present invention provides a lighting control system and the like that can control lighting according to various movements of people.

A lighting control system according to an aspect of the present invention includes an acquisition unit that acquires image data of a space in which a person exists; an identification unit that identifies the skeletal coordinates of the person based on the obtained image data; an estimation unit for estimating the motion of the person based on the time-series data of the skeletal coordinates; and a control unit for controlling lighting in the space based on the estimated motion of the person.

A lighting control method according to an aspect of the present invention includes an acquisition step of acquiring image data of a space in which a person exists; an specifying step of specifying the skeletal coordinates of the person based on the acquired image data; an estimation step of estimating the movement of the person based on the time-series data of the skeletal coordinates; and a control step of controlling lighting in the space based on the estimated movement of the person.

A program according to one aspect of the present invention is a program for causing a computer to execute the lighting control method.

The lighting control system of the present invention can control lighting according to various human movements.

1 is a block diagram illustrating an example of a functional configuration of a lighting control system according to an embodiment; FIG. FIG. 2 is a flowchart illustrating an example of the operation of the lighting control system according to the embodiment; FIG. 3 is a diagram conceptually showing identification of a two-dimensional human skeleton model. FIG. 4 is a diagram conceptually showing skeletal coordinate estimation. FIG. 5 is a diagram showing an example of specific operation. FIG. 6 is a diagram showing an example of specific operation.

Hereinafter, embodiments will be described with reference to the drawings. It should be noted that the embodiments described below are all comprehensive or specific examples. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are examples and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements not described in independent claims will be described as optional constituent elements.

It should be noted that each figure is a schematic diagram and is not necessarily strictly illustrated. Moreover, in each figure, the same code|symbol is attached|subjected with respect to substantially the same structure, and the overlapping description may be abbreviate|omitted or simplified.

(Embodiment)
[composition]
First, the configuration of the lighting control system according to the embodiment will be described.

FIG. 1 is a block diagram showing an example of the functional configuration of the lighting control system 10 according to the embodiment.

The lighting control system 10 is a system that acquires spatial image data output by the camera 20 and controls lighting in the space based on the acquired image data. The space is, for example, an office space, but it may also be a space in a commercial facility or a space in other facilities such as a space in a house. As shown in FIG. 1 , lighting control system 10 includes camera 20 , control device 30 , and lighting device 40 .

The camera 20 is installed, for example, on the ceiling or wall of a space, and captures an image (moving image composed of a plurality of images) including a person existing in the space as a subject. The camera 20 also transmits image data of the captured image to the control device 30 . The camera 20 may be a camera using a CMOS (Complementary Metal Oxide Semiconductor) image sensor, or may be a camera using a CCD (Charge Coupled Device) image sensor.

Also, the camera 20 may be a camera using an image sensor capable of detecting infrared rays (infrared light). That is, camera 20 may be an infrared camera. This allows the camera 20 to capture an image (infrared image) even when the space is dark.

Note that the lighting control system 10 may include two or more cameras 20 . For example, when an image captured by one camera 20 shows only a part of the human body, it is difficult to estimate the skeletal coordinates (described later) of the entire human body. In such a case, the estimating unit 36 can generate a three-dimensional skeletal model of the entire human body by estimating skeletal coordinates from each of the two images captured by the two cameras 20 and synthesizing them.

The control device 30 receives image data from the camera 20 and controls the lighting device 40 based on the received image data. Lighting in the space is controlled by controlling the lighting device 40 . The control device 30 is, for example, a local controller (i.e., edge computer) installed in the same facility as the facility in which the space is provided, but is a server device (i.e., cloud computer) installed outside the facility. good too. The control device 30 includes a communication section 31 , an information processing section 32 and a storage section 33 .

The communication unit 31 is a communication module (communication circuit) for the control device 30 to communicate with the camera 20 and the lighting device 40. The communication unit 31 , for example, receives image data from the camera 20 and transmits control signals to the lighting device 40 . The communication performed by the communication unit 31 may be wireless communication or wired communication. The communication standard used for communication is also not particularly limited.

The information processing section 32 acquires the image data of the image received by the communication section 31, and performs information processing for controlling the lighting device 40 based on the acquired image data. The information processing section 32 is specifically realized by a processor or a microcomputer. The information processing section 32 includes an acquisition section 34 , an identification section 35 , an estimation section 36 , an identification section 37 and a control section 38 . The function of the acquisition unit 34, the identification unit 35, the estimation unit 36, the identification unit 37, and the control unit 38 is that the processor or microcomputer constituting the information processing unit 32 executes a computer program stored in the storage unit 33. realized by Details of the functions of the acquisition unit 34, the identification unit 35, the estimation unit 36, the identification unit 37, and the control unit 38 will be described later.

The storage unit 33 is a storage device that stores image data received by the communication unit 31, computer programs executed by the information processing unit 32, and the like. The storage unit 33 also stores a machine learning model, an estimation model, and the like, which will be described later. Specifically, the storage unit 33 is implemented by a semiconductor memory, HDD (Hard Disk Drive), or the like.

The lighting device 40 is installed on the ceiling of the space or the like to illuminate the space. For example, the brightness or color temperature of the light output from the lighting device 40 is adjusted based on the control signal transmitted from the control device 30 .

[Example of operation]
Next, an operation example of the lighting control system 10 will be described.

FIG. 2 is a flowchart showing an example of the operation of the lighting control system 10. FIG.

The communication unit 31 of the control device 30 receives the image data of the space image from the camera 20 (S11). The information processing section 32 stores the received image data in the storage section 33 (S12).

Next, the acquisition unit 34 acquires the image data received by the communication unit 31 and stored in the storage unit 33 (S13), and the specifying unit 35 determines the image data that appears in the image based on the acquired image data. A two-dimensional skeleton model of a person is specified (S14).

FIG. 3 is a diagram conceptually showing identification of a two-dimensional human skeleton model.

As shown in Fig. 3, the two-dimensional skeleton model is a model in which the joint positions (spheres) of a person in the image are connected by links (lines). An existing pose and skeleton identification algorithm is used to identify the two-dimensional skeleton model.

Next, the identifying unit 35 identifies skeleton coordinates (three-dimensional coordinate data of each joint) from the identified two-dimensional skeleton model (S15). The identifying unit 35 identifies skeletal coordinates using, for example, a machine learning model.

FIG. 4 is a diagram conceptually showing skeletal coordinate estimation.

A machine learning model is a learning model pre-constructed by machine learning using a two-dimensional skeletal model in which the skeletal coordinates of each joint are known as learning data and skeletal coordinates as training data. Such a machine learning model can take a two-dimensional skeleton model as an input and output its skeleton coordinates (in other words, a three-dimensional skeleton model). The specifying unit 35 can specify the time-series data of the skeleton coordinates by specifying the skeleton coordinates for each of a plurality of images (frames) forming the moving image.

Next, the estimation unit 36 estimates the movement of the person in the image from the specified time-series data of the skeletal coordinates (S16). The estimating unit 36 detects a specific motion of a person based on the estimation result (S17). For example, if the storage unit 33 stores a discriminative model that has learned the motion of a joint when a specific action is performed, the estimating unit 36 inputs time-series data of skeletal coordinates to the discriminative model to perform a specific motion. Motion can be detected. Here, an example of the specific operation will be described with reference to FIGS. 5 and 6. FIG.

　FIGS. 5 and 6 are diagrams showing an example of a specific operation.

For example, the specific action includes an action instructing to control lighting. For example, the specific action may be a gesture instructing to control lighting. FIG. 5 shows, as an example of the action of instructing to control the lighting, the action of raising the left hand after the right hand is raised. For example, by making the person aware in advance that the lighting can be brightened by raising the left hand after raising the right hand, when the person wants to brighten the lighting, he/she raises the right hand and then raises the left hand. It is possible to make a person perform the action of raising. Note that the action of raising the left hand after raising the right hand is an example of the action of instructing to brighten the lighting, and is not limited to this. In addition to the operation of instructing to brighten the lighting, the operation of instructing to control the lighting includes the operation of instructing to dim the lighting, the operation of instructing to increase the color temperature of the lighting, Alternatively, there is an operation of instructing to lower the color temperature of lighting. The time-series data of the skeleton coordinates are specified from the moving image including the person performing such actions as the subject, and the estimation unit 36 calculates the lighting conditions of the person reflected in the image from the specified time-series data of the skeleton coordinates. Detects an action directed to control the

Also, for example, specific actions include actions that people perform on a daily basis. Actions that people perform on a daily basis are, for example, not gestures that instruct to control lighting, but actions such as desk work, stretching during desk work, or conversation with people. FIG. 6 shows a motion of a person stretching during desk work as an example of a motion that a person performs on a daily basis. Time-series data of skeleton coordinates are identified from a moving image including a person performing such actions as a subject. Detect intentional actions.

Next, the identification unit 37 identifies a person based on the acquired image data (S18). For example, if the storage unit 33 stores a discriminant model that has learned individual features (for example, face, physique, or bone structure), the estimating unit 36 inputs image data of a person into the discriminant model. , can identify a person.

Next, the control unit 38 controls lighting in the space based on the estimated human movement (S19). Specifically, the controller 38 controls the lighting device 40 . The lighting device 40 is controlled by transmitting a control signal from the communication unit 31 to the lighting device 40 .

For example, when a specific action is detected, the control unit 38 controls lighting in the space with control details corresponding to the specific action.

For example, when an action instructing to control lighting is detected as a specific action, the control unit 38 controls the lighting in the space with control details corresponding to the action. For example, the control unit 38 brightens the lighting in the space when an operation instructing to brighten the lighting is detected, and dims the lighting in the space when an operation instructing to dim the lighting is detected. , when an operation instructing to increase the color temperature of the lighting is detected, the color temperature of the lighting in the space is increased, and when an operation instructing to decrease the color temperature of the lighting is detected, the lighting in the space lower the color temperature of the

Also, for example, when an action that a person performs on a daily basis is detected as a specific action, the control unit 38 controls the lighting in the space with the control content corresponding to the action. For example, the control part 38 brightens the brightness in the space when a person's deskwork motion is detected, and darkens the space brightness when a person's stretching motion is detected during the deskwork.

For example, when a plurality of lighting devices 40 are provided in the space, the control unit 38 controls the lighting device 40 around the person who performed the specific action, thereby controlling the lighting around the person in the space. may For example, when an action of conversing with a person is detected, the control unit 38 brightens the brightness around the person in the space.

The control unit 38 may also control lighting in the space based on the estimated human movement and the identified person. For example, since different people have different tastes in lighting control details, the control details corresponding to the specific action may be different for each identified person.

Also, the control unit 38 may further control lighting in the space based on the current time. For example, the brightness in a space can change according to the time of day due to the movement of the sun throughout the day. may brighten the lighting in the space.

The control unit 38 may also control lighting in the space based on the estimated human movement and the current time. For example, even if the same motion is detected, the content of lighting control may be changed depending on the time. Specifically, when it is detected that a person stretches while working at a desk in the morning, control to reduce the brightness in the space is not performed, and when it is detected that a person stretches while working at a desk in the afternoon. Also, control may be performed to darken the brightness in the space. For example, by dimming the brightness of the space during desk work, the person can take a break. This is because it may not be necessary to take a break. In this way, depending on the estimated movement of the person, the content of lighting control may be varied according to the current time.

The control unit 38 may also control lighting in the space based on the estimated movement of the person, the current time, and the identified person. For example, since people have different tastes in lighting control details depending on the time of day, the lighting control details depending on the time of day may be different for each identified person.

[Effects, etc.]
As described above, the lighting control system 10 includes the acquiring unit 34 that acquires image data of a space in which a person exists, the specifying unit 35 that specifies the skeletal coordinates of the person based on the acquired image data, and the specifying unit 35. An estimation unit 36 for estimating human motion based on the time-series data of skeletal coordinates, and a control unit 38 for controlling lighting in space based on the estimated human motion.

According to this, it is possible to estimate a person's fine movements based on the specified time-series data of the skeletal coordinates, so various movements of the person can be estimated. Therefore, lighting can be controlled according to various movements of a person.

Also, for example, the estimation unit 36 detects a specific action of a person by estimating the movement of the person, and when the specific action is detected, the control unit 38 adjusts the lighting in the space to correspond to the specific action. Control by control contents.

According to this, the lighting in the space can be controlled with different control details for each specific action.

Also, for example, specific actions include actions that people perform on a daily basis.

According to this, since the lighting can be controlled by the daily actions of the person, for example, the control contents corresponding to the daily actions of the person can be used without instructing the person to control the lighting. Lighting can be controlled automatically.

Also, for example, the specific action includes an action instructing to control lighting.

According to this, the lighting can be controlled by gestures.

Further, for example, the lighting control system 10 further includes an identification unit 37 that identifies a person based on the acquired image data, and the control unit 38 controls the estimated movement of the person and the identified person based on the , controls the lighting in the space.

According to this, even with the same movement, the contents of lighting control can be changed depending on the person.

Also, for example, the control unit 38 controls lighting in the space based on the estimated human movement and the current time.

According to this, the brightness of the lighting can be changed according to the current time. For example, depending on the estimated movement of a person, it is possible to vary lighting control details according to the current time.

Also, for example, the acquisition unit 34 acquires image data of an image captured by an infrared camera.

According to this, it is possible to identify the skeletal coordinates of a person even when the space is dark, based on the image data of the image taken by the infrared camera.

Further, the lighting control method executed by a computer such as the lighting control system 10 includes an acquisition step (S13 in FIG. 2) of acquiring image data of a space in which a person exists, and based on the acquired image data, A specifying step of specifying skeletal coordinates (S15 in FIG. 2), an estimating step of estimating a person's movement (S16 in FIG. 2) based on the specified time-series data of the skeletal coordinates, and an estimated human movement. and a control step (S19 in FIG. 2) of controlling the lighting in the space based on.

According to this, it is possible to provide a lighting control method that can control lighting according to various movements of a person.

(Other embodiments)
Although the lighting control system 10 and the lighting control method according to the embodiment have been described above, the present invention is not limited to the above embodiment.

Although the lighting control system is implemented by a plurality of devices in the above embodiment, it may be implemented as a single device. For example, the lighting control system may be implemented as a single device that corresponds to the controller. When the lighting control system is implemented by a plurality of devices, each component included in the lighting control system may be distributed to the plurality of devices in any way.

For example, in the above embodiment, an example in which the lighting control system 10 includes the identification unit 37 has been described, but the lighting control system 10 does not have to include the identification unit 37 . In this case, controller 38 may not control the lighting in the space based on the estimated human movement and the identified person.

For example, in the above embodiment, the control unit 38 controls the lighting in the space based on the current time, but the control unit 38 does not need to control the lighting in the space based on the current time. good too.

Further, in the above embodiment, the processing executed by a specific processing unit may be executed by another processing unit. In addition, the order of multiple processes may be changed, and multiple processes may be executed in parallel.

Also, in the above embodiments, each component may be realized by executing a software program suitable for each component. Each component may be realized by reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or processor.

Also, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form one circuit as a whole, or may be separate circuits. These circuits may be general-purpose circuits or dedicated circuits.

In addition, general or specific aspects of the present invention may be implemented in a system, apparatus, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM. Also, any combination of systems, devices, methods, integrated circuits, computer programs and recording media may be implemented. For example, the present invention may be realized as a program for causing a computer to execute the lighting control method of the above embodiments, or as a computer-readable non-temporary recording medium storing such a program. may be implemented.

In addition, forms obtained by applying various modifications to each embodiment that a person skilled in the art can think of, or realized by arbitrarily combining the constituent elements and functions of each embodiment without departing from the spirit of the present invention. Also included in the present invention.

At least the following matters are described in this specification.

(1) an acquisition unit that acquires image data of a space in which a person exists; an identification unit that identifies skeletal coordinates of the person based on the obtained image data; and time-series data of the identified skeletal coordinates. and a controller that controls lighting in the space based on the estimated movement of the person.

(2) The estimating unit detects a specific action of the person by estimating the movement of the person, and the control unit adjusts the lighting in the space to the specific action when the specific action is detected. The lighting control system according to (1), which is controlled by control contents corresponding to the operation.

(3) The lighting control system according to (2), wherein the specific action includes a daily action performed by the person.

(4) The lighting control system according to (2) or (3), wherein the specific operation includes an instruction to control lighting.

(5) Further, an identification unit that identifies the person based on the acquired image data, the control unit, based on the estimated movement of the person and the identified person, in the space The lighting control system according to any one of (1) to (4) for controlling lighting.

(6) The lighting control system according to any one of (1) to (5), wherein the control unit controls lighting in the space based on the estimated movement of the person and the current time.

(7) The lighting control system according to any one of (1) to (6), wherein the acquisition unit acquires the image data of an image captured by an infrared camera.

(8) an acquiring step of acquiring image data of a space in which a person exists; an identifying step of identifying the skeletal coordinates of the person based on the obtained image data; and time-series data of the identified skeletal coordinates. and a control step of controlling lighting in the space based on the estimated movement of the person.

(9) A program for causing a computer to execute the lighting control method described in (8).

10 lighting control system 34 acquisition unit 35 identification unit 36 estimation unit 37 identification unit 38 control unit

Claims

an acquisition unit that acquires image data of a space in which a person exists;
an identifying unit that identifies the skeletal coordinates of the person based on the acquired image data;
an estimating unit that estimates the movement of the person based on the specified time-series data of the skeletal coordinates;
A lighting control system, comprising: a controller that controls lighting in the space based on the estimated movement of the person.
The estimation unit detects a specific motion of the person by estimating the motion of the person,
The lighting control system according to claim 1, wherein when the specific action is detected, the control unit controls the lighting in the space with control details corresponding to the specific action.
The lighting control system according to claim 2, wherein the specific action includes a daily action performed by the person.
The lighting control system according to claim 2 or 3, wherein the specific action includes an action instructing to control lighting.
Furthermore, an identification unit that identifies the person based on the acquired image data,
2. The lighting control system of claim 1, wherein the controller controls lighting in the space based on the estimated movement of the person and the identified person.
The lighting control system according to claim 1, wherein the controller controls lighting in the space based on the estimated movement of the person and the current time.
The lighting control system according to claim 1, wherein the acquisition unit acquires the image data of an image captured by an infrared camera.
an acquisition step of acquiring image data of a space in which a person is present;
an identifying step of identifying the skeletal coordinates of the person based on the acquired image data;
an estimating step of estimating the movement of the person based on the specified time-series data of the skeletal coordinates;
and a control step of controlling lighting in the space based on the estimated movement of the person.
A program for causing a computer to execute the lighting control method according to claim 8.