WO2023286154A1 - Operation determination device and control method - Google Patents

Abstract

A signal acquisition unit (152) acquires a signal that has been transmitted from an elevator control device (200) and indicates whether or not a car (400) is in a closed state. An image acquisition unit (151) acquires an image that has been photographed by a camera (402) and shows the inside of the car (400). An extraction unit (153, 156) extracts person information from the acquired image. A calculation unit (154) calculates the number of passengers on the basis of the extracted person information. A determination unit (155) determines whether or not a specific action of a passenger has occurred on the basis of the extracted person information. The image acquisition unit (151) acquires images regularly if determining that the car (400) is in a closed state on the basis of the acquired signal. The determination unit (155) determines, if the calculated number of passengers is equal to or larger than one, whether or not the specific action has occurred on the basis of a difference between sets of person information extracted from the plurality of regularly acquired images.

Description

Motion determination device and control method

The present disclosure relates to a motion determination device and a control method for determining a passenger's motion in an elevator car.

Conventionally, devices have been developed that analyze images captured by cameras installed in elevator cars and determine passenger movements. For example, in Japanese Patent Laid-Open No. 2006-276969 (Patent Document 1), the direction and magnitude of movement of each point in the image are calculated from two images taken at different times, and based on the calculation result, the person in the car is detected. A technique for judging a rampage is disclosed.

JP 2006-276969 A

However, when trying to determine a specific action such as a person's rampage based on the difference between images, as in Patent Document 1, even actions unrelated to the specific action may be determined as the specific action. . For example, there is a problem that even a change in the background caused by opening and closing a car door, a person's movement caused by getting in and out of a car, or even a movement unrelated to a specific movement such as gymnastics or a golf swing is determined as a specific movement.

The present disclosure has been made to solve such problems, and its object is to provide a motion determination device and a control method that can accurately determine a specific motion of a passenger in an elevator car. It is to be.

A motion determination device according to the present disclosure is a motion determination device that determines the motion of a passenger in an elevator car. The motion determination device can communicate with an elevator controller that controls the elevator and a camera installed in the car. The motion determination device includes a signal acquisition section, an image acquisition section, an extraction section, a calculation section, and a determination section. The signal acquisition unit acquires a signal indicating whether or not the car is in the closed state, which is transmitted from the elevator control device. The image acquisition unit acquires an image of the interior of the car captured by the camera. The extraction unit extracts information about the person from the acquired image. The calculation unit calculates the number of passengers based on the extracted information about the person. The determination unit determines whether or not the specific motion of the passenger has occurred based on the extracted information about the person. The image acquisition unit periodically acquires an image when it is determined from the acquired signal that the car is in the closed state. When the calculated number of passengers is one or more, the determination unit determines whether or not the specific action has occurred based on the difference in the information regarding the person in the plurality of images acquired periodically.

A control method according to the present disclosure is a method of controlling a motion determination device that determines the motion of a passenger in an elevator car. The motion determination device can communicate with an elevator controller that controls the elevator and a camera installed in the car. The control method includes the step of obtaining a signal transmitted from the elevator control device indicating whether or not the car is in the closed state, and if the obtained signal determines that the car is in the closed state, A step of periodically acquiring a photographed image of the inside of the car; a step of extracting information about the person from the acquired image; a step of calculating the number of passengers based on the extracted information about the person; and determining, when the number of passengers is one or more, whether or not a particular action of the passenger has occurred based on the information about the person in the plurality of images acquired periodically.

According to the present disclosure, it is possible to accurately determine specific actions of passengers in the car of the elevator.

It is a figure showing an example of hardware constitutions of an elevator control system concerning a 1st embodiment. It is a figure which shows an example of the functional block diagram of the motion determination apparatus which concerns on 1st Embodiment. 4 is a flowchart of main processing executed by the motion determination device according to the first embodiment; FIG. 4 is a diagram for explaining determination of a violent motion according to the first embodiment; It is a figure which shows an example of the functional block diagram of the motion determination apparatus which concerns on 2nd Embodiment. 9 is a flowchart of main processing executed by the motion determination device according to the second embodiment; FIG. 11 is a diagram for explaining determination of a violent motion according to the second embodiment; It is a figure which shows an example of the functional block diagram of the motion determination apparatus which concerns on 3rd Embodiment. 10 is a flowchart of main processing executed by a motion determination device according to a third embodiment; FIG. 11 is a diagram for explaining determination of a violent motion according to the third embodiment; FIG. 14 is a diagram for explaining determination of a violent motion according to the fourth embodiment; FIG. 21 is a diagram for explaining determination of a violent motion according to the fifth embodiment; FIG. 14 is a flowchart of main processing executed by a motion determination device according to a sixth embodiment; FIG. FIG. 21 is a diagram for explaining determination of a violent motion according to the sixth embodiment;

Embodiments will be described below with reference to the drawings. In the following description, the same parts are given the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

[First embodiment]
First, an elevator control system 1 according to the first embodiment will be described. FIG. 1 is a diagram showing an example of the hardware configuration of an elevator control system according to the first embodiment.

The elevator control system 1 includes an elevator control device 200 , a motion determination device 100 , a monitoring device 300 , a car 400 , and an in-car device 401 and a camera 402 installed in the car 400 . Two cameras 402 may be installed in the car 400 . If there are multiple cars, a camera 402 is installed for each car.

The elevator control device 200 controls the elevator. Elevator controller 200 controls the operation of car 400 and in-car devices 401 . The in-car device 401 is a device including an operation panel provided with a door open/close button for opening and closing a door, a car call button for registering a destination floor, and the like.

The motion determination device 100 determines the motion of the passenger inside the elevator car 400 based on the image acquired from the camera 402 . Specifically, it is determined whether or not a "random motion", which will be described later, has occurred. Motion determination device 100 can communicate with elevator control device 200 and camera 402 installed in car 400 .

The monitoring device 300 can communicate with the motion determination device 100 . The monitoring device 300 is connected with an input device 322 and a display device 321 . The monitoring device 300 can receive the results output by the motion determination device 100 and display the results on the display device 321 . By looking at the display device 321, it is possible to confirm the image of the inside of the car 400 photographed by the camera 402 and whether or not a violent action has occurred.

The operation determination device 100 includes a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, a storage unit 114, a communication interface 115, and an I/O interface 116. have. These are communicably connected to each other via a bus.

The CPU 111 comprehensively controls the motion determination device 100 as a whole. The CPU 111 develops a program stored in the ROM 112 in the RAM 113 and executes it. The ROM 112 stores a program in which processing procedures for processing performed by the motion determination device 100 are described.

The RAM 113 serves as a work area when the CPU 111 executes programs, and temporarily stores programs and data used when executing the programs. The storage unit 114 is a non-volatile storage device such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive).

The communication interface 115 controls communication with the elevator control device 200 and the monitoring device 300. The motion determination device 100 can communicate with these devices via the communication interface 115 . The I/O interface 116 is an interface for connecting with the camera 402 .

The elevator control device 200 has a CPU 211 , a ROM 212 , a RAM 213 and a communication interface 215 . These are communicably connected to each other via a bus.

The CPU 211 comprehensively controls the elevator control device 200 as a whole. The CPU 211 develops a program stored in the ROM 212 in the RAM 213 and executes it. ROM212 stores the program in which the processing procedure of the process which the elevator control apparatus 200 performs was described.

The RAM 213 serves as a work area when the CPU 211 executes the program, and temporarily stores the program and data used when executing the program.

The communication interface 215 controls communication with the motion determination device 100 and the in-car device 401 . Elevator controller 200 may communicate with these devices via communication interface 215 .

The monitoring device 300 has a CPU 311 , a ROM 312 , a RAM 313 , a storage section 114 , a communication interface 315 and an I/O interface 316 . These are communicably connected to each other via a bus.

The CPU 311 comprehensively controls the monitoring device 300 as a whole. The CPU 311 develops a program stored in the ROM 312 in the RAM 313 and executes it. The ROM 312 stores a program in which processing procedures for processing performed by the monitoring device 300 are described.

The RAM 313 serves as a work area when the CPU 311 executes programs, and temporarily stores programs and data used when executing the programs. Also, the storage unit 314 is a non-volatile storage device such as an HDD or an SSD.

The communication interface 315 controls communication with the motion determination device 100 . The monitoring device 300 can communicate with the motion determination device 100 via the communication interface 315 . The I/O interface 316 is an interface for connecting with the input device 322 and the display device 321 .

The display device 321 is, for example, a display. The display device 321 displays the results output by the monitoring device 300 . Input device 322 is, for example, a keyboard or a mouse. The user can give instructions to the monitoring device 300 by operating the input device 322 . A plurality of cars 400 may be installed. In this case, each car 400 is equipped with a camera 402 . The motion determination device 100 determines motions of passengers for each of the plurality of cars 400 .

FIG. 2 is a diagram showing an example of a functional block diagram of the motion determination device 100 according to the first embodiment. As shown in FIG. 2 , the motion determination device 100 includes an image acquisition section 151 , a signal acquisition section 152 , an extraction section 153 , a calculation section 154 and a determination section 155 .

These operations are described below according to the flowchart. FIG. 3 is a flowchart of main processing executed by the motion determination device 100 according to the first embodiment. The main process may be activated, for example, after the motion determination device 100 is powered on. Hereinafter, "step" is also simply referred to as "S".

As shown in FIG. 3, when the main process starts, in S11, the signal acquisition unit 152 acquires the door-closed signal transmitted from the elevator control device 200, and based on the door-closed signal, the car 400 is in the door-closed state. or not. The door-closed signal is a signal that indicates whether or not the car 400 is in the door-closed state (the door of the car 400 is closed).

When the signal acquisition unit 152 determines that the car 400 is closed (YES in S11), the process proceeds to S12. If the signal acquisition unit 152 does not determine that the car 400 is in the closed state (NO in S11), the process returns to S11. In other words, the process proceeds to S12 after waiting until the door of the car 400 is closed.

In S12, the image acquisition unit 151 acquires an image inside the car 400 captured by the camera 402 (also referred to as an "in-car image"), and advances the process to S13. That is, the image acquisition unit 151 acquires the image on condition that the car 400 is in the closed state.

In S13, the extracting unit 153 extracts a person area in which a person exists as information about a person from the acquired image, and advances the process to S14.

In extracting the person area, the extracting unit 153 extracts, for example, feature points that express the characteristics of a person in the image acquired this time, and specifies the person area based on this. Alternatively, the person area may be extracted based on the difference between the image (reference image) inside the car 400 photographed in an unmanned state and the obtained image. Alternatively, the person area may be extracted based on the difference between the image acquired this time and the image taken last time (or an image taken a predetermined time before the image acquired this time). may be extracted using any known technique.

In S14, the calculation unit 154 calculates the number of passengers based on the extracted information about the person, and advances the process to S15. For example, the number of passengers may be calculated according to the number of extracted person regions, the number of passengers may be calculated according to the area of the person regions, or the number of passengers may be calculated using a known method. Just do it.

At S15, the determination unit 155 determines whether the calculated number of passengers is two or more. If the determination unit 155 determines that the calculated number of passengers is two or more (YES in S15), the process proceeds to S16. If the determination unit 155 does not determine that the calculated number of passengers is two or more (NO in S15), the process returns to S11.

In S16, the determination unit 155 determines whether or not the specific motion of the passenger has occurred based on the extracted information about the person (person area), and returns the process to S11. In the present embodiment, a passenger's "specific action" refers to an abnormal action (behavior) of a passenger in the elevator car 400. FIG. For example, a certain passenger violently punches, kicks, or grabs another passenger. This refers to actions that damage equipment in the car 400, such as hitting or kicking walls, buttons, displays, etc. (these actions are hereinafter referred to as "random actions").

When the process returns to S11, if it is again determined that the car 400 is in the closed state, an image is obtained, and it is determined whether or not the passenger has made a violent motion based on the image. In this manner, the image acquisition unit 151 periodically acquires images when it determines that the car 400 is in the door-closed state from the acquired signal.

When the calculated number of passengers is two or more, the determination unit 155 determines whether or not a violent action has occurred based on the difference in the information (human area) about the person in a plurality of images acquired periodically. judge.

Specifically, the determination unit 155 calculates a difference between an image acquired this time (also referred to as a “current image”) and an image captured last time (also referred to as a “previous image”), and performs a violent action based on the difference. It is determined whether or not has occurred. Here, the previous image is an image with the door closed that was captured a predetermined time before the current image with the door closed (for example, several hundred milliseconds to several seconds before).

The image is divided into a predetermined number of frames (eg, 4800 frames). Then, the determination unit 155 calculates the optical flow of each frame within the person area. Optical flow is an apparent velocity distribution representing the direction and magnitude of motion in each frame of an image. The determination unit 155 calculates an optical flow based on the previous image and the current image within the person area.

The determination unit 155 determines whether or not a violent motion has occurred by analyzing variations in the direction and magnitude of the optical flow. In the present embodiment, for example, a violent motion occurs based on the average value of the magnitude of motion of each frame in the person region between the previous image and the current image (this value is hereinafter referred to as the “motion amount”). determine whether or not

The determination unit 155 determines that a violent motion has occurred when the amount of motion exceeds a predetermined threshold, and determines that a violent motion has not occurred when the amount of motion does not exceed the predetermined threshold. . In addition to the above, the optical flow is analyzed to detect abnormal movements of passengers, such as each frame moving rapidly up, down, left and right, or moving large and small irregularly, and judgment of violent movement is made. It may be something like

The determination result of the determining unit 155 is transmitted to the monitoring device 300 together with the in-car image. The user can check the determination result and the car interior image on the display device 321 of the monitoring device 300 . Incidentally, when the calculated number of passengers is one or more, it may be determined whether or not a violent action has occurred.

FIG. 4 is a diagram for explaining determination of a violent motion according to the first embodiment. A screen 331 a (left diagram) and a screen 331 b (right diagram) are screens displaying images inside the car 400 photographed by the camera 402 and displayed on the display device 321 of the monitoring device 300 . The screen 331a shows a state in which the passenger has almost no movement, and the screen 331b shows a state in which it is determined that there is a violent movement.

As shown in FIG. 4, in screen 331a, two passengers (passenger 21 surrounded by a solid line and passenger 22 surrounded by a dotted line) are in the car 400 . None of the passengers are nearly motionless. Further, the entrance/exit 11 of the car 400 is in a state where the door is closed (door closed state).

The changes in the amount of movement of the passengers 21 and 22 are shown in the lower part of FIG. The vertical axis indicates the amount of passenger movement, and the horizontal axis indicates time. A motion amount 31 indicated by a solid line indicates the motion amount of the passenger 21 . A motion amount 32 indicated by a dotted line indicates the motion amount of the passenger 22 . A dashed-dotted line is a predetermined threshold. The same applies to the graphs shown below.

In the graph (left graph) corresponding to the screen 331a, both the amount of movement 31 of the passenger 21 and the amount of movement 32 of the passenger 22 change very little, and the amount of movement does not exceed the threshold.

On the other hand, on the screen 331b, the passenger 21 is greatly moving his arm. In the graph (right graph) corresponding to the screen 331b, the change in the motion amount 31 of the passenger 21 is large, and the motion amount exceeds the threshold. When the motion amount exceeds the threshold, the determination unit 155 determines that the passenger 21 has made a violent motion.

As described above, by judging a violent movement when the door is closed, it is possible to eliminate as much as possible changes other than the movement of passengers during boarding, such as changes in the background due to the opening and closing of the car doors and movement of people due to getting in and out of the car. can be done. Furthermore, since the determination is limited to situations where the number of passengers is two or more, where there is a possibility of harming others, it is possible to accurately determine the violent movements of passengers in the elevator car. .

Here, if there are passengers, the background may change due to the reflection of the passengers on the walls and doors. In order to prevent the background from changing due to reflections on doors and walls, mask processing is required. However, in a see-through elevator constructed with transparent or translucent walls, the image on the wall surface changes greatly, and the illuminance inside the car changes greatly, making it difficult to apply mask processing. In the present embodiment, since the human region is extracted and the amount of movement in the human region is calculated, even in the case described above, it is possible to accurately determine the violent motion of the passengers in the elevator car. can.

[Second embodiment]
A second embodiment will be described below. In the description of the second embodiment, points different from the first embodiment will be described, and descriptions of common parts will be omitted. FIG. 5 is a diagram showing an example of a functional block diagram of the motion determination device 100 according to the second embodiment.

In the first embodiment, the motion determination device 100 includes a signal acquisition unit 152, an image acquisition unit 151, an extraction unit 153, a calculation unit 154, and a determination unit 155. On the other hand, in the second embodiment, instead of the extraction unit 153, an extraction unit 156 is provided.

The following will be explained using the flowchart below. FIG. 6 is a flowchart of main processing executed by the motion determination device 100 according to the second embodiment.

As shown in FIG. 6, when the main process starts, in S21, the signal acquisition unit 152 acquires the door-closed signal transmitted from the elevator control device 200, and based on the door-closed signal, the car 400 is in the door-closed state. or not.

When the signal acquisition unit 152 determines that the car 400 is closed (YES in S21), the process proceeds to S22. If the signal acquisition unit 152 does not determine that the car 400 is in the closed state (NO in S21), the process returns to S21. In S22, the image acquisition unit 151 acquires an in-car image, and the process proceeds to S23.

In S23, the extraction unit 156 extracts positional information of multiple parts of the human body as information about the person from the acquired image, and advances the process to S24.

Here, the positional information of multiple parts of the human body may be extracted using a known skeleton estimation (orientation estimation) technique such as OpenPose. In this case, two cameras 402 may be installed in the car 400 and the position information of multiple parts of the human body may be extracted based on the images captured by the two cameras 402 . If there are multiple passengers, the extraction unit 156 extracts position information for each passenger.

A plurality of parts of the human body are, for example, the head, neck, right shoulder, left shoulder, right wrist, left wrist, right hip joint, left hip joint, right heel, and left heel. Not limited to this, the plurality of parts of the human body may include fingertips, palms, elbows, knees, toes, and the like.

The positions of these parts are indicated by three-dimensional coordinates (X, Y, Z). For example, as shown in the screen 332a of FIG. 7, the central position of the floor surface in the car is (X, Y, Z)=(0, 0, 0). Hereinafter, the right, left, front, and rear refer to the right, left, front, and rear, respectively, as seen from the passenger when the passenger stands at the center and faces the direction of the doorway 11 . In this case, the right wall is called the right wall, the left wall is called the left wall, the front wall is called the front wall, and the rear wall is called the rear wall.

When the passenger stands facing the doorway 11 at the center position, the direction to the front doorway 11 is the X axis, the direction to the right is the Y axis, and the direction to the ceiling is the Z axis (see FIG. 7). Here, the position of X = 1000 is the position of the front wall (entrance 11), the position of X = -1000 is the position of the rear wall, the position of Y = 1000 is the position of the right wall, Let the position of Y=-1000 be the position of the left wall. The position of the head of the passenger 21 (X, Y, Z)=(-100, -700, 1600). Passenger 22 head position (X, Y, Z) = (400, 700, 1600). Note that the positions of the plurality of parts may be indicated by two-dimensional coordinates (X, Z).

In S24, the calculation unit 154 calculates the number of passengers based on the extracted information about the person (position information of multiple parts of the human body), and advances the process to S15. The calculation unit 154 counts the number of persons extracted by the extraction unit 156 as the number of passengers.

In S25, the determination unit 155 determines whether the calculated number of passengers is two or more. If the determination unit 155 determines that the calculated number of passengers is two or more (YES in S25), the process proceeds to S26. If the determination unit 155 does not determine that the calculated number of passengers is two or more (NO in S25), the process returns to S21.

In S26, the determination unit 155 determines whether or not the passenger has violently behaved based on the extracted information about the person (positional information of a plurality of parts of the human body). Repeat process.

In other words, when the calculated number of passengers is two or more, the determination unit 155 determines whether or not a violent motion has occurred based on the difference in the positional information of the plurality of parts of the human body in the plurality of images acquired periodically. determine whether In the present embodiment, the multiple parts of the human body are the head, neck, right shoulder, left shoulder, right wrist, left wrist, right hip joint, left hip joint, right heel, and left heel.

The determination unit 155 has a plurality of determination modes. Judgment mode 1 is based on the positional information of all extracted parts (head, neck, right shoulder, left shoulder, right wrist, left wrist, right hip joint, left hip joint, right heel, and left heel). This is the mode for judging whether or not the

Judgment modes 2 to 4 are modes for judging whether or not a violent action has occurred based on the position information of some of the multiple parts. In this embodiment, elevator maintenance personnel, building managers, and the like can set any one of judgment modes 1 to 4 from the input device 322 of the monitoring device 300 . The determination unit 155 performs determination based on the set determination mode.

Determination mode 2 is a mode in which it is determined whether or not a violent motion has occurred based on hand position information. Specifically, determination unit 155 determines whether or not a violent motion has occurred based on the position information of the right wrist and left wrist. It should be noted that the judgment may include the right shoulder and the left shoulder, and if there is position information related to "hands" such as fingertips and elbows as position information, such information may be included in the judgment.

Determination mode 3 is a mode in which it is determined whether or not a violent motion has occurred based on the position information of the foot. Specifically, the determination unit 155 determines whether or not a violent motion has occurred based on the positional information of the right and left heels. It should be noted that the judgment may include the right hip joint and the left hip joint, and if there is position information related to "legs" such as toes and knees as position information, such information may be included in the judgment.

Determination mode 4 is a mode in which it is determined whether or not a violent motion has occurred based on the positional information of the hands and feet. Specifically, the determination unit 155 determines whether or not a violent motion has occurred based on the positional information of the right wrist, left wrist, right heel, and left heel.

For example, in determination mode 1, it is determined whether or not a violent action has occurred based on the average value of the amount of positional change of each part between the previous image and the current image (this value is hereinafter referred to as the "movement amount"). do.

For example, if the position of the right wrist of the passenger 21 in the previous image is (-100, -800, 900) and the position of the right wrist of the passenger 21 in the current image is (-100, -820, 920), the passenger The amount of change in the position of the right wrist of 21 is -20 in the Y-axis direction and 20 in the Z-axis direction, so ((-20)^2+(20)^2)^0.5=28.3 be. In this way, the amount of positional change is obtained for each part, and the average value of these values is calculated as the amount of movement.

The determination unit 155 determines that a violent motion has occurred when the amount of motion exceeds a predetermined threshold, and determines that a violent motion has not occurred when the amount of motion does not exceed the predetermined threshold. .

It should be noted that the above is just an example of a method for determining a violent motion. Not limited to this, it analyzes the positional changes of each part so that it can detect abnormal movements of passengers, such as sudden movements of each part up, down, left and right, large and small irregular movements, and judges violent movements. It should be something that does. Moreover, the above-mentioned "a plurality of images acquired periodically" may be three or more images captured in time series. In this case, the current position information may be corrected by curve fitting based on the time-series position information.

In determination mode 2, it is determined that a violent motion has occurred when the average value of the positional change of the right wrist and the positional change of the left wrist in the previous image and the current image exceeds the threshold. In determination mode 3, when the average value of the positional change of the right heel and the positional change of the left heel in the previous image and the current image exceeds the threshold, it is determined that a violent motion has occurred. In determination mode 4, the same determination is made. Judgment modes 2 to 4 may be used when it is desired to pay particular attention to the movement of the hand or foot, such as hitting or kicking.

The rotational direction and rotational speed of the right hand may be calculated from the positional relationship between the right wrist and right shoulder in the previous image and the positional relationship between the right wrist and right shoulder in the current image. The same is true for the left hand. The rotational direction and rotational speed of the right foot may be calculated from the positional relationship between the right heel and the right hip joint in the previous image and the positional relationship between the right heel and the right hip joint in the current image. The same is true for the left foot.

The determination unit 155 may use these rotation directions and rotation speeds as criteria for determination. Also, in Mode 1, the weight may be changed according to the part. For example, among a plurality of parts, the weights of hands and feet may be increased.

FIG. 7 is a diagram for explaining determination of a violent motion according to the second embodiment. The screen 332a (left figure) shows a state in which the passenger has almost no movement, and the screen 332b (right figure) shows a state in which it is determined that there is a violent movement.

Screens 332 a and 332 b are images of the interior of the car 400 captured by the camera 402 and displayed on the display device 321 of the monitoring device 300 .

As shown in FIG. 7, on the screen 332a, two passengers (passengers 21 and 22) are in the car 400. In the passenger 21, solid lines are lines connecting a plurality of parts of the human body. In the passenger 22, dotted lines are lines connecting multiple parts of the human body. Both passenger 21 and passenger 22 are substantially motionless. Further, the doorway 11 of the car 400 is closed.

In the graph (left graph) corresponding to the screen 332a, both the amount of movement 31 of the passenger 21 and the amount of movement 32 of the passenger 22 change very little, and the amount of movement does not exceed the threshold.

On the other hand, on the screen 332b, the passenger 21 is greatly moving his arm. In the graph (right graph) corresponding to the screen 332b, the change in the motion amount 31 of the passenger 21 is large, and the motion amount 31 exceeds the threshold. When the movement amount 31 exceeds the threshold, the determination unit 155 determines that the passenger 21 has made a violent motion.

As explained above, by judging when the doors are closed, it is possible to eliminate as much as possible changes other than the movements of the passengers during boarding, such as changes in the background due to the opening and closing of the car doors and movements of people due to boarding and exiting. Moreover, it is the case where the number of passengers is two or more that there is a possibility of inflicting harm on others. For this reason, the determination unit 155 performs determination by limiting the number of passengers to cases of two or more and focusing on movements of a plurality of parts of the human body. As a result, it is possible to accurately determine the violent movements of the passengers in the car 400 of the elevator.

Here, in the first embodiment, the person area is extracted from the features of the image, but in this case, the background other than the person may be extracted as the person area. In the second embodiment, since each part of the human body is estimated, the extraction accuracy of the person is higher than in the first embodiment. Therefore, even in a see-through elevator where the image on the wall surface changes greatly or the illuminance inside the car changes greatly, it is possible to accurately determine the violent motion of the passengers inside the elevator car. Further, in modes 2 to 4, it is possible to make a determination focusing on movements of hands and feet that may cause harm to others.

[Third embodiment]
A third embodiment will be described below. In the description of the third embodiment, points different from the first and second embodiments will be described, and descriptions of common parts will be omitted.

FIG. 8 is a diagram showing an example of a functional block diagram of the motion determination device 100 according to the third embodiment. In the first embodiment, the motion determination device 100 includes a signal acquisition section 152 , an image acquisition section 151 , an extraction section 153 , a calculation section 154 and a determination section 155 . On the other hand, as shown in FIG. 8, in the third embodiment, a weight calculator 157 is further provided. Note that the weight calculator 157 may be further provided in the second embodiment as well.

The following will be explained using a flowchart. FIG. 9 is a flowchart of main processing executed by the motion determination device 100 according to the third embodiment. Since the processing from S31 to S35 is the same as the processing from S11 to S15, the description is omitted here.

In S36, the weight calculation unit 157 reads the weight information stored in the storage unit 114, sets it as a weight, and advances the process to S37. In S<b>37 , the determination unit 155 determines whether or not a violent motion occurs based on the value obtained by multiplying the difference by the weight, and returns the process to S<b>31 . Specifically, the determination unit 155 calculates a value obtained by multiplying the amount of movement of each passenger by a weight, and determines that a violent motion has occurred when the value exceeds a threshold value.

The processing of the weight calculation unit 157 and the determination unit 155 will be explained using FIG. FIG. 10 is a diagram for explaining determination of a violent motion according to the third embodiment.

In the third embodiment, the weight is set to be greater when the distance between passengers is distance A than when distance B is longer than distance A. Specifically, the weight increases as the distance between passengers decreases, and the weight decreases as the distance between passengers increases. Proximity has a weight>1, and non-proximity has a weight<1.

If the distance between passengers is close, there is a high possibility of harming others. In such a situation, it is likely to be determined that the motion is violent, so it is possible to accurately determine the violent motion of the passenger in the car of the elevator.

The screen 333a (left figure) shows a state in which the distance between passengers is long, and the screen 333b (right figure) shows a state in which the distance between passengers is long. As shown in FIG. 10, the distance between the passenger 21 and the passenger 22 is long on the screen 333a. Passenger 21 is moving his hand violently, but passenger 22 hardly moves.

In the graph corresponding to the screen 333a (left graph), the passenger 21 is moving his hand violently. Normally, as in the examples shown in FIGS. exceeding , it is determined that there is a violent motion. However, in this case, since the distance between the passengers is long, the weight < 1, and the motion amount x weight does not exceed the threshold. Therefore, it is determined that the passenger 21 does not make a violent motion.

On the other hand, on screen 333b, the distance between passenger 21 and passenger 22 is short. Passenger 21 is moving his hand violently, but passenger 22 hardly moves.

In the graph corresponding to screen 333b (right graph), passenger 21 is moving his hand violently, and the distance between passengers 21 and 22 is short. Therefore, weight>1, and the motion amount×weight exceeds the threshold. As a result, it is determined that the passenger 21 is acting violently.

[Fourth embodiment]
A fourth embodiment will be described below. In the explanation of the fourth embodiment, the points different from those of the first to third embodiments will be explained, and the explanation of the common parts will be omitted.

FIG. 11 is a diagram for explaining determination of a violent motion according to the fourth embodiment. In the fourth embodiment, weighting is performed as follows, and the determination unit 155 determines whether or not a violent motion has occurred based on the value obtained by multiplying the amount of motion by the weight.

If the distance between the passenger 21 and the passenger 22 when the car 400 changes from the door open state to the door closed state is equal to or less than the distance C, the weight calculation unit 157 reduces the weight of the passenger 21 and the passenger 22 (weight <1) Set as follows.

On the other hand, weight calculator 157 determines that the distance between passenger 21 and passenger 22 when car 400 changes from the door open state to the door closed state is greater than or equal to distance D, which is greater than distance C. , the weight for the first passenger is increased (weight>1).

If the distance between the passengers is close at the time of boarding, there is a high possibility that they are acquaintances, and if the distance between the passengers is long at the time of boarding, but they approach afterward, there is a high possibility that a stranger will cause harm. be done. Since the latter situation is more likely to be determined to be a violent motion, it is possible to accurately determine the violent motion of the passenger in the car of the elevator.

Specifically, as shown in FIG. 11, in the example of screen 334a, the distance between passenger 21 and passenger 22 when car 400 changes from the door open state to the door closed state is short (distance C or less). . In this case, the weight calculator 157 sets the weight <1 for the passenger 21 and the passenger 22 .

After that, suppose passenger 21 violently moves his/her hand, as shown in screen 334b. Normally, the motion amount of the passenger 21 exceeds the threshold value, so it is determined that there is a violent motion. However, the passenger's 21 motion amount×weight (<1) does not exceed the threshold value, and the determination unit 155 determines that there is no violent movement.

On the other hand, in the example of the screen 334c, the distance between the passenger 21 and the passenger 22 when the car 400 changes from the door open state to the door closed state is long (at least the distance D, which is greater than the distance C). In this case, the weight calculator 157 sets weight>1 for the passenger 21 and the passenger 22 .

After that, let's say that passenger 21 moves his hand violently, as shown in screen 334d. In this case, the motion amount of the passenger 21×weight (>1) exceeds the threshold, and the determination unit 155 determines that there is a violent motion.

[Fifth embodiment]
The fifth embodiment will be described below. In the description of the fifth embodiment, the points different from those of the first to fourth embodiments will be described, and descriptions of common parts will be omitted.

FIG. 12 is a diagram for explaining determination of a violent motion according to the fifth embodiment. In the fifth embodiment, weighting and determination are performed as follows.

The determination unit 155 determines whether the passenger is a child based on the position information of multiple parts of the human body. The determination unit 155 determines whether or not the specific action has occurred based on the value obtained by multiplying the motion amount by the weight. The weight is set so that passengers determined to be children are less than those not determined to be children.

By doing so, it is possible to make judgments by excluding the movements of children who move around without intending to harm others, so it is possible to accurately judge violent movements of passengers in the elevator car.

Specifically, as shown in FIG. 12, in the example of screen 335a, both passengers 21 and 22 are determined to be adults based on the position information. For example, the height is calculated from the difference in the position (height) between the head and the heel, and if the height is equal to or above a predetermined value, it is determined that the person is an adult, and if the height is less than the predetermined value, it is determined that the person is under the height.

If it is determined that the person is an adult, the weight is set to 1. Passenger 21 is moving his hand violently. As a result, the amount of movement of the passenger 21 exceeds the threshold value, and it is determined that there is a violent motion.

On the other hand, in the example of screen 335b, passenger 21 is determined to be a child and passenger 22 is determined to be an adult based on the position information.

If determined to be a child, a weight <1 (eg, 0.5) is set. Passenger 21 is moving his hand violently. Normally, the amount of movement of the passenger 21 exceeds the threshold, and it is determined that there is a violent motion. However, in this case, the motion amount of the passenger 21×weight (=0.5) does not exceed the threshold, and it is determined that there is no violent motion.

[Sixth embodiment]
The sixth embodiment will be described below. In the description of the sixth embodiment, the differences from the first to fifth embodiments will be described, and descriptions of common parts will be omitted.

In the first to fifth embodiments, when it is determined that there are two or more passengers, the determination unit 155 makes a determination. On the other hand, in the sixth embodiment, when it is determined that there is only one passenger, the determination unit 155 makes a determination.

　If there is only one passenger in the car, no harm will occur to others. However, actions such as hitting or kicking walls or attempting to destroy equipment in the car (rambling action) may occur. On the other hand, passengers may perform actions such as golf swings and gymnastics while riding alone. These motions are motions unrelated to the rampage motion. It should be noted that when two or more people are riding in the vehicle, it is generally less likely that actions such as a golf swing or gymnastics will be performed, and even if they are performed, they are often performed in small motions.

The main processing in the sixth embodiment will be described below. FIG. 13 is a flowchart of main processing executed by the motion determination device 100 according to the sixth embodiment.

The processing of S41 to S44 is the same as the processing of S21 to S24, so the explanation is omitted. In S45, unlike S25, when the calculated number of passengers is one, the process proceeds to the next process (S46). After the weight calculation unit 157 calculates the weight in S47, the determination unit 155 makes a determination in S48.

A specific description will be given below with reference to FIG. FIG. 14 is a diagram for explaining determination of a violent motion according to the sixth embodiment. In the sixth embodiment, weighting is performed as follows.

When the passenger's position is in the first position, it is closer to the walls of the car 400 and farther from the center position of the car 400 than when it is in the second position. The weight calculator 157 sets the weight to be smaller when the passenger is at the second position (closer to the center position) than at the first position (closer to the wall). In addition, the weight calculator 157 determines that when the position of the passenger is at the first position (position close to the wall), the position of the hands and feet in the wall direction is greater than when the position is at the second position (position close to the center position). set so that the weight for is increased.

Specifically, the weight calculator 157 sets a smaller weight as the position of the passenger is closer to the center position of the car 400 (weight < 1). The weight calculator 157 sets a greater weight for the motion of the hand or foot moving toward the wall as the position of the passenger is closer to the wall (weight > 1).

As shown in FIG. 14, in the example of the screen 336a, the passenger 21 is standing in the center position, so a weight <1 (eg, 0.5) is set. Although the passenger 21 is moving his hand violently, the amount of movement×weight (=0.5) does not exceed the threshold, and it is determined that there is no violent movement.

On the other hand, in the example of screen 336b, passenger 21 is standing near the wall. The position of the left heel of the passenger 21 is (X, Y, Z)=(400, -900, 0). The position of -1000 on the Y axis is the position of the left wall, and it is determined that the passenger 21 is near the left wall.

In addition, he was moving his hand violently toward the wall (in the direction of the wall). Assume that the position of the left wrist of the passenger 21 in the previous image is (600, -900, 1400), and the position of the left wrist of the passenger 21 in the current image is (650, -1000, 1400). The difference (X, Y, Z) of the position of the left wrist of the passenger 21 is (50, -100, 0).

In this case, it is set so that the weight regarding the position of the hands and feet in the direction of the left wall where the passenger 21 is close is increased. In FIG. 14, the wall direction on the left side is the negative direction of the Y-axis. Therefore, weighting (for example, weight=2) is performed so that the value in the negative direction of the Y-axis is amplified.

In the above example, the difference "-100" in the negative direction of the Y-axis is multiplied by the weight "2" to obtain -200. As a result, the value obtained by multiplying the difference in the position of the left wrist of the passenger 21 by the weight is (50, -200, 0). The position change amount after multiplication by the weight is ((50)̂2+(−200)̂2)̂0.5=201.

Similarly, the amount of position change is obtained for the right wrist, left heel, and right heel. For parts other than the hands and feet, the position change amount is obtained without multiplying the weight. Then, the average value (movement amount) of the position change amounts of all parts is calculated. In this example, since the amount of motion exceeds the threshold, it is determined that there is a violent motion.

As explained above, it is possible to eliminate changes in images of people other than passengers as much as possible by making judgments when the doors are closed. In addition, when there is one passenger in the car, there is a possibility that the passenger may perform actions such as punching or kicking the wall or trying to destroy the equipment in the car. For this reason, by increasing the weighting of the movement of the hand or foot toward the wall in the vicinity of the wall of the car, it becomes easier to detect the above behavior. On the other hand, passengers sometimes perform actions unrelated to violent actions such as golf swings and gymnastics near the center position of the car. Therefore, by reducing the weighting of motions near the center position, erroneous detection of violent motions can be reduced. In this way, it is possible to accurately determine the violent movements of passengers in the car of the elevator.

In the third to sixth embodiments, the extracting units 153 and 156 extract either one of the "personal region" and the "positional information of multiple parts of the human body" as the information about the person from the acquired image. I made it However, the present invention is not limited to this, and may be configured to extract either the "human area" or the "positional information of a plurality of parts of the human body". Also, a plurality of methods among the third to sixth embodiments may be combined. In the first to fifth embodiments, when the number of passengers is two or more, the weight is calculated and the violent action is determined. may be calculated or a violent action may be determined. In addition, in the sixth embodiment, the weight is calculated and the violent action is determined when the number of passengers is one. You may make it determine an operation|movement.

[Main configuration and effects]
Main configurations and effects of the above-described embodiment will be described below.

(1) The motion determination device 100 is a motion determination device 100 that determines the motion of a passenger inside the car 400 of the elevator. The motion determination device 100 can communicate with an elevator control device 200 that controls the elevator and a camera 402 installed inside the car 400 . The motion determination device 100 includes a signal acquisition section 152 , an image acquisition section 151 , extraction sections 153 and 156 , a calculation section 154 and a determination section 155 . The signal acquisition unit 152 acquires a signal indicating whether or not the car 400 is in the closed state, which is transmitted from the elevator control device 200 . The image acquisition unit 151 acquires an image inside the car 400 photographed by the camera 402 . Extraction units 153 and 156 extract information about a person from the acquired image. The calculation unit 154 calculates the number of passengers based on the extracted information about the person. The determination unit 155 determines whether or not the specific motion of the passenger has occurred based on the extracted information about the person. The image acquisition unit 151 periodically acquires an image when determining that the car 400 is in the closed state from the acquired signal. When the calculated number of passengers is one or more, the determination unit 155 determines whether or not a specific action has occurred based on the difference in the information regarding the person in the plurality of images acquired periodically. In this way, by making judgments in the door closed state, changes in images other than passengers are eliminated as much as possible, and judgments are made when passengers are present in the car. Accurate determination is possible.

(2) The extraction unit 153 extracts a person area in which a person exists as information related to the person from the acquired image. The determination unit 155 determines whether or not the specific action has occurred when the calculated number of passengers is two or more. In this way, by judging in the door closed state, changes in the image of people other than the passengers are eliminated as much as possible, and the situation is limited to situations where there are two or more passengers who may harm others. Since the determination is made, it is possible to accurately determine the specific motion of the passenger in the car of the elevator.

(3) The extraction unit 156 extracts position information of multiple parts of the human body from the acquired image as information about the person. The determination unit 155 determines whether or not the specific action has occurred when the calculated number of passengers is two or more. In this way, by judging in the door closed state, changes in the image of people other than passengers are eliminated as much as possible, and the situation is limited to situations where the number of passengers is two or more, where there is a possibility of harming others, Furthermore, since the determination is made by focusing on the movement of the parts of the human body, it is possible to accurately determine the specific motion of the passenger in the car of the elevator.

(4) The determination unit 155 determines whether or not a specific action has occurred based on the position information of some of the multiple parts. In this way, since it is possible to make a determination by focusing on the movement of a part of the human body that may cause harm to another person, it is possible to accurately determine the specific motion of the passenger in the elevator car. .

(5) Some parts are one or more parts of hands and feet. In this way, since it is possible to make a judgment by focusing on the movements of the hands and feet that may cause harm to others, it is possible to accurately judge the specific motion of the passenger in the elevator car. .

(6) Based on the position information, the determination unit 155 determines whether the passenger is a child. The determination unit 155 determines whether or not the specific action has occurred based on the value obtained by multiplying the difference by the weight. The weight is set so that passengers determined to be children are less than those not determined to be children. In this way, since the determination can be made by excluding the motions of children who move around without intending to harm others, it is possible to accurately determine the specific motions of the passengers in the car of the elevator.

(7) The determination unit 155 determines whether or not the specific action has occurred based on the value obtained by multiplying the difference by the weight. The weight is set to be greater when the distance between passengers is the first distance than when the distance is the second distance, which is longer than the first distance. If the distance between passengers is short, it is considered that there is a high possibility of harming others. In such a situation, it is easier to determine that the motion is the specific motion, so the specific motion of the passenger in the car of the elevator can be determined with high accuracy.

(8) Passengers include first and second passengers. The determination unit 155 determines whether or not the specific action has occurred based on the value obtained by multiplying the difference by the weight. If the distance between the first passenger and the second passenger when the car 400 changes from the door open state to the door closed state is equal to or less than the third distance, the weights for the first passenger and the second passenger are set to be small. It is The distance between the first passenger and the second passenger when the car 400 changes from the door open state to the door closed state is equal to or greater than a fourth distance larger than the third distance, and then the first passenger approaches the second passenger. If it is determined that the first passenger is the first passenger, the weight is set to be large. If the distance between the passengers is close at the time of boarding, there is a high possibility that they are acquaintances, and if the distance between the passengers is long at the time of boarding, but they approach afterward, there is a high possibility that a stranger will cause harm. be done. In such a situation, it is easier to determine that the motion is the specific motion, so the specific motion of the passenger in the car of the elevator can be determined with high accuracy.

(9) When the calculated number of passengers is one, the determination unit 155 determines whether or not the specific action has occurred based on the value obtained by multiplying the difference by the weight. The determination unit 155 determines whether or not a specific action has occurred. When the passenger position is in the first position, it is closer to the walls of the car 400 and farther from the center position of the car 400 than it is in the second position. When the position of the passenger is the second position, the weight is set to be smaller than when it is the first position. When the position of the passenger is in the first position, the weight regarding the positions of the hands and feet in the direction of the wall is set to be greater than in the case of the second position. In this way, the judgment is made in the door closed state to eliminate changes in the image of other passengers as much as possible, and the judgment is made when there is one passenger in the car. When there is only one passenger, by lowering the weighting of motions unrelated to specific motions such as golf swings and gymnastics, false detections of these motions can be reduced, and the hand facing the wall near the car wall can be reduced. By increasing the weighting of the movement of the legs and feet, it is easier to detect actions such as hitting or kicking a wall or trying to destroy a device in the car. As a result, it is possible to accurately determine the specific motion of the passenger in the car of the elevator.

(10) The specific action is a violent action. By doing so, it is possible to accurately determine the violent motion of the passenger in the car of the elevator.

(11) The control method is a method of controlling the motion determination device 100 that determines the motion of the passenger inside the car 400 of the elevator. The motion determination device 100 can communicate with an elevator control device 200 that controls the elevator and a camera 402 installed inside the car 400 . The control method comprises the step of acquiring a signal indicating whether or not the car 400 is in the closed state, which is transmitted from the elevator control device 200; a step of periodically acquiring images of the interior of the car 400 photographed by the camera 402; a step of extracting information about the person from the acquired image; and a step of calculating the number of passengers based on the extracted information about the person. and, when the calculated number of passengers is one or more, a step of determining whether or not a specific action of the passenger has occurred based on the information regarding the person in the plurality of images acquired periodically. In this way, by making judgments in the door closed state, changes in images other than passengers are eliminated as much as possible, and judgments are made when passengers are present in the car. Accurate determination is possible.

The embodiments disclosed this time should be considered illustrative in all respects and not restrictive. The scope of the present disclosure is indicated by the scope of claims rather than the above description, and is intended to include all changes within the meaning and scope of equivalence to the scope of claims.

1 Elevator control system, 11 entrance, 21, 22 passengers, 31, 32 movement amount, 100 motion determination device, 111, 211, 311 CPU, 112, 212, 312 ROM, 113, 213, 313 RAM, 114, 314 storage unit , 115, 215, 315 communication interface, 116, 316 I/O interface, 151 image acquisition unit, 152 signal acquisition unit, 153, 156 extraction unit, 154 calculation unit, 155 determination unit, 200 elevator control device, 300 monitoring device, 321 Display device, 322 Input device, 331a, 331b, 332a, 332b, 333a, 333b, 334a to 334d, 335a, 335b, 336a, 336b Screen, 400 Car, 401 In-car device, 402 Camera.

Claims (11)

1. A motion determination device for determining a motion of a passenger in an elevator car,
   The motion determination device is capable of communicating with an elevator control device that controls the elevator and a camera installed in the car,
   The motion determination device is
   a signal acquisition unit configured to acquire a signal indicating whether or not the car is in a closed state, which is transmitted from the elevator control device;
   an image acquisition unit that acquires an image of the interior of the car captured by the camera;
   an extraction unit that extracts information about a person from the acquired image;
   a calculation unit that calculates the number of passengers based on the extracted information about the person;
   A determination unit that determines whether or not the specific action of the passenger has occurred based on the extracted information about the person,
   The image acquisition unit periodically acquires the image when it is determined from the acquired signal that the car is in the closed state,
   When the calculated number of passengers is one or more, the determination unit determines whether or not the specific action has occurred based on a difference in information regarding the person in the plurality of images acquired periodically. Judging, operation judgment device.
2. The extracting unit extracts a person region in which a person exists as information about the person from the acquired image,
   The action determination device according to claim 1, wherein the determination unit determines whether or not the specific action has occurred when the calculated number of passengers is two or more.
3. The extracting unit extracts position information of a plurality of parts of a human body as information about the person from the acquired image,
   The action determination device according to claim 1, wherein the determination unit determines whether or not the specific action has occurred when the calculated number of passengers is two or more.
4. The motion determination device according to claim 3, wherein the determination unit determines whether or not the specific motion has occurred based on the position information of a part of the plurality of parts.
5. The motion determination device according to claim 4, wherein the partial parts are one or more parts of hands and feet.
6. The determination unit is
   determining whether the passenger is a child based on the location information;
   Determining whether the specific action has occurred based on the value obtained by multiplying the difference by a weight,
   6. The weight is set so that the passenger determined to be a child is smaller than the passenger not determined to be a child. The motion determination device according to 1.
7. The determination unit determines whether or not the specific action has occurred based on a value obtained by multiplying the difference by a weight,
   The weight is set to be larger when the distance between the passengers is the first distance than when the distance is the second distance, which is longer than the first distance. 6. The motion determination device according to any one of 5.
8. the passengers include a first passenger and a second passenger;
   The determination unit determines whether or not the specific action has occurred based on a value obtained by multiplying the difference by a weight,
   When the distance between the first passenger and the second passenger when the car changes from the door open state to the door closed state is equal to or less than a third distance, the weight related to the first passenger and the second passenger is set to be small, and
   The distance between the first passenger and the second passenger when the car changes from the door open state to the door closed state is a fourth distance or more larger than the third distance, and then the first passenger 6. The motion determination device according to any one of claims 2 to 5, wherein the weight related to the first passenger is set to be increased when it is determined that the second passenger approaches the second passenger.
9. The determination unit determines whether the specific action has occurred based on a value obtained by multiplying the difference by a weight when the calculated number of passengers is one,
   The determination unit determines whether the specific action has occurred,
   when the passenger's position is in the first position, it is closer to the walls of the car and farther from the center position of the car than when it is in the second position;
   When the position of the passenger is the second position, the weight is set to be smaller than when it is the first position,
   4. When the position of the passenger is the first position, the weight related to the position of the hands and feet in the direction of the wall is set to be larger than when the position is the second position. A motion determination device as described.
10. The action determination device according to any one of claims 1 to 9, wherein the specific action is a violent action.
11. A control method for controlling a motion determination device that determines a motion of a passenger in an elevator car, comprising:
    The motion determination device is capable of communicating with an elevator control device that controls the elevator and a camera installed in the car,
    The control method is
    obtaining a signal transmitted from the elevator control device indicating whether the car is in a closed state;
    a step of periodically acquiring an image of the inside of the car captured by the camera when it is determined from the acquired signal that the car is in the closed state;
    extracting information about the person from the captured image;
    calculating the number of passengers based on the extracted information about the person;
    a step of determining whether or not a specific action of the passenger has occurred, based on information about the person in the plurality of images acquired periodically, when the calculated number of passengers is one or more; Have a control method.

Images