JP6377796B1 - Elevator boarding detection system - Google Patents

Abstract

An elevator boarding detection system capable of simplifying the setting of a detection area when detecting a user using a camera, reducing the burden on an operator, and detecting the user correctly.
An elevator system according to the present embodiment includes an imaging unit capable of shooting a predetermined range from the vicinity of the door of the car toward the landing when the car arrives at the landing, and shooting by the imaging unit. Detection area setting means for detecting a sill between the car and the landing included in the captured image, and setting a detection area on the image based on the detected sill, and continuous in time series taken by the imaging means The user detection means that detects the presence or absence of a user by paying attention to the movement of people and objects in the detection area using the plurality of images, and the door opening / closing operation based on the detection result of the user detection means Control means for controlling.
[Selection] Figure 1

Description

  Embodiments described herein relate generally to an elevator boarding detection system that detects a user who gets in a car.

  Normally, when the elevator car arrives at the landing and opens, the door is closed after a predetermined time. At that time, since the elevator user does not know when the car is to be closed, when the user gets on the car from the landing, the user may hit the door that is in the middle of closing.

  In order to avoid such a collision between the user and the door during the boarding, the user is detected by a camera or the like attached above the car and reflected in the door opening / closing control.

Japanese Patent No. 5201826 Japanese Patent No. 5772156

  A user's detection area by a camera attached to a car is different for each elevator and may be different for each floor. Therefore, in order to obtain the optimum detection area, it is necessary for an operator to input the specification value of the elevator and the specification value of the hall for each floor and set the detection area in advance. However, when it is difficult to obtain these specification values, it is necessary to set standard specification values instead or to set specification values manually determined by an operator. For this reason, the setting man-hour of the detection area increases, and the detection performance may not be sufficiently obtained.

  The problem to be solved by the present invention is to ride an elevator that can detect a user correctly by simplifying the setting of a detection area when detecting a user using a camera, reducing the burden on a worker. It is to provide a detection system.

The elevator boarding detection system according to the present embodiment includes an imaging unit capable of photographing a predetermined range from the vicinity of the door of the car toward the landing when the car arrives at the landing, and the imaging unit The detection area setting means for setting the detection area on the image photographed by the method, and a plurality of time-series continuous images photographed by the imaging means are used to detect the movement of the person / thing in the detection area. User detection means for detecting presence / absence of a user by paying attention and control means for controlling the opening / closing operation of the door based on the detection result of the user detection means . The detection area setting means detects a sill between the platform and the landing included in the image, detects a three-sided frame region based on a corner on the landing side among the corners of the sill, and displays on the image Set a detection area that does not include the three-way frame area.

FIG. 1 is a diagram illustrating a configuration of an elevator boarding detection system according to the first embodiment. FIG. 2 is a diagram illustrating an example of an image photographed by the camera according to the first embodiment. FIG. 3 is a diagram illustrating an example of a detection area of the elevator boarding detection system according to the first embodiment. FIG. 4 is a diagram for explaining a coordinate system in the real space according to the first embodiment. FIG. 5 is a flowchart illustrating the flow of detection area setting processing according to the first embodiment. FIG. 6 is a diagram illustrating an example of edge detection by the detection area setting unit in the first embodiment. FIG. 7 is a diagram illustrating an example of detection of a sill region by the detection area setting unit in the first embodiment. FIG. 8 is a diagram illustrating an example of detecting the origin by the detection area setting unit in the first embodiment. FIG. 9 is a diagram illustrating an example of detection of a three-way frame by the detection area setting unit in the first embodiment. FIG. 10 is a flowchart illustrating an example of a three-sided frame detection process performed by the detection area setting unit according to the first embodiment. FIG. 11 is a diagram illustrating an example of detection of the front pillar of the door pocket by the detection area setting unit in the first embodiment. FIG. 12 is a flowchart illustrating an example of detection processing of the front pillar of the door pocket by the detection area setting unit in the first embodiment. FIG. 13 is a flowchart showing the overall processing flow of the elevator boarding detection system according to the first embodiment. FIG. 14 is a flowchart illustrating the flow of detection area setting processing according to the second embodiment. FIG. 15 is a diagram illustrating a detection example of a sill region by a detection area setting unit according to the second embodiment.

  Hereinafter, each embodiment will be described with reference to the drawings. In the following description, substantially or substantially the same functions and components are denoted by the same reference numerals and will be described as necessary.

[First Embodiment]
FIG. 1 is a diagram illustrating a configuration of an elevator boarding detection system 1 according to the present embodiment. Note that, here, a single car will be described as an example, but a plurality of cars have the same configuration.

  A camera 12 is installed above the entrance of the car 11. Specifically, the lens portion of the camera 12 is installed in the curtain plate 11a covering the upper part of the entrance / exit of the car 11, with the lens 15 facing the landing 15 side. The camera 12 is a small surveillance camera such as an in-vehicle camera, and has a wide-angle lens and can continuously shoot images of several frames (for example, 30 frames / second) per second. When the car 11 arrives at each floor and opens, the state of the landing 15 is photographed including the state near the car door 13 in the car 11.

  The photographing range at this time is vertical width L1 + vertical width L2 (vertical width L1 >> vertical width L2). The vertical width L1 is a shooting range on the landing 15 side, and is preferably about 3 m from the car door 13 toward the landing 15, for example. The vertical width L2 is an imaging range on the car side, and is preferably about 50 cm from the car door 13 toward the back of the car, for example. The vertical width L1 and the vertical width L2 are ranges in the depth direction. The imaging range in the width direction (direction perpendicular to the depth direction) is at least larger than the lateral width of the car 11.

  In the landing 15 on each floor, a landing door 14 is installed at the arrival entrance of the car 11 so as to be freely opened and closed. The landing door 14 opens and closes by engaging with the car door 13 when the car 11 arrives. The power source (door motor) is on the car 11 side, and the landing door 14 simply opens and closes following the car door 13. In the following description, it is assumed that when the car door 13 is opened, the landing door 14 is also opened, and when the car door 13 is closed, the landing door 14 is also closed.

  Each image (video) taken by the camera 12 is analyzed and processed in real time by the image processing device 20. In FIG. 1, the image processing device 20 is taken out from the car 11 for convenience, but the image processing device 20 is actually housed in the curtain plate 11 a together with the camera 12.

  Here, the image processing apparatus 20 includes a storage unit 21 and a user detection unit 22. The storage unit 21 has a buffer area for sequentially storing images taken by the camera 12 and temporarily holding data necessary for processing by the user detection unit 22. The user detection unit 22 pays attention to the movement of a person / thing closest to the car door 13 among a plurality of time-sequential images taken by the camera 12 and determines whether there is a user who intends to get on the vehicle. Detect.

If this user detection part 22 is divided functionally, it will be comprised by the detection area setting part 22a, the motion detection part 22b, the position estimation part 22c, and the boarding intention estimation part 22d.

  The detection area setting unit 22 a sets a detection area for an image captured by the camera 12. A description of the detection area and a detailed setting method of the detection area will be described later.

  For example, the motion detection unit 22b divides the detection area into blocks of a certain size, and compares the brightness of each image in units of blocks to detect the movement of a person / thing. The “movement of a person / thing” here refers to the movement of a person on the hall 15 or a moving body such as a wheelchair.

  The position estimation unit 22c extracts the block closest to the car door 13 from the blocks with motion detected for each image by the motion detection unit 22b, and estimates the block as the user's foot position. The boarding intention estimation unit 22d determines whether or not the user has a boarding intention based on a time-series change in the user's foot position estimated by the position estimation unit 22c.

  Note that these functions (the detection area setting unit 22a, the motion detection unit 22b, the position estimation unit 22c, and the boarding intention estimation unit 22d) may be provided in the camera 12 or may be provided in the car control device 30. Good.

  The car control device 30 is connected to an elevator control device (not shown), and transmits and receives various signals such as a hall call and a car call to and from this elevator control device. The “call hall” is a call signal registered by operating a hall call button (not shown) installed at the hall 15 on each floor, and includes information on the registered floor and the destination direction. The “car call” is a call signal registered by operating a destination call button (not shown) provided in the car room of the car 11, and includes information on the destination floor.

  In addition, the car control device 30 includes a door opening / closing control unit 31. The door opening / closing control unit 31 controls the door opening / closing of the car door 13 when the car 11 arrives at the landing 15. Specifically, the door opening / closing control unit 31 opens the car door 13 when the car 11 arrives at the landing 15 and closes the door after a predetermined time has elapsed. However, when the person detection unit 22 of the image processing device 20 detects a person who intends to get in the vehicle while the car door 13 is open, the door opening / closing control unit 31 prohibits the door closing operation of the car door 13. And keep the door open.

  FIG. 2 is a diagram illustrating an example of an image photographed by the camera 12. This example represents an image that is captured when the car door 13 is double-opened and the camera 12 is attached to the center of the car door 13. The mounting position of the camera 12 is not limited to the center of the car door 13. For example, the car door 13 may be a single door. In this case, the installation position of the camera 12 is preferably attached close to the door stop side.

  The camera 12 is installed above the entrance of the car 11. Therefore, when the car 11 is opened at the landing 15, the predetermined range (vertical width L1) on the landing 15 side and the predetermined range (vertical width L2) in the passenger car 11 are photographed.

  The image captured by the camera 12 includes a three-sided frame left frame A1, a three-sided frame right frame A2, a sill A3, a left front column (left entrance column) A4, and a right front column (right entrance column) A5. The sill A3 is an area including both the sill on the landing 15 side and the sill on the car 11 side. The lateral width of the sill A3 is equal to the width of the elevator opening. These areas are independent areas on the image.

  FIG. 3 is a diagram illustrating an example of a detection area E of the elevator boarding detection system. The detection area E includes a position estimation area E1, a boarding intention estimation area E2, a sill area E3, and a front pillar area E4.

  The position estimation area E1 is an area for estimating a part of the user's body coming from the landing 15 toward the car door 13, specifically the position of the user's feet.

  The boarding intention estimation area E2 is an area for estimating whether or not the user detected in the position estimation area E1 has a boarding intention. The boarding intention estimation area E2 is included in the position estimation area E1 and is an area for estimating the user's foot position. That is, in the boarding intention estimation area E2, the user's stepping position is estimated and the user's boarding intention is estimated.

  In real space, the position estimation area E <b> 1 has a distance of a vertical width L <b> 3 from the center of the car door 13 toward the landing 15. The vertical width L3 is set to 2 m, for example (vertical width L3 ≦ vertical width L1). The lateral width W1 of the position estimation area E1 is set to a distance greater than or equal to the lateral width W0 of the car door 13.

  The boarding intention estimation area E <b> 2 has a distance of a vertical width L <b> 4 from the center of the car door 13 toward the landing 15. The vertical width L4 is set to 1 m, for example (vertical width L4 ≦ vertical width L3). The lateral width W2 of the boarding intention estimation area E2 is set to be approximately the same distance as the lateral width W0 of the car door 13. The lateral width W2 of the boarding intention estimation area E2 may be larger than the lateral width W0. The boarding intention estimation area E2 may be substantially rectangular in real space as described above, or may be the same shape as the position estimation area E1 up to the vertical width L4, for example.

The sill area E3 is preferably the same region as the sill A3 in FIG.
The front pillar area E4 is a series of areas including the left front pillar A4 and the right front pillar A5 of FIG.
The sill area E3 and the front pillar area E4 are, for example, areas in which proximity detection is performed (hereinafter referred to as proximity detection areas). Proximity detection refers to detecting an obstacle close to the car door 13 or the landing door 14 by image processing on an image taken by the camera 12 or a sensor such as a proximity switch. The proximity detection is performed by, for example, the motion detection unit 22b of the user detection unit 22.

  In the position estimation area E1 and the boarding intention estimation area E2, for example, proximity detection may be performed in a predetermined range area close to the sill area E3. The position estimation area E1 and the boarding intention estimation area E2 may include a proximity detection area, or may be set as an area excluding the proximity detection area.

  In the present embodiment, the detection area setting unit 22 a sets the detection area E in the image captured by the camera 12. Thereafter, the motion detection unit 22b, the position estimation unit 22c, and the boarding intention estimation unit 22d perform boarding detection using the detection area E. Specific boarding detection processing will be described later with reference to FIG.

FIG. 4 is a diagram for explaining a coordinate system in real space.
The origin O of the coordinate system is the center of the car door 13. That is, the origin O corresponds to the center of the edge on the hall 15 side of the rectangular area representing the sill A3 in the image taken by the camera 12 (see FIG. 2).

  The X coordinate axis is a seat axis extending in a horizontal direction with the car door 13 provided at the entrance / exit of the car 11 from the origin O. The Y coordinate axis is a coordinate axis extending in the direction of the hall 15 perpendicularly to the car door 13 from the origin O. The Z coordinate axis is a coordinate axis extending from the origin O in the height direction of the car 11.

  It is assumed that the camera 12 is attached at a height h just above the origin O. That is, the coordinates of the installation position of the camera 12 in the real space are (0, 0, h).

  Further, as shown in FIG. 3, the coordinate system used in the image taken by the camera 12 is, for example, a coordinate having the upper left corner of the image as the origin, the horizontal direction of the image as the X axis, and the vertical direction of the image as the Y axis. It is a system. The origin O of the coordinate system in real space is the center of the edge on the hall 15 side of the rectangular area representing the sill A3 on the image. The deviation of the coordinate axis between the coordinate system on the image and the coordinate system in the real space is represented by, for example, a rotation parameter with the origin O as the rotation center. That is, the coordinate system on the image can be converted to the coordinate system in the real space by multiplying the X, Y, and Z axes of the coordinate system on the image by the rotation parameter. Note that the inverse transformation from the coordinate system in the real space to the coordinate system on the image is similarly possible using the rotation parameter.

  The camera 12 holds internal parameters when taking an image. The internal parameters are, for example, focal length, lens center, resolution, distortion coefficient, and the like.

  In the present embodiment, the detection area setting unit 22a uses the three-frame left frame A1, the three-frame right frame A2, the sill A3, the left front column A4, and the right front column A5 shown in FIG. Detection is performed by image processing, and a detection area E (position estimation area E1, boarding intention estimation area E2, sill area E3, front pillar area E4) shown in FIG. 3 is set based on these detection results. Hereinafter, the setting process of the detection area E by the detection area setting unit 22a will be specifically described with reference to FIGS.

FIG. 5 is a flowchart illustrating the flow of the detection area setting process.
In the present embodiment, it is assumed that the internal parameters of the camera 12 described above have been input to the user detection unit 22 in advance. Further, it is assumed that the installation position and rotation parameters of the camera 12 are unknown.

  In step S <b> 101, the detection area setting unit 22 a performs edge detection on the image captured by the camera 12. For the edge detection, for example, various filters such as a Sobel filter and a Laplacian filter are used. In the image after the edge detection, a large number of edges including areas indicating elevator sills, three-way frames, and front pillars are detected. In addition, when a lot of noise is included in the image after edge detection, it is preferable to perform noise removal on the image.

  FIG. 6 shows an example of edge detection by the detection area setting unit. As shown in FIG. 6, noise such as a person U may be included in the image after edge detection.

  In step S102 of FIG. 5, the detection area setting unit 22a detects the sill A3. Since a floor-shaped square pattern is unlikely to exist on the floor, a candidate for a sill is extracted by searching for a rectangular area from the image after edge detection. More specifically, the detection area setting unit 22a performs straight line detection on the image after edge detection, for example, by Hough transform. The detection area setting unit 22a searches for a rectangular area in the image after the straight line detection.

  The search for the rectangular area may be performed by template matching, for example. For example, the detection area setting unit 22 a stores a square template image in the storage unit 21. The detection area setting unit 22a uses the template image read from the storage unit 21 to detect the sill A3 by performing template matching processing on the image after edge detection. When a plurality of candidates for sill A3 are detected, for example, a region having the highest match rate based on the position, size, aspect ratio, and the like of sill A3 may be set as sill A3. Note that the template image is preferably an image representing a square with a sill average shape. In addition, the detection area setting unit 22a may adjust the shape, size, and the like of the template image during the template matching process based on the internal parameters of the camera 12 given in advance.

  The search for the rectangular area is performed by combining processes such as extraction of the intersection of two straight lines detected by the Hough transform, extraction of the angle formed by the two straight lines, recognition of the number of straight lines forming the closed area and the closed area, and the like. It may be performed without using the template matching process.

  FIG. 7 is a diagram illustrating an example of detecting a sill region by the detection area setting unit. FIG. 7 is an enlarged view of the region M in FIG. In the example of FIG. 7, using the template image TI, the template matching process is performed on the image after edge detection of FIG. 6, and as a result, two candidate areas C1 and C2 are detected. It is recognized as sill A3.

  In step S103 of FIG. 5, the detection area setting unit 22a calculates the camera installation position and rotation parameters.

  First, as shown in FIG. 8, the detection area setting unit 22a calculates the center of the edge on the hall 15 side of the area of the sill A3 on the image as a position corresponding to the origin O in the coordinate system in real space. . The detection area setting unit 22a calculates the height h of the camera 12 in the real space using the position of the origin O on the image and the internal parameters of the camera 12. That is, the three-dimensional coordinates (0, 0, h) of the installation position of the camera 12 in the coordinate system in the real space are calculated.

  Further, the end side of the sill A3 including the origin O on the image corresponds to the X axis in the coordinate system in the real space, and the Y axis in the real space is a direction orthogonal to the end side. The detection area setting unit 22a includes a deviation between the X and Y axes in the coordinate system on the image and the X and Y axes in the coordinate system in the real space, and the coordinates of the origin O in the coordinate system on the image. The rotation parameters for the X axis, the Y axis, and the Z axis are calculated by comparing the coordinates (0, 0, h) of the camera installation position in the coordinate system in the real space. Thereby, the correspondence between the coordinate system in the real space and the coordinate system on the image becomes clear. More specifically, for example, by performing perspective projection conversion processing based on the calculated installation position and rotation parameter of the camera 12, the vertical width L3 used for setting the position estimation area E1 and the boarding intention estimation area E2 , L4 and the positions and lengths on the images of the horizontal widths W1 and W2.

In step S104 of FIG. 5, the detection area setting unit 22a detects a three-sided frame.
Here, FIG. 9 is a diagram illustrating an example of detection of a three-way frame by the detection area setting unit 22a. FIG. 10 is a flowchart illustrating an example of a three-sided frame detection process performed by the detection area setting unit 22a. Since the three-way frame is adjacent to the sill and perpendicular to the floor surface, the left frame A1 of the three-way frame and the right frame A2 of the three-way frame can be detected based on the position of the sill A3.

  In step S104A of FIG. 10, the detection area setting unit 22a first detects corners Cn1 and Cn2 on the sill A3 on the landing 15 side. That is, the corners Cn1 and Cn2 are corners close to the sill hall 15 included in the hall 15 of the sill A3. The corners Cn1 and Cn2 are detected by image processing such as corner detection for the sill A3, for example.

  In step S104B, the detection area setting unit 22a detects a line segment G1 (line segment G2) extending from the corner Cn1 (corner Cn2) toward the landing 15. This line segment G1 (line segment G2) is a line segment representing the edge of the three-sided frame on the floor surface. More specifically, in the image after edge detection obtained by the process of step S102, the detection area setting unit 22a extends from the vicinity of the corner Cn1 (corner Cn2) above the image and substantially parallel to the Y axis. The segment G1 (line segment G2) is detected.

  In step S104C, the detection area setting unit 22a detects line segments G3 and G4 (line segments G5 and G6) extending vertically from both ends of the line segment G1 (line segment G2). These line segments G3 and G4 (line segments G5 and G6) are line segments representing the edge of the left frame (right frame) of the three-way frame.

  For example, the detection area setting unit 22a uses the image after edge detection, and uses line segments G3 and G4 (line segments G5 and G6) extending from the vicinity of both ends of the line segment G1 (line segment G2) toward the end of the image in three directions. It is detected as the edge of the left frame (right frame) of the frame. As another method, the detection area setting unit 22a uses an internal parameter and a rotation parameter of the camera, and the straight line on the image representing the vertical direction in the real space from the positions of both ends of the line segment G1 (line segment G2). An inclination may be calculated, and line segments G3 and G4 (line segments G5 and G6) that match the inclination may be detected as edges of the left frame (right frame) of the three-way frame.

  In step S104D, the detection area setting unit 22a performs area division using the line segment obtained by the process of step S104C, and detects the left frame A1 of the three-way frame (the right frame A2 of the three-way frame). The left frame A1 of the three-way frame (the right frame A2 of the three-way frame) is an area surrounded by the line segments G1, G3, and G4 (line segments G2, G5, and G6) detected in step S104B and step S104C.

In step S105 of FIG. 5, the detection area setting unit 22a detects the front pillar of the door pocket.
Here, FIG. 11 is a diagram illustrating an example of detection of the front pillar of the door pocket by the detection area setting unit 22a. FIG. 12 is a flowchart illustrating an example of detection processing for the front column of the door pocket by the detection area setting unit 22a. Since the front pillar of the door pocket is adjacent to the sill and perpendicular to the floor surface, as in the three-sided frame, the left front pillar A4 and the right front pillar A5 of the door pocket can be detected based on the position of the sill A3.

  In step S105A in FIG. 12, the detection area setting unit 22a first detects corners Cn3 and Cn4 on the sill A3 side of the car 11. That is, the corners Cn3 and Cn4 are corners close to the sill car 11 included in the car 11 in the sill A3. Corners Cn3 and Cn4 are detected in the same manner as the corners Cn1 and Cn2 in FIG.

  In step S105B, the detection area setting unit 22a detects a line segment G7 (line segment G8) extending in the direction of the car 11 from the corner Cn3 (corner Cn4). This line segment G7 (line segment G8) is a line segment representing the edge of the front pillar of the door pocket on the floor surface. More specifically, in the image after edge detection obtained by the process of step S102, the detection area setting unit 22a extends from the vicinity of the corner Cn3 (corner Cn4) below the image and substantially parallel to the Y axis. The minute G7 (line segment G8) is detected.

  In step S105C, the detection area setting unit 22a detects line segments G9 and G10 (line segments G11 and G12) extending vertically from both ends of the line segment G7 (line segment G8). These line segments G9 and G10 (line segments G11 and G12) are line segments representing the edge of the left front column (right front column) of the door pocket. The method for detecting line segments G9 and G10 (line segments G11 and G12) is the same as the method for detecting line segments G3 and G4 (line segments G5 and G6) in FIG.

  In step S105D, the detection area setting unit 22a performs area division using the line segment obtained by the process of step S105C, and detects the left front pillar A4 of the door pocket (the right front pillar A5 of the door pocket). The front left pillar A4 of the door pocket (right front pillar A5 of the door pocket) is an area surrounded by the line segments G7, G9, and G10 (line segments G8, G11, and G12) detected in steps S105B and S105C.

  In step S106 of FIG. 5, the detection area setting unit 22a is based on the three-sided frame left frame A1, the three-sided frame right frame A2, the sill A3, the left front column A4, and the right front column A5 obtained in the processes of steps S102 to S105. Thus, the detection area E shown in FIG. 3 is set.

  The position estimation area E1 is, for example, an area limited by the horizontal width W1 and the vertical width L3 among the areas surrounded by the left frame A1 of the three-way frame, the right frame A2 of the three-way frame, and the sill A3.

  The front pillar area E4 is, for example, a series of areas obtained by adding a substantially rectangular shape having sides facing the line segment G7 of the left front pillar A4 and the line segment G8 of the right front pillar A5 to the detected left front pillar A4 and right front pillar A5. .

  The boarding intention estimation area E2 is set as an area included in the position estimation area E1 having the vertical width L4 as described above. The sill area E3 is the same region as the sill A3 as described above.

  As described above, the detection area setting unit 22a automatically sets an optimal detection area E for the image by performing image processing on the image captured by the camera 12.

  In addition, it is preferable that the detection area setting unit 22a performs the detection area setting process with the car door 13 and the landing door 14 fully opened so that the camera 12 can display the range of all the detection areas E. However, the detection area setting unit 22a may perform the setting process of the front pillar area E4 in the detection area E with the car door 13 and the landing door 14 being fully closed. The image photographed by the camera 12 in the fully closed state is suitable for detection area setting processing because there is little noise due to people, objects or light from the landing 15 side. In this case, the detection area setting unit 22a first performs edge detection on the image captured by the camera 12 in the fully closed state. The detection area setting unit 22a detects the end of the sill A3 on the side of the car 11 (the frontage of the elevator) from the obtained image after edge detection, and detects corners Cn3 and Cn4 that are both ends of this end. This edge may be performed, for example, by searching for a straight line substantially parallel to the X axis of the coordinate system on the image, or may be detected by a template matching process or the like. The detection area setting unit 22a sets the front pillar area E4 by executing the processes of steps S105B to S105D in FIG. 12 based on the corners Cn3 and Cn4.

  FIG. 13 is a flowchart showing the overall processing flow of the elevator boarding detection system 1.

  In the present embodiment, the detection area E is set for each floor in consideration of the case where the specifications of the three-way frame differ for each floor.

  When the car 11 arrives at the landing 15 on an arbitrary floor (Yes in Step S11), the car control device 30 opens the car door 13 and waits for a user to get on the car 11 (Step S12).

  At this time, a predetermined range on the landing side (vertical width L1) and a predetermined range in the car (vertical width L2) are set at a predetermined frame rate (for example, 30 frames / second) by the camera 12 installed at the upper entrance of the entrance of the car 11. Taken. The image processing apparatus 20 acquires images captured by the camera 12 in time series, and sequentially stores these images in the storage unit 21 (step S13).

  And the detection area setting part 22a of the user detection part 22 performs a detection area setting process, when the detection area E of the floor where the car 11 arrived has not been set (No in step S14) (step S15). ). The detection area setting process has been described with reference to FIGS.

  Further, the detection area setting unit 22a may store the detection area E for each floor set in step S15 in the storage unit 21. Thereby, detection area setting part 22a can omit calculation of detection area E when arriving at the same floor after that. More specifically, each time the detection area setting unit 22a arrives at a different floor, the detection area setting unit 22a first searches the storage unit 21 to search whether or not the detection area E of this floor has been stored. When the detection area E of the floor has been stored, the detection area setting unit 22a reads out and sets the detection area E from the storage unit 21 in step S15, so that the detection area E need not be calculated. .

  Next, the user detection unit 22 of the image processing apparatus 20 executes the following user detection process in real time (step S16). The user detection process includes a motion detection process (step S16-1) executed by the motion detection unit 22b, a position estimation process (step S16-2) executed by the position estimation unit 22c, and a boarding intention performed by the boarding intention estimation unit 22d. This is divided into estimation processing (step S16-3).

  First, the motion detection unit 22b first detects whether there is a movement of a person or an object in the detection area E in order to detect the motion of a user who intends to get on the vehicle from the captured image. More specifically, the motion detection unit 22b divides the current frame image and the previous frame image stored in the storage unit 21 into blocks each having a predetermined size, and average brightness for each block. Calculate and compare values. As a result, when there is a block having a luminance difference equal to or greater than a preset value, the motion detection unit 22b determines the block as a block with motion. Thereafter, similarly, the motion detection unit 22b repeatedly determines the presence or absence of motion while comparing the luminance value of each image captured by the camera 12 in block order in time series.

  Moreover, the motion detection part 22b transmits a user detection signal to the car control apparatus 30, when a block with a motion is detected in proximity detection areas, such as the sill area E3 and the front pillar area E4. In this case, the process proceeds to step S17. The proximity detection for the sill area E3 and the front pillar area E4 is effective for detecting a person or an object that goes out to the landing 15 from the car 11.

  Next, the position estimation unit 22c extracts a block closest to the car door 13 from among the blocks with motion in the current image based on the detection result of the motion detection unit 22b. Then, the motion detection unit 22 b obtains the Y coordinate of this block as data on the user's foot position and stores it in the storage unit 21.

  Thereafter, in the same manner, the position estimation unit 22 c obtains data on the user's foot position and stores it in the storage unit 21. The foot position estimation process is performed not only in the position estimation area E1 but also in the boarding intention estimation area E2.

  Furthermore, the boarding intention estimation part 22d smoothes the data of the user's foot position obtained by the position estimation process. As a smoothing method, for example, a generally known method such as an average value filter or a Kalman filter is used, and detailed description thereof is omitted here.

  In the smoothed foot position data series, when there is data whose change amount is equal to or greater than a predetermined value, the boarding intention estimation unit 22d excludes the data as an outlier. The predetermined value is determined by the standard walking speed of the user and the frame rate of the captured image. Moreover, the boarding intention estimation part 22d may find and exclude an outlier before smoothing the data of the foot position. When a user having a boarding intention is detected as a result of the boarding intention estimation process, a user detection signal is output from the image processing device 20 to the car control device 30.

  As a result of the above-described user detection process (step S16), when the car control device 30 receives a user detection signal (Yes in step S17), the door opening / closing control unit 31 prohibits the door closing operation of the car door 13. The door open state is maintained (step S18).

  More specifically, when the car door 13 is fully opened, the car control device 30 starts counting the door opening time, and when the predetermined time T (unit is minutes, for example, 1 minute) is counted, Close. During this period, when a user who has a boarding intention is detected and a user detection signal is received, the car control device 30 stops the count operation and clears the count value. Thereby, the door open state of the car door 13 is maintained for the predetermined time T. If a new user detection signal is received during this period, the count value is cleared again, and the car door 13 is kept open for the predetermined time T. The count value may be cleared every time a user detection signal is received, or may be performed every T minutes.

  However, if the user detection signal is received many times during the predetermined time T, the car door 13 cannot be closed indefinitely. Therefore, the allowable time Tx (in minutes, for example, 3 minutes) It is preferable to forcibly close the car door 13 when the allowable time Tx has elapsed.

  When the counting operation for the predetermined time T is completed (step S19), the car control device 30 closes the car door 13 and starts the car 11 toward the destination floor (step S20).

  As described above, according to the present embodiment, by analyzing the image obtained by photographing the landing 15 with the camera 12 installed at the upper part of the entrance / exit of the car 11, for example, from the place slightly away from the car 11 toward the car door 13. A coming user or a user who goes out to the landing from the car 11 can be detected and reflected in the door opening / closing operation.

  According to the embodiment described above, the elevator boarding detection system 1 performs image processing on the left frame A1, the three-way frame, the right frame A2, the sill A3, the left front column A4, and the right front column A5 when the elevator is in use. Detect automatically. Furthermore, the elevator boarding detection system 1 automatically sets the detection area E (position estimation area E1, boarding intention estimation area E2, sill area E3, front pillar area E4) based on these detection results. Thereby, for example, the optimal detection area E is set regardless of the specifications of the elevator such as the size of the frontage, the size of the door pocket, or the specification of the three-way frame for each floor where the elevator stops. A highly stable elevator boarding detection system can be provided.

  Further, since the detection area E is automatically set, it is not necessary to check the specifications at the time of factory shipment and the setting work at the time of installing the elevator.

  In the elevator boarding detection system 1 according to the present embodiment, internal parameters of the camera 12 are given to the elevator boarding detection system 1 in advance as prior knowledge, and the installation position and rotation parameters of the camera 12 are automatically obtained by image processing. Calculated. As a result, for example, a maintenance worker who attaches the camera 12 does not need to finely adjust the installation position of the camera 12, and thus the burden on the maintenance worker can be reduced.

  The detection area setting unit 22a calculates the position of the origin O on the image by the perspective projection conversion process based on the calculated coordinates of the installation position of the camera 12 and the rotation parameter, and based on the position of the origin O, the detection area setting unit 22a has already calculated the position. The position of the origin O calculated based on the area of the sill A3 may be corrected.

  In the present embodiment, the vertical widths L3 and L4 and the horizontal widths W1 and W2 used for setting the position estimation area E1 and the boarding intention estimation area E2 may be automatically set according to the environment near the landing 15 on each floor. Good. For example, the detection area setting unit 22a automatically detects the depth and width of the hall 15 for each floor from the image taken by the camera 12, and the vertical widths L3 and L4 and the horizontal widths W1 and W2 are determined accordingly. It may be stretched. Thereby, the position estimation area E1 and the boarding intention estimation area E2 can be set more appropriately for each floor.

  In the present embodiment, the detection area setting unit 22a includes the coordinates of the vertices on the image that constitute the respective areas of the left frame A1, the right frame A2, the sill A3, the left front column A4, and the right front column A5. May be stored in the storage unit 21. Further, based on the coordinates of the vertices and the lengths of the sides, specifications such as the dimensions of each part of the elevator in the real space may be calculated and stored in the storage unit 21. For example, the lateral width of the sill A3 may be calculated and stored in the storage unit 21 as the length of the elevator opening.

  In the present embodiment, the detection area setting unit 22a detects at least one of the left frame A1 of the three-way frame, the right frame A2 of the three-way frame, the sill A3, the left front column A4, and the right front column A5. At least one of E (position estimation area E1, boarding intention estimation area E2, sill area E3, front pillar area E4) may be set.

[Second Embodiment]
In the present embodiment, a modified example of the first embodiment will be described.
In the detection area setting process (see FIG. 5) according to the first embodiment, internal parameters of the camera 12 are input in advance to the elevator boarding detection system 1, and the detection area setting unit 22a determines the installation position and rotation of the camera 12. Calculate the parameters.

  On the other hand, in this embodiment, the internal parameters, installation position, and rotation parameters of the camera 12 are input in advance to the elevator boarding detection system 1. That is, by making the correspondence between the coordinate system on the image captured by the camera 12 and the coordinate system in the real space known, the processing load of the detection area setting unit 22a is reduced.

In the present embodiment, a modified example of the detection area setting process by the detection area setting unit 22a will be described with reference to FIGS.
FIG. 14 is a flowchart illustrating the flow of detection area setting processing in the present embodiment.

  In step S <b> 201, the detection area setting unit 22 a performs edge detection on the image captured by the camera 12. The edge detection process is the same as step S101 in FIG.

In step S202, the detection area setting unit 22a detects the sill A3.
Here, in the present embodiment, the detection area setting unit 22a can calculate the position of the origin O on the image captured by the camera 12 based on the internal parameters, the installation position, and the rotation parameters of the camera 12. And the detection area setting part 22a detects the square area | region below the origin O as sill A3 on the image after the edge detection obtained by the process of step S201. The search and detection of the rectangular area is performed by template matching processing or the like, similar to step S102 in FIG.

  FIG. 15 is a diagram illustrating an example of detecting a sill region by the detection area setting unit 22a. In FIG. 15, a quadrangle whose upper side is a line segment including the origin O is recognized as a sill A3.

  Thereafter, the detection area setting unit 22a detects the left frame A1 of the three-way frame and the right frame A2 of the three-way frame based on the sill A3 (step S203). Similarly, the detection area setting unit 22a detects the left front pillar A4 and the right front pillar A5 of the door pocket based on the sill A3 (step S204). Furthermore, the detection area setting unit 22a detects the detection area E (position estimation area E1, boarding) based on the detected left frame A1 of the three-way frame, right frame A2 of the three-way frame, sill A3, left front column A4 of the door pocket, and right front column A5. The intention estimation area E2, the sill area E3, and the front pillar area E4) are set. The processing in steps S203 to S205 is the same as that in steps S104 to S106 in FIG.

  According to this embodiment described above, the internal parameters, installation position, and rotation parameters of the camera 12 are given in advance to the elevator boarding detection system 1 as prior knowledge. Thus, the elevator boarding detection system 1 can calculate the position of the origin O in the real space on the image, and can accurately detect the sill A3 based on the position of the origin O on the image. That is, the detection accuracy of the left frame A1 of the three-way frame, the right frame A2 of the three-way frame, the left front column A4 and the right front column A5 of the door pocket, which are detected based on the sill A3, can be improved. Moreover, the processing load of the elevator boarding detection system 1 can be reduced by not calculating the rotation parameter.

  In addition, although some embodiment of this invention was described, these embodiment is shown as an example and is not intending limiting the range of invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 1 ... Elevator boarding detection system, 11 ... Car, 11a ... Screen board, 12 ... Camera, 13 ... Car door, 14 ... Landing door, 15 ... Landing, 20 ... Image processing device, 21 ... Memory | storage part, 22 ... Use Person detection unit, 22a ... detection area setting unit, 22b ... motion detection unit, 22c ... position estimation unit, 22d ... boarding intention estimation unit, 30 ... car control device, 31 ... door opening / closing control unit.

Claims (6)

An imaging means capable of photographing a predetermined range from the vicinity of the door of the car toward the hall when the car arrives at the hall;
Detection area setting means for setting a detection area on the image photographed by the imaging means;
User detection means for detecting the presence or absence of a user by paying attention to the movement of a person or an object in the detection area, using a plurality of time-sequential images taken by the imaging means,
Control means for controlling the opening and closing operation of the door based on the detection result of the user detection means ,
The detection area setting means includes
Detecting a sill between the car and the hall included in the image,
On the image, the area of the three-sided frame is detected based on the corner on the landing side among the corners of the sill,
An elevator boarding detection system, wherein the detection area that does not include the three-way frame region is set on the image . The detection area setting means includes
  Detecting a first line segment extending from the vicinity of the corner on the landing side to the landing side;
  Detecting second and third line segments extending perpendicularly from both ends of the first line segment and toward the edge of the image;
  An area surrounded by the first to third line segments is detected as an area of the three-sided frame
  The elevator boarding detection system according to claim 1. The detection area setting unit, on the image, based on the corners of the car side of the corner of the sill to detect an area of the pillar of the door pocket, does not include the area of the pillars of the door pocket on the image the The elevator boarding detection system according to claim 1 or 2 , wherein a detection area is set . The detection area setting means includes
  Detecting a fourth line extending from the vicinity of the car side corner to the car side;
  Detecting fifth and sixth line segments extending perpendicularly from both ends of the fourth line segment and toward the edge of the image;
  A region surrounded by the fourth to sixth line segments is detected as a column region of the door pocket.
  The elevator boarding detection system according to claim 3. The detection area setting unit calculates a three-dimensional coordinate indicating an installation position of the imaging unit in a real space based on the position of the sill and an internal parameter of the imaging unit, and based on the three-dimensional coordinate, the actual area The elevator boarding detection system according to claim 1, wherein a rotation parameter that associates the coordinate system of the space with the coordinate system of the image is calculated. The detection area setting means, three-dimensional coordinates indicating the installation position of the imaging unit in the real space, the internal parameters of the imaging means, and, based on rotation parameters associating the coordinate system of the image with the coordinate system of the real space The elevator boarding detection system according to claim 1, wherein a position of the sill is detected.

Images