JP2023024068A - User detection system of elevator - Google Patents

Abstract

To suppress erroneous detection of a shade caused by lighting environment and correctly detect a user in a car and at an elevator hall.SOLUTION: A user detection system of an elevator includes: reflection estimation means for estimating a reflection region including a part where illumination light is reflected in a photographic image of a camera installed in a car and its periphery; moving body detection means for changing processing when detecting a moving body from the photographic image according to a reflection level of the reflection region estimated by the reflection estimation means; and person detection means for detecting the moving body as a person on the basis of information on the moving body detected by the moving body detection means.SELECTED DRAWING: Figure 1

Description

Embodiments of the present invention relate to elevator occupant detection systems.

When the door of the car of the elevator is opened, the fingers of the user in the car may be drawn into the door pocket. In addition, when a user at the boarding point gets into the car, he or she may bump into the tip of the door that is in the middle of closing. In order to prevent such door accidents, a camera installed in the car is used to detect users at the boarding point and inside the car, and a system that reflects this information in door opening/closing control is developed. It is

JP-A-5-151356 Japanese Patent No. 3933453

In the system described above, the presence or absence of a user is detected based on the luminance difference between the frames of the captured image. However, in a car or a hall, when the light of illumination is reflected on the floor through the user, the amount of light incident on the camera changes, resulting in a change in brightness. At this time, a phenomenon called "shadow" occurs in and around the light-reflecting portion, and the "shadow" may be over-detected as the user (see S2 in FIG. 7). It should be noted that "excessive detection" is the same as "erroneous detection" in the sense that shadows are erroneously detected as a user.

The problem to be solved by the present invention is to provide an elevator user detection system capable of suppressing over-detection of shadows caused by the lighting environment and correctly detecting a user in a car or at a landing. is.

An elevator user detection system according to one embodiment includes reflection estimation means for estimating a reflection area including a portion where illumination light is reflected and its surroundings in an image captured by a camera installed in a car; moving object detection means for changing processing when detecting a moving object from the photographed image according to the reflection level of the reflection area estimated by the reflection estimation means; and moving object information detected by the moving object detection means. a person detection means for detecting the moving object as a person based on the above.

FIG. 1 is a diagram showing the configuration of an elevator user detection system according to an embodiment. FIG. 2 is a diagram showing the configuration of the entrance and exit peripheral portion in the car in the same embodiment. FIG. 3 is a diagram for explaining a coordinate system in real space in the embodiment. FIG. 4 is a diagram showing an example of an image captured by a camera in the same embodiment. FIG. 5 is a diagram schematically showing the configuration of a boarding detection area in the same embodiment. FIG. 6 is a diagram for explaining erroneous detection of a shadow that is drawn into the detection area in the same embodiment. FIG. 7 is a diagram for explaining shadows caused by reflection of illumination light in the same embodiment. FIG. 8 is a diagram showing an example of a reflective area in the same embodiment. FIG. 9 is a diagram showing an example of a photographed image including a person and a shadow in the same embodiment. FIG. 10 is a diagram showing changes in the luminance value of a person on the photographed image of FIG. 9 as viewed in the x-axis direction. FIG. 11 is a diagram showing changes in luminance values of shadows on the photographed image of FIG. 9 as viewed in the x-axis direction. FIG. 12 is a diagram for explaining a method of calculating the strength of a mountain-shaped edge in the same embodiment. FIG. 13 is a diagram showing a concrete example of calculating the strength of the mountain-shaped edge. FIG. 14 is a flow chart showing the processing operation of the user detection system. FIG. 15 is a flow chart showing the detection process executed in step S103 of FIG. 14 above.

Embodiments will be described below with reference to the drawings.
The disclosure is merely an example, and the invention is not limited by the contents described in the following embodiments. Modifications that can be easily conceived by those skilled in the art are naturally included in the scope of the disclosure. In order to make the explanation clearer, in the drawings, the size, shape, etc. of each part may be changed from the actual embodiment and shown schematically. Corresponding elements in multiple drawings may be denoted by the same reference numerals and detailed descriptions thereof may be omitted.

FIG. 1 is a diagram showing the configuration of an elevator user detection system according to an embodiment. Here, one car will be described as an example, but the configuration is the same for a plurality of cars.

A camera 12 is installed above the doorway of the car 11 . Specifically, the camera 12 is installed in the curtain plate 11a that covers the upper part of the doorway of the car 11 with the lens portion tilted directly below, or toward the hall 15 side or inside the car 11 by a predetermined angle. .

The camera 12 is, for example, a small surveillance camera such as an in-vehicle camera, has a wide-angle lens or a fish-eye lens, and is capable of continuously photographing several frames per second (for example, 30 frames/second). The camera 12 is activated, for example, when the car 11 arrives at the landing 15 of each floor, and photographs the vicinity of the car door 13 and the landing 15 as well. Note that the camera 12 may always be in operation while the car 11 is running.

The shooting range at this time is adjusted to L1+L2 (L1>>L2). L1 is an imaging range on the landing side, and has a predetermined distance from the car door 13 toward the landing 15 . L2 is an imaging range on the car side, and has a predetermined distance from the car door 13 toward the back of the car. Note that L1 and L2 are ranges in the depth direction, and the range in the width direction (direction perpendicular to the depth direction) is at least greater than the width of the car 11 .

At the landing 15 of each floor, a landing door 14 is installed at the arrival gate of the car 11 so as to be openable and closable. The landing door 14 engages with the car door 13 to open and close when the car 11 arrives. The power source (door motor) is located on the car 11 side, and the landing door 14 simply follows the car door 13 to open and close. In the following description, it is assumed that when the car door 13 is open, the landing door 14 is also open, and when the car door 13 is closed, the landing door 14 is also closed.

Each image (video) captured continuously by the camera 12 is analyzed and processed in real time by the image processing device 20 . Although FIG. 1 shows the image processing device 20 removed from the car 11 for the sake of convenience, the image processing device 20 is actually housed in the curtain plate 11a together with the camera 12. As shown in FIG.

The image processing device 20 includes a storage section 21 and a detection section 22 . The storage unit 21 is composed of a memory device such as a RAM, for example. The storage unit 21 sequentially stores images captured by the camera 12 and has a buffer area for temporarily storing data required for processing by the detection unit 22 . Note that the storage unit 21 may store an image that has undergone processing such as distortion correction, enlargement/reduction, and partial cutout as preprocessing for the captured image.

The detection unit 22 is composed of, for example, a microprocessor, and uses the image captured by the camera 12 to detect a user in the car 11 or at the hall 15 . The detection unit 22 is functionally divided into a detection area setting unit 23 and a detection processing unit 24 . Note that these may be realized by software, hardware such as an IC (Integrated Circuit), or both software and hardware. Alternatively, the elevator control device 30 may have a part or all of the functions of the image processing device 20 .

The detection area setting unit 23 sets at least one or more detection areas for detecting the user on the captured image obtained from the camera 12 . In this embodiment, a detection area E1 for detecting a user of the hall 15 and detection areas E2 and E3 for detecting a user inside the car 11 are set. The detection area E<b>1 is used as a boarding detection area and is set from the doorway (car door 13 ) of the car 11 toward the hall 15 . The detection area E<b>2 is used as a retraction detection area and is set at the entrance pillars 41 a and 41 b inside the car 11 . The detection area E3 is drawn in and used as a detection area in the same manner as the detection area E2, and is set on the floor surface 19 on the entrance/exit side of the car 11 (see FIG. 3).

The detection processing unit 24 includes an edge extraction unit 24a, a moving object detection unit 24b, a person detection unit 24c, and a reflection estimation unit 24d, and analyzes the photographed image obtained from the camera 12 to detect the inside of the car 11 or the landing 15. Detect existing users. The edge extraction unit 24a, the moving object detection unit 24b, the person detection unit 24c, and the reflection estimation unit 24d will be described later in detail with reference to FIGS. 7 to 13. FIG. When the user detected by the detection processing unit 24 exists in one of the detection areas E1 to E3, a predetermined corresponding process (door opening/closing control) is executed.

The elevator control device 30 is composed of a computer having a CPU, a ROM, a RAM, and the like. The elevator control device 30 performs operation control of the car 11 and the like. The elevator control device 30 also includes a door opening/closing control section 31 and a warning section 32 .

The door opening/closing control unit 31 controls opening/closing of the car door 13 when the car 11 arrives at the hall 15 . Specifically, the door opening/closing control unit 31 opens the car door 13 when the car 11 arrives at the hall 15, and closes the car door 13 after a predetermined time has elapsed. However, when the user is detected within the detection area E1 by the detection processing unit 22b during the closing operation of the car door 13, the door opening/closing control unit 31 prohibits the closing operation of the car door 13. , the car door 13 is reopened in the fully open direction to maintain the door open state.

Further, when the detection processing unit 22b detects a user within the detection area E2 or E3 during the opening operation of the car door 13, the door opening/closing control unit 31 detects a door accident (an accident caused by being pulled into the door pocket). Door opening/closing control is performed to avoid Specifically, the door opening/closing control unit 31 temporarily stops the door opening operation of the car door 13, moves the car door 13 in the opposite direction (door closing direction), or slows down the door opening speed of the car door 13.

FIG. 2 is a diagram showing the configuration of the entrance and exit peripheral portion in the car 11. As shown in FIG.
A car door 13 is provided at the doorway of the car 11 so as to be freely opened and closed. In the example of FIG. 2, a car door 13 of a double-opening type is shown, and two door panels 13a and 13b constituting the car door 13 open and close in opposite directions along the frontage direction (horizontal direction). Note that the “frontage” is the same as the doorway of the car 11 .

Entrance pillars 41a and 41b are provided on both sides of the doorway of the car 11, and surround the doorway of the car 11 together with the curtain plate 11a. The "entrance pillar" is also called a front pillar, and is generally provided with a door pocket for housing the car door 13 on the back side. In the example of FIG. 2, when the car door 13 is opened, one door panel 13a is accommodated in the door pocket 42a provided on the back side of the entrance pillar 41a, and the other door panel 13b is provided on the back side of the entrance pillar 41b. It is housed in the door pocket 42b. One or both of the entrance pillars 41a and 41b are provided with a display 43, an operating panel 45 having destination floor buttons 44, and a speaker 46. As shown in FIG. In the example of FIG. 2, a speaker 46 is installed on the entrance pillar 41a, and a display 43 and an operation panel 45 are installed on the entrance pillar 41b.

The camera 12 is provided in a panel 11a horizontally arranged above the doorway of the car 11. - 特許庁Here, a camera 12 is attached in accordance with the closed position of the car door 13 in order to detect the user of the hall 15 until just before the door closes. Specifically, if the car door 13 is of a double-opening type, the camera 12 is attached to the central portion of the curtain plate 11a. A lighting device 48 using, for example, an LED is installed on the ceiling surface inside the car 11 .

As shown in FIG. 3, the camera 12 measures the direction horizontal to the car door 13 provided at the entrance of the car 11 as the X axis, and the direction from the center of the car door 13 to the landing 15 (vertical to the car door 13). direction) is taken as the Y-axis, and the height direction of the car 11 is taken as the Z-axis.

FIG. 4 is a diagram showing an example of an image captured by the camera 12. As shown in FIG. The upper side is the landing 15 and the lower side is the inside of the car 11 . In the figure, 16 indicates the floor of the landing 15 and 19 indicates the floor of the car 11 . E1, E2, and E3 represent detection areas.

The car door 13 has two door panels 13a, 13b that move on the car sill 47 in opposite directions. The landing door 14 is similar and has two door panels 14a, 14b that move in opposite directions on the landing sill 18. As shown in FIG. The door panels 14a and 14b of the landing door 14 move together with the door panels 13a and 13b of the car door 13 in the door opening/closing direction.

A camera 12 is installed above the doorway of the car 11 . Therefore, when the car 11 is opened at the landing 15, as shown in FIG. 1, a predetermined range (L1) on the landing side and a predetermined range (L2) inside the car are photographed. Among them, a detection area E1 for detecting a user getting on the car 11 is set in a predetermined range (L1) on the platform side.

In the real space, the detection area E1 has a distance of L3 from the center of the doorway (frontage) toward the hall (L3 ≤ hall-side shooting range L1). The width W1 of the detection area E1 when fully open is set to a distance equal to or greater than the width W0 of the entrance (frontage). The detection area E1 includes the sills 18 and 47 and is set excluding blind spots of the three-sided frames 17a and 17b, as indicated by diagonal lines in FIG. The size of the detection area E<b>1 in the horizontal direction (X-axis direction) may be changed in accordance with the opening/closing operation of the car door 13 . Further, the size of the detection area E1 in the vertical direction (Y-axis direction) may also be changed according to the opening/closing operation of the car door 13 .

As shown in FIG. 5, the detection area E1 used as the boarding detection area consists of a boarding intention estimation area E1a, a proximity detection area E1b, and an on-sill detection area E1c. The boarding intention estimation area E1a is an area for estimating whether or not the user is heading toward the car 11 with the boarding intention. The proximity detection area E1b is an area for detecting that the user is approaching the doorway of the car 11 . The on-sill detection area E1c is an area for detecting that the user is passing over the sills 18,47.

Here, in this system, detection areas E2 and E3 are provided separately from the detection area E1 for boarding detection. The detection areas E2 and E3 are drawn in and used as detection areas. The detection area E2 is set along the inner side surfaces 41a-1 and 41b-1 of the entrance pillars 41a and 41b of the car 11 with a predetermined width. Note that the detection area E2 may be set according to the width of the inner side surfaces 41a-1 and 41b-1. The detection area E3 is set with a predetermined width along the car sill 47 on the floor 19 of the car 11 .

When the user is detected within the detection area E2 or E3 during the opening operation of the car door 13, for example, the opening operation of the car door 13 is temporarily stopped, moved in the opposite direction (door closing direction), or , a corresponding process such as slowing down the opening speed of the car door 13 is executed. In addition, a warning, such as "Keep away from the door", is issued by voice announcement.

(Problem of detection processing)
Usually, the attraction detection is based on the premise that the change in brightness of the image in the detection areas E2 and E3, which are the attraction detection areas, is correctly displayed by the user's intrusion. However, since the detection areas E2 and E3 are set within the car 11, they are strongly affected by the lighting environment inside the car. In other words, as shown in FIG. 6, even when the user P1 gets on the car at a place away from the car door 13, the shadow S1 of the user P1 is in the detection area due to the illumination light of the lighting device 48. It may enter E2 or E3. If the shadow S1 enters the detection area E2 or E3, there is a possibility that the shadow S1 will be over-detected as the user P1 due to a large change in luminance on the image as the shadow S1 moves.

This also applies to boarding detection processing. That is, the detection area E<b>1 that is the boarding detection area is set at the landing 15 around the entrance/exit of the car 11 . If a shadow enters the detection area E1 due to the lighting environment of the hall 15, there is a possibility that the shadow will be over-detected due to a change in luminance on the image.

・Brightness change due to shadows Fig. 7 is a diagram for explaining shadows caused by reflection of illumination light. 11 shows a case where the user P1 is present.

As shown in FIG. 7(a), when there is no user P1 in the car 11, when the light emitted from the lighting equipment 48 is reflected on the floor surface 19, almost the same amount of light is reflected as the emitted light. Light is incident on camera 12 . However, as shown in FIG. 7B, when the user P1 stands between the lighting device 48 and the floor surface 19, the light from the lighting device 48 is reflected by the floor surface 19 through the user P1. Therefore, the amount of reflected light incident on the camera 12 changes, resulting in a luminance change. At this time, a phenomenon called "shadow" occurs near the user P1, and this shadow may be over-detected as the user P1. S2 in the figure is the shadow. The shadow S1 shown in FIG.

The same applies to the hall 15. If a shadow due to reflected light is generated near the user due to the lighting environment of the hall 15, the shadow may be over-detected as the user. In particular, when a downlight is used as the lighting equipment, the floor surface is locally illuminated, so shadows due to reflected light are likely to occur.

Therefore, in this embodiment, the detection processing unit 24 of the image processing apparatus 20 shown in FIG. When the user is detected using the edge change between images (between frames) obtained in the above, a reflection area in which a shadow exists is estimated, and detection processing in the reflection area is changed. "Edge change" refers to a change in an edge extracted from the same position between images. "Edge change" includes "edge difference" which is the difference in edge strength. In the following, the functions (edge extraction, moving object detection, human detection, and reflection estimation section) provided in the detection processing section 24 will be described in detail, taking a case of obtaining an edge difference as an example of edge change.

(a) Reflection Estimation First, reflection estimation will be described.
Reflection estimation is one of the functions necessary to suppress false detection of "shadow". Since the place where the "shadow" occurs cannot be specified, the reflection estimation unit 24d estimates a reflection area including a part where the illumination light is reflected in the captured image and its surroundings (floor surface and shadow). The reflective area is estimated by analyzing variations in luminance value (luminance distribution) of each pixel in the captured image. This is because if only the luminance value is focused, for example, when there is a person wearing white clothes, it is impossible to distinguish between the reflective area and the white clothes.

The reflective area is estimated by analyzing the luminance distribution of each pixel in a range of 13×13 pixels, for example, using one image or a plurality of images. The analysis range of the luminance distribution may be fixed, or may be automatically changed according to parameter settings or an object to be photographed.

The analysis range of the luminance distribution may be changed depending on the height of the installation position of the camera 12, the type of lighting equipment, and the like. As the height of the camera 12 is farther from the floor, there is a tendency for the occurrence of "shadows" to expand. Therefore, it is preferable to analyze the luminance distribution of each pixel over a wide range. Further, when a downlight is used as the lighting equipment, "shadows" are likely to occur, so it is preferable to analyze the luminance distribution of each pixel over a wide range.

Moreover, when estimating the reflection area, the edge distribution may be analyzed. Since the edge distribution also reflects the variation in the luminance value of each image, the reflection area can be estimated in the same manner as the luminance distribution. A method for extracting edges will be described later.

FIG. 8 is a diagram showing an example of a reflective area. Reference numeral 51 in the figure denotes a person, specifically a user in the car 11 .

The reflection area RE includes a plurality of areas RE-1 to RE-4 with different reflection levels. Regions containing more bright luminance values have a higher reflection level. In the example of FIG. 8, the reflection level of the reflection area RE existing near the person 51 is classified into 0 to 4 stages (0<1<2<3<4, 0 means no reflection).
Region RE-1: reflection level 4
Region RE-2: Reflection level 3
Region RE-3: Reflection level 2
Region RE-4: Reflection level 1

In the example of FIG. 8, the regions RE-1 to RE-4 are schematically shown for the sake of simplicity of explanation, but in reality the form will be more complicated. Normally, within the car 11, since the irradiated light is strongly reflected by metal parts such as the car sill 47, the reflection level is high. However, if the shadow of the person 51 occurs around the portion where the illumination light is reflected, the reflection level will not be uniform due to the shadow. For example, when the light of illumination is reflected on the floor through the person 51, the light passing through the gaps between the fingers of the person 51 and the light blocked by the palm of the person 51 may affect the surroundings of the person 51. The reflection level of the changes in a complicated manner.

In this way, a portion where the reflection level changes near the person 51 , that is, a portion where a shadow occurs, causes a luminance change, and the shadow may be over-detected as the person 51 . Therefore, in the present embodiment, in order to suppress such over-detection of shadows, an area near the person 51 in which luminance values vary is estimated as a reflection area RE in which shadows exist. The detection processing (moving object detection processing, which will be described later) in the RE is changed.

(b) Edge Extraction Edge extraction is a function necessary to suppress erroneous detection of "shadow", and is not necessarily required when focusing only on erroneous detection of "shadow". The edge extraction unit 24 a extracts edge information from the image captured by the camera 12 . In this case, edge information may be extracted from one image or from a plurality of images. An "edge" is a boundary line where the luminance value of each pixel of an image changes discontinuously. For example, an edge extraction filter such as a Sobel filter or a Laplacian filter is used to extract, as an edge, a portion in which the luminance value characteristically changes on the image. The edge information includes the direction and strength of the luminance gradient.

Edge strength is determined by a luminance gradient. The range for obtaining the luminance gradient may be, for example, a range of 3×3 pixels, or may be obtained in a range other than that. Also, the range for obtaining the luminance gradient may be fixed, or may be automatically changed according to parameter settings or an object to be photographed.

・Combination of direction and strength of luminance gradient The edge extracting unit 24a obtains the direction and strength of the luminance gradient for each pixel of the captured image, and extracts edges excluding shadow regions based on the combined information. . In addition to the four directions (horizontal and vertical directions) of top-to-bottom, bottom-to-top, left-to-right, and right-to-left, there are four directions of luminance gradient: upper-left to lower-right, lower-left to upper-right, upper-right to lower-left, and lower-right to There are four directions (diagonal directions) to the upper left. In order to suppress over-detection of shadows, it is preferable to obtain luminance gradients in at least two directions.

It should be noted that it is also possible to extract edges for which co-occurrence is established. For example, it is also possible to extract edges having luminance gradient directions in the left and right directions with respect to the pixel of interest. The edge strength is calculated by averaging luminance values in each selected direction.

Mountain-Shaped Edge The edge extraction unit 24a extracts an edge whose luminance value changes in a mountain-like shape as an edge excluding the shadow region.

FIG. 9 is a diagram showing an example of a photographed image including a person and a shadow. Reference numeral 51 in the figure denotes a person, specifically a user in the car 11 . Reference numeral 52 in the drawing denotes a shadow that appears on the floor inside the car 11, and schematically represents a shadow of the hand of the person 51 protruding forward. FIG. 10 is a diagram showing changes in luminance values of an image 53 corresponding to a person 51 as seen in the x-axis direction. FIG. 11 is a diagram showing changes in the luminance value of the image 54 corresponding to the shadow 52 as seen in the x-axis direction.

As shown in FIG. 10, an image 53 corresponding to a person 51 has many edges whose luminance values change discontinuously due to the fingers of the person 51, creases in clothes, and the like. On the other hand, as shown in FIG. 11, the change in the brightness value inside the image 54 corresponding to the shadow 52 is flat, and the brightness value changes at the boundary, but the direction of the brightness gradient is unidirectional. Therefore, in order to suppress over-detection of the shadow 52, an edge having a combination of luminance gradients and their intensities in two or more directions and whose luminance value changes continuously in a mountain-like shape (hereinafter referred to as a mountain-shaped edge) ) is effective. By performing edge extraction focusing on such mountain-shaped edges, it is possible to efficiently extract edges other than the shadow area from the captured image. Detection processing unaffected by movement can be realized.

12 and 13, a method for calculating the strength of the mountain-shaped edge will be described.
For example, a pixel located in the center of a range of 3×3 pixels is taken as a pixel of interest, and luminance differences in four directions of up, down, left, and right are obtained for the pixel of interest. A value obtained by averaging these luminance differences is calculated as the intensity of the mountain-shaped edge.

Assume that the luminance value of the pixel of interest is "191" in 256 gradations. The luminance value of the pixel positioned above the pixel of interest is "0 (black)", the luminance value of the pixel positioned to the right of the pixel of interest is "64", and the luminance value of the pixel positioned below the pixel of interest is "127". , the luminance value of the pixel positioned to the left of the pixel of interest is "255 (white)", the intensity of the mountain-shaped edge is obtained by the following calculation.
{(191-0)+(191-64)+(191-127)+0}/4=95.5
Note that the luminance value of the pixel positioned to the left of the pixel of interest is greater than that of the pixel of interest, and is therefore calculated as "0." According to the above formula, the edge intensity at the position of the pixel is obtained as "96 (95.5 normalized to an integer)".

(c) Moving Object Detection Moving object detection is a function of detecting an object that has some kind of movement on a captured image. Usually, a method of detecting the presence or absence of a moving object based on a luminance change (luminance difference) between images is used. However, since luminance changes occur even in a portion where a "shadow" occurs, there is a possibility that that portion will be over-detected as a moving object (that is, a user).

Therefore, in the present embodiment, the moving object detection is performed using the edge difference. The moving object detection unit 24b compares the edges extracted by the edge extraction unit 24a between images continuously obtained as captured images to obtain edge differences, and detects a moving object based on the edge differences.

"Edge difference" is specifically a difference in edge intensity. In the example of FIG. 13, assume that the edge strength of the target pixel of the first image is calculated as "96". If the edge strength of the same pixel of interest in the next image is "10", the edge strength difference is 96-10=86. For example, if the threshold value is "40", "86" is greater than or equal to the threshold value, so it is determined that there is movement at the pixel of interest.

As another method, it is also possible to obtain the difference after binarizing the edge strength.
For example, when the threshold value is "40", edge strength "96" is binarized to "255" and edge strength "40" is binarized to "0". The difference between the two is 255-0=255, which is not "0", so it is determined that there is movement.

In the example of FIG. 9, 55 in the figure indicates pixels (moving pixels) determined to have motion. The image 53 of the person 51 has many motion pixels 55 in the hands and clothes, but the image 54 of the shadow 52 has no motion pixels 55 . As will be described later, it can be determined from the distribution of the motion pixels 55 whether or not the moving object is a person.

- Edge difference and luminance difference Moving object detection may be performed using both the edge difference and the luminance difference. In this case, in addition to the edge difference, the moving object detection unit 24b obtains the luminance difference (difference in luminance value) between the images successively obtained as the captured images, and detects the moving object based on the luminance difference and the edge difference. detect. As a method of integrating the result of the edge difference and the result of the luminance difference, there are the following logical operation (AND/OR operation, etc.), parameter change, and the like.

AND operation: When a motion pixel on the image is detected by both the edge difference and the luminance difference, it is determined that a moving object exists in a predetermined range including the motion pixel.

OR operation: The luminance difference is used for areas with many edges (areas with low possibility of being shadowed), and the edge difference is used for areas with few edges (areas with high possibility of being shadowed). A “region with many edges” is a region in which the number of edges (the number of pixels) extracted by the edge extraction unit 24a is equal to or greater than a prescribed number defined as a shadow determination criterion. A “area with few edges” is an area in which the number of edges (the number of pixels) extracted by the edge extracting unit 24a is smaller than the specified number defined as the shadow determination criterion.

Parameter change: For areas with many edges (areas with little possibility of shadows), the luminance difference parameter is made easier to detect (that is, the luminance difference threshold is set lower than the standard value), and for areas with few edges For (regions with a high possibility of being shadowed), the parameter of the luminance difference is made difficult to detect (that is, the threshold of the luminance difference is set higher than the standard value).

・Moving object detection considering reflective areas Not only “shadows” but also “shadows” caused by illumination light are factors of brightness changes. Therefore, if a moving object is simply detected based on only edge changes, there is a possibility that the shaded portion may be over-detected as a moving object (that is, the user). As described with reference to FIG. 8, the area where the shadow occurs on the captured image is estimated by the reflection area including the part where the illumination light is reflected and its surroundings.

The moving object detection unit 24b changes the processing when detecting a moving object from the captured image according to the reflection level of the reflection area estimated by the reflection estimation unit 24d. Specifically, "changing the processing when detecting a moving object from a captured image" means changing the threshold value for the edge difference. In this case, an area with a higher reflection level (that is, a brighter area) is considered to be an area where overdetection due to shadows is more likely to occur. Therefore, the threshold for the edge difference is raised above the standard value to make it difficult to detect the object as a moving object.

For example, if the threshold value for the edge difference is TH1, TH1 is increased by one step in the region RE-4 of reflection level 1 shown in FIG. Similarly, in the region RE-3 of the reflection level 2, TH1 is raised by two steps, in the region RE-3 of the reflection level 3, TH1 is raised by three steps, and in the region RE-4 of the reflection level 4, TH1 is raised by four steps.

Note that even when the moving object is detected using only the luminance difference, the threshold for the luminance difference may be changed stepwise according to the reflection level. As a result, the determination of the luminance difference for the region with a high reflection level becomes stricter, and overdetection due to shadows can be suppressed.

Moreover, it is good also as a structure which detects a moving body using both a luminance difference and an edge difference. In this case, the moving object detection unit 24b selectively uses the luminance difference and the edge difference according to the reflection level of the reflection area. In other words, when the reflection level of the reflection area is lower than a certain level (an area where overdetection due to shadows is unlikely), the luminance difference is used, and when the reflection level of the reflection area is above a certain level (overdetection due to shadows is unlikely) moving object detection is performed using the edge difference in the area where it is likely to occur). The "constant level" can be arbitrarily set, and may be level 3 among reflection levels 0 to 4, for example.

(d) Person Detection The person detection unit 24c detects the moving object as a person based on the information on the moving object detected by the moving object detection unit 24b. A “person” specifically means a user present in the car 11 or at the hall 15 . The “moving object information” includes at least one of motion pixel distribution, moving object size, and number of times of moving object detection.

“Motion pixel distribution” indicates the distribution state of motion pixels within a predetermined range. For example, if there are 40 motion pixels (that is, about 10%) or more in a range of 20×20 pixels, it is determined that the motion is that of a person. "Moving object size" indicates the size of an aggregate of continuous motion pixels. For example, if 40 or more motion pixels exist as a continuous aggregate, it is determined that the motion is that of a person. “Number of moving object detections” indicates the number of times each image is detected as a moving object. For example, if the same position on the image is detected as a moving object more than a certain number of times, it is determined that the movement is that of a person.

- Edge information and moving object information A configuration may be adopted in which the edge information and the moving object information are used together to detect a person. In this case, the human detection unit 24c changes the determination criteria for human detection using any one of the motion pixel distribution obtained as the moving object information, the moving object size, and the number of times of moving object detection based on the edge information. detect.

Specifically, the human detection unit 24c makes the distribution of motion pixels or the size of the moving object smaller for areas with many edges (areas with less possibility of shadows) in the captured image than areas with few edges. Perform person detection. Alternatively, for areas with many edges in the captured image (areas with little possibility of shadows), the number of moving object detections is set to be lower than for areas with few edges. You can judge.

- Change the parameter for human detection according to the reflection level The parameter for human detection may be changed according to the reflection level of the reflection area estimated by the reflection estimation unit 24d. For example, when the reflection level of the reflective area is above a certain level (an area where over-detection due to shadows is likely to occur), human detection is performed by setting the motion pixel distribution or moving object size judgment criteria higher than those in other areas. . In addition, when the reflection level of a reflective area is above a certain level (an area where over-detection due to shadows is likely to occur), the determination criteria for the number of moving object detections is set higher than for other areas, and a moving object is detected more than a certain number of times. is determined to be a person. The "constant level" can be arbitrarily set, and may be level 3 among reflection levels 0 to 4, for example.

This system uses the detection processing unit 24 having the configuration described above to detect a person (user) from the captured image, and the person exists in one of the detection areas E1 to E3 shown in FIG. If so, a predetermined response process (door opening/closing control) is executed. The processing operation of this system will be described below using the detection of being pulled in as an example.

FIG. 14 is a flow chart showing the processing operation of this system. The processing shown in this flowchart is executed by the image processing device 20 and the elevator control device 30 shown in FIG.

First, as an initial setting, detection area setting processing is executed by the detection area setting unit 23 of the detection unit 22 provided in the image processing apparatus 20 (step S100). This detection area setting process is executed as follows, for example, when the camera 12 is installed or when the installation position of the camera 12 is adjusted.

That is, the detection area setting unit 22a sets a detection area E1 having a distance L3 from the entrance toward the landing 15 with the car door 13 fully opened. As shown in FIG. 4, the detection area E1 includes the sills 18 and 47 and is set excluding the blind spots of the three-sided frames 17a and 17b. Here, when the car door 13 is fully open, the size of the detection area E1 in the lateral direction (X-axis direction) is W1, and has a distance equal to or greater than the lateral width W0 of the entrance (frontage). Further, the detection area setting unit 22a sets a detection area E2 having a predetermined width along the inner side surfaces 41a-1 and 41b-1 of the entrance pillars 41a and 41b of the car 11, and also sets a detection area E2 having a predetermined width. A detection area E3 having a predetermined width is set along the car sill 47 of the surface 19. As shown in FIG.

During normal operation, when the car 11 reaches the landing 15 of an arbitrary floor (Yes in step S101), the elevator control device 30 starts opening the car door 13 (step S102). Along with this door opening operation, the camera 12 photographs a predetermined range (L1) on the landing side and a predetermined range (L2) in the car at a predetermined frame rate (eg, 30 frames/second). It should be noted that the camera 12 may take pictures continuously from the state where the door of the car 11 is closed.

The image processing device 20 acquires images captured by the camera 12 in time series, and while sequentially storing these images in the storage unit 21, performs the following detection processing (pull-in detection processing) in real time. (Step S103). It should be noted that distortion correction, enlargement/reduction, cutout of a part of the image, and the like may be performed as preprocessing for the captured image.

FIG. 15 shows the detection process executed in step S103. This detection processing is executed by the detection processing unit 24 of the image processing device 20 . In the following description, it is assumed that mountain-shaped edges are extracted from a photographed image.

First, in order to eliminate the influence of “shadow” included in the captured image, when the captured image (original image) of the camera 12 is acquired (step S201), the detection processing unit 24 sets the luminance of each pixel of the captured image. By analyzing the value distribution, an image representing the luminance distribution (hereinafter referred to as a luminance distribution variation image) is created (step S202). As shown in FIG. 8, the reflection area RE has a mixture of bright luminance values of the reflected portion of the illumination light and dark luminance values of the floor and shadows, and the luminance distribution varies greatly.

The detection processing unit 24 estimates the reflection area RE using the luminance distribution variation image (step S302). The detection processing unit 24 creates an edge difference threshold image based on the reflection levels of the regions RE-1 to RE4 included in the reflection region RE (step S303). The “edge difference threshold image” is an image in which the threshold TH1 used when binarizing the edge difference is expressed in units of pixels. The threshold TH1 is set according to the reflection level of each region RE-1 to RE4. In this case, the higher the reflection level, the more likely the area is to be over-detected due to shadows, so the threshold TH1 is set higher than the standard value. Conversely, the lower the reflection level, the less likely the area is to be overdetected due to shadows, so the threshold TH1 is set lower than the standard value.

When the threshold value TH1 is set according to the reflection level of each of the regions RE-1 to RE4 in this way, the detection processing unit 24 acquires each image (original image) from the storage unit 21 in chronological order (step S201), and for each of these images, an image composed only of chevron-shaped edges is created (step S202). More specifically, the detection processing unit 24 extracts an edge that has a combination of two or more luminance gradient directions and intensities and whose luminance value changes in a mountain-like shape as a mountain-shaped edge. An image (hereinafter referred to as a chevron-shaped edge image) is created.

Subsequently, the detection processing unit 24 performs differential binarization of the mountain-shaped edge image using the threshold image created in step S303 (step S203). Specifically, as described with reference to FIG. 13, the detection processing unit 24 obtains the luminance gradient for each pixel of the mountain-shaped edge image, and compares the intensity of the luminance gradient at the same pixel position in the next image. Find the difference. The detection processing unit 24 acquires the threshold TH1 set for each reflection area from the threshold image, and binarizes the edge difference based on the threshold TH1. In this case, since the threshold value TH1 is set high in areas where the reflection level is high (areas where over-detection due to shadows is likely to occur), over-detection due to shadows is suppressed, and differential binarization of mountain-shaped edge images is performed. It can be carried out.

Further, the detection processing unit 24 performs differential binarization of the original image, which is the captured image (step S204). Specifically, the detection processing unit 24 compares the luminance value of each pixel of the image at the same pixel position of the next image to obtain a luminance difference, and binarizes the luminance difference with a preset threshold TH2. As with the threshold TH1, the threshold TH2 at this time may also be set to a value corresponding to the reflection level by reflecting the reflection level of the reflection area.

The detection processing unit 24 integrates the value obtained by binarizing the edge difference of each pixel obtained from the mountain-shaped edge image and the value obtained by binarizing the luminance difference of each pixel obtained from the original image (step S205), the presence or absence of a moving object is detected from the result of the integrated processing (step S206). Methods for integrating the edge difference and the luminance difference include logical operation (AND/OR operation, etc.) and parameter change, as described above.

When a moving object (moving pixels) is detected in this manner, the detection processing unit 24 detects a person based on information on the moving object (step S207). Specifically, the detection processing unit 24 determines whether or not the moving object is the movement of a person, based on at least one of the motion pixel distribution, the moving object size, and the number of moving object detections obtained as moving object information. . For example, in the case of detecting a person based on the distribution of motion pixels, if about 10% or more motion pixels exist in a predetermined pixel range, the human detection unit 24c detects the motion of the person in the range including the motion pixels. I judge.

Further, the detection processing unit 24 changes parameters for human detection according to the reflection level of the reflection area. For example, when the reflection level of the reflection area is a certain level or higher (area where over-detection due to shadows is likely to occur), human detection is performed by increasing the motion pixel distribution or moving object size determination criteria. Alternatively, the determination criterion for the number of moving body detections is set high, and when a moving body is detected a certain number of times or more, it is determined to be a person.

In this embodiment, the "person" means a user in the car 11 or at the boarding hall 15, and on the photographed image, the movement of the user's clothes or hands appears as motion pixels (FIG. 7). reference).

In the example of FIG. 15, the edge difference and the luminance difference are used in combination, but it is also possible to perform the moving object detection processing using only the edge difference and detect a person (user) from the distribution of motion pixels obtained as a detection result. . In this case, the processing of steps S204 and S205 of FIG. 15 becomes unnecessary.

Returning to FIG. 14, when the user is detected by the above-described detection processing during the door opening operation, the detection processing unit 24 detects the user drawn into the car 11 and the detection area E2 or the detection area set as the detection area. It is determined whether or not the object is within the detection area E3 (step S104). If the user is in the detection area E2 or the detection area E3 (Yes in step S104), the detection processing unit 24 is pulled in to the elevator control device 30 and a detection signal is output. As a result, the elevator control device 30 suspends the door-opening operation of the car door 13 through the door opening/closing control unit 31 as a corresponding process related to the detection area of being pulled in, and after a few seconds resumes the door-opening operation from the stop position. (Step S105).

As the countermeasure, the door opening speed of the car door 13 may be made slower than usual, or the car door 13 may be slightly moved in the opposite direction (door closing direction) before resuming the door opening operation. Also, by starting the warning unit 32 of the elevator control device 30, a voice announcement may be made through the speaker 46 in the car 11 to alert the user to move away from the car door 13. (step S106). The above process is repeated while the user is detected in the detection area E2 or the detection area E3. As a result, for example, when the user is near the car door 13, it is possible to prevent the user from being pulled into the door pocket 42a or 42b.

(boarding detection processing)
In the example of FIG. 14, the pulled-in detection process has been described as an example, but the boarding detection process is the same. That is, when the car 11 starts closing the door on an arbitrary floor, the detection process described with reference to FIG. 15 is executed. In this case, considering the "shadow" caused by the lighting environment of the hall 15, the reflection area including the reflection part of the illumination light in the hall 15 and its surroundings is estimated, and the edge difference threshold value is calculated according to the reflection level of the reflection area. be set. Note that the luminance difference threshold may also be set according to the reflection level of the reflection area.

As described above, when the user is detected based on the edge difference and the luminance difference of the captured image, it is determined whether or not the user is within the detection area E1 set as the boarding detection area in the hall 15 . When it is detected that the user is in the detection area E1 and is facing the door 13 of the car 11, the detection processing unit 24 outputs a boarding detection signal to the elevator control device 30. . As a result, the elevator control device 30 temporarily stops the door closing operation of the car door 13 through the door opening/closing control unit 31, or moves the car door 13 in the opposite direction (door closing direction) as a corresponding process related to the boarding detection area. or lower the opening speed of the car door 13 than usual.

As described above, according to the present embodiment, by estimating the reflection area including the reflection portion of the illumination light and its surroundings and changing the moving object detection processing according to the reflection level of the reflection area, the shadow caused by the illumination light can be detected. It is possible to suppress the over-detection of the user and detect the user.

In particular, when the detection process is performed using the edge difference, by setting the threshold value of the edge difference according to the reflection level of the reflection area, the influence of the shadow is reliably eliminated and the user is detected correctly. It is possible to realize a corresponding process according to the detection result.

In the above embodiment, the user is detected from the entire photographed image. However, the user may be detected for each detection area preset on the photographed image. For example, if the door is being opened, attention is focused on the images within the detection areas E2 and E3 shown in FIG. 4, and the user in the detection areas E2 or E3 is detected from the edge difference of the images. If the door is being closed, attention is paid to the image within the detection area E1 shown in FIG. 4, and the user within the detection area E1 is detected from the edge difference of the image.

Further, in the above-described embodiment, edge difference (difference in edge intensity) was used as an example of edge change, but edge change may be determined using a rectangle such as normalized correlation. In short, any method other than the edge difference may be used as long as it can detect a state in which edges change between images.

According to at least one embodiment described above, there is provided an elevator user detection system capable of suppressing over-detection of shadows caused by the lighting environment and correctly detecting users in the car or at the landing. can do.

It should be noted that although several embodiments of the invention have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.

DESCRIPTION OF SYMBOLS 11... Car, 11a... Curtain plate, 12... Camera, 13... Car door, 13a, 13b... Door panel, 14... Hall door, 14a, 14b... Door panel, 15... Hall, 17a, 17b... Three-way frame, 18... Hall sill 47 basket sill 48 illumination device 20 image processing device 21 storage unit 22 detection unit 23 detection area setting unit 24 detection processing unit 24a edge extraction unit 24b moving object Detection unit 24c Person detection unit 24d Reflection estimation unit 30 Elevator control device 31 Door opening/closing control unit 32 Warning unit E1, E2, E3 Detection area.

An elevator user detection system according to one embodiment includes reflection estimation means for estimating a reflection area including a portion where illumination light is reflected and its surroundings in an image captured by a camera installed in a car; when detecting a moving object from the photographed image so as to suppress over-detection of shadows occurring on the floor surface of the car according to the reflection level of the reflection area estimated by the reflection estimation means; A moving body detecting means for changing processing, and a person detecting means for detecting the moving body as a person based on information of the moving body detected by the moving body detecting means.

An elevator user detection system according to one embodiment includes reflection estimation means for estimating a reflection area including a portion where illumination light is reflected and its surroundings in an image captured by a camera installed in a car; According to the reflection level of the reflection area estimated by the reflection estimation means, so as to suppress overdetection caused by reflection of the illumination light in the reflection area via the user in the car, A moving body detecting means for changing processing when detecting a moving body from the photographed image, and a person detecting means for detecting the moving body as a person based on the information of the moving body detected by the moving body detecting means.

Claims (10)

In an elevator user detection system equipped with a camera installed in a car and photographing a predetermined range including the inside of the car,
Reflection estimating means for estimating a reflection area including a portion where the illumination light is reflected and its surroundings in the captured image of the camera;
moving body detection means for changing processing when detecting a moving body from the captured image according to the reflection level of the reflection area estimated by the reflection estimation means;
and person detection means for detecting the moving object as a person based on information on the moving object detected by the moving object detection means. The reflection estimation means includes:
2. The elevator user detection system according to claim 1, wherein said reflection area is estimated based on a distribution of luminance values of pixels of said photographed image. The reflection estimation means includes:
2. The elevator user detection system according to claim 1, wherein said reflection area is estimated based on a distribution of edges extracted from said photographed image. The moving body detection means is
2. The method according to claim 1, further comprising processing for detecting a moving object based on an edge change of each image successively obtained as the photographed images, wherein the higher the reflection level of the reflection area, the higher the threshold for the edge change. 2. The elevator user detection system according to claim 1. The moving body detection means is
It has a process of detecting a moving object based on the luminance difference and edge change of each image continuously obtained as the photographed image, and uses the luminance difference when the reflection level of the reflection area is lower than a certain level. 2. The elevator user detection system according to claim 1, wherein when the reflection level of said reflection area is equal to or higher than said predetermined level, said edge change is used to detect a moving object. The person detection means is
2. The elevator user detection according to claim 1, wherein the moving object is detected as a person based on at least one of a motion pixel distribution, a moving object size, and a moving object detection frequency obtained as information on the moving object. system. The person detection means is
7. The elevator user detection system according to claim 6, wherein when the reflection level of said reflection area is equal to or higher than a predetermined level, the determination criterion for said motion pixel distribution or said moving object size is increased. The person detection means is
7. The elevator user detection system according to claim 6, wherein when the reflection level of said reflection area is equal to or higher than a predetermined level, the criterion for determining the number of moving body detections is increased. 2. The apparatus according to claim 1, further comprising control means for executing a corresponding process associated with said detection area when said person is detected within a preset detection area on said photographed image. Elevator user detection system. The detection area is set near the door in the car,
The control means are
10. The elevator user detection system according to claim 9, wherein, as the corresponding processing, door opening/closing operation is controlled so that the person is not caught in the door while the car door is being opened.

Images