JP2000272863A - Monitor for passenger conveyor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically detect the displacement of the direction and position of a camera without using a device for detecting an angle of the camera, by comparing a position of a monitoring area set in the last time with a position of an area set in this time, and giving the information when the difference therebetween is above a predetermined value. SOLUTION: The data on an area set by an area setting part 84 in the last time is stored in an area position judging part 85, and the data is compared with the data on the area set in this time. When the difference of the area data is greater than a predetermined value as a result of the comparison, it is judged that the direction and position of an image pick-up means is displaced, and the fact is informed to a control room. Further in a case when an area can not be set by the area setting part 84, and the data on the area is not transmitted to the area position judging part 85, it is also judged that the direction and position of the image pick-up means is displaced, and the fact is informed to the control room. According to this system, the displacement of the direction and position of the image pick-up means such as a camera can be automatically detected and informed, so that the displacement of the direction and position of the image pick-up means can be rapidly corrected.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for monitoring a passenger conveyor, and more particularly to an apparatus for automatically detecting a shift in the direction or position of an image pickup means such as a camera.

[0002]

2. Description of the Related Art As a conventional man-conveyor monitoring device,
There is a monitoring device that monitors a man conveyor based on image data. For example, Japanese Patent Application Laid-Open No. 5-319762 discloses an area in which a step of a man conveyor is shown as a monitoring area.
Is set in advance as an area, and an area in which the vicinity of the entrance / exit of the man conveyor is reflected is set in advance as a second area,
A technique is described in which the flow of passengers on a conveyor is monitored based on image data, and the passengers are detected to fall or to get in the opposite direction.

Further, as a conventional monitoring device, there is a device for displaying information of an object appearing in a video image captured by a monitoring camera. For example, in Japanese Patent Application Laid-Open No. 9-307882, the position of the plant equipment and the position of the camera are stored as coordinate values on the same coordinate, the camera angle is obtained from a device for adjusting the direction of the camera, and these coordinate values and There is described a technique of judging which plant equipment is an object reflected on a camera from an angle, and displaying the name of the plant superimposed on a captured image.

[0004]

In the case of the technique described in the former publication, it is possible to monitor the flow of passengers on the conveyor on the basis of image data, and detect a fall of the passenger or a reverse ride. However, no specific method is described for coping with the case where the direction or position of the imaging means such as a camera is shifted.
Here, if the direction or position of the imaging means is shifted, the monitoring area is also shifted, so that there is a problem that a place to be originally monitored cannot be monitored.

In the case of the technique described in the latter publication, it can be detected from the position of the camera and the plant and the angle of the camera that the object to be monitored has deviated from the field of view of the camera. There is a problem that a device for setting the table of objects and detecting the angle of the camera is required.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a monitoring device for a man-conveyor which can automatically detect a shift in the direction or position of a camera without using a device for detecting the angle of the camera.

[0007]

According to a first aspect of the present invention, a passenger conveyor monitoring apparatus captures an image showing a situation of a passenger conveyor and its surroundings, and performs image processing based on the captured image. In a passenger conveyor monitoring device that automatically sets an area for monitoring based on image data that has been subjected to image processing, compare the position of the previously set area for monitoring with the position of the currently set area Then, when the difference is equal to or more than a predetermined value, the notification is performed.

[0008] A passenger conveyor monitoring apparatus according to a second aspect of the present invention captures an image showing the condition of the passenger conveyor and its surroundings, performs image processing based on the captured image,
In a passenger conveyor monitoring device that automatically sets an area for monitoring based on image-processed image data, a notification is made when an area for monitoring cannot be automatically set. Is what you do.

According to a third aspect of the present invention, there is provided a passenger conveyor monitoring apparatus which captures an image showing a condition of a passenger conveyor and its surroundings, and monitors the passenger conveyor based on the captured image. An area for monitoring is automatically set based on the image, a passenger using the passenger conveyor in an irregular state is detected based on the image in the set area, and the detected passenger is detected. When the number exceeds a predetermined value, the area for monitoring based on the captured image is automatically set again, and the position of the previously set area for monitoring and the position of the area set this time are automatically set. Are compared, and when the difference is equal to or more than a predetermined value, a notification is made.

[0010] With this configuration, it is possible to automatically detect that the direction or position of the camera has shifted without using a device for detecting the angle of the camera.

[0011]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of the present invention.

In FIG. 1, 1 is a handrail of a passenger conveyor, 2 is a step, 3 is a gear connected to the step 2, 4 is an electric motor, 5 is a control device for controlling the electric motor 4, 6 is a floor at the entrance and exit, 7 Denotes an imaging unit such as a camera, 8 denotes a monitoring device, 9 denotes a display unit such as a display, and 10 denotes an audio output unit such as a speaker. Reference numeral 15 denotes a maintenance center for maintaining and managing the passenger conveyor by a management room or a maintenance person of a building where the passenger conveyor is installed. Reference numeral 16 denotes a notifying unit for notifying that the direction and the position of the imaging unit 7 have been shifted.
It is an LED character display, a speaker, a buzzer, or the like.

An image pickup means 7 such as a camera takes an image of a passenger conveyor and its surroundings, and sends an image pickup signal to the monitoring device 8. Although the imaging unit 7 and the monitoring device 8 are illustrated separately in FIG. 1, the monitoring device 8 may be built in the imaging unit 7. The monitoring device 8 detects embarking and mischief, and automatically sets an area to be detected. Here, the embarking means that the passenger embarks on the outside of the passenger conveyor and gets on. The mischief refers to a case where the passenger is playing near the entrance of the passenger conveyor or is hanging around.

A display means 9 such as a display and a sound output means 10 such as a speaker are used to guide a passenger on a passenger conveyor to draw attention. As the contents of the guidance,
"It is dangerous to get on.", "Please ride in the correct position", "Do not play near the entrance".

Although an escalator is shown in FIG. 1 as an example of a passenger conveyor, the same function as that of the present embodiment can be realized in the case of a moving sidewalk.

FIG. 2 is a block diagram showing the internal configuration of the monitoring device 8.

The input section 81 receives an image pickup signal from the image pickup means 7, converts it into digital data, and sends it to the image memory 82.

The image memory 82 fetches digital image data at a constant period. The image processing unit 83 performs image processing on the image data and creates image data to be used for the next area setting and image recognition determination processing. First, the area setting unit 84 automatically sets a detection area for detecting embarking or mischief based on the image data processed by the image processing unit 83.

The area position determining section 85 stores the data of the area previously set by the area setting section 84, and compares the data with the data of the currently set area. As a result of the comparison,
If the difference between the area data is larger than the predetermined value, it is determined that the direction and the position of the imaging unit 7 have shifted, and the management unit 15 is notified.

If the area setting unit 84 cannot set the area and the area data is not sent to the area position judging unit 85, it is determined that the direction and the position of the image pickup means 7 are shifted, and the administrator Notify room 15.

The image recognizing section 86 includes an area setting section 84 of the image data processed by the second and subsequent image processing sections 83.
The detection of embarking, mischief, etc. is performed using the image data in the detection area set in. The number-of-times determining unit 87 determines whether the number of embarkings or mischiefs detected by the image recognizing unit 86 within a certain time exceeds a predetermined value. If the value exceeds the predetermined value, there is a possibility that erroneous detection has occurred frequently due to a shift in the position or orientation of the camera. Therefore, the area is set again by the area setting unit 84, and It is checked whether the position of the area is shifted or not, and if it is shifted, it is notified to the manager room 15.

The guidance unit 88 provides various guidance using the display unit 9 or the voice output unit 10 according to the determination result of the image recognition unit 86. The output unit 89 transmits a signal for causing the display unit 9, the audio output unit 10, and the notification unit 16 to provide guidance.

Next, the detection area setting procedure of the area setting section 84 in the monitoring device 8 of the present embodiment will be described with reference to the flowchart shown in FIG. Here, it is assumed that the image data captured by the photographing unit 7 is binarized by the image processing unit 83 and passed to the area setting unit 84.

In step 1, an inter-frame difference is calculated. The inter-frame difference is to take a difference between an image taken at the current time and an image taken immediately before for an image taken at a fixed period. In step 2, the difference images obtained in step 1 are added as needed. Next, step 3
According to the determination of the above, the processing of steps 1 and 2 is repeated until the number of times reaches a predetermined number. In step 4, image processing such as expansion and contraction of the difference image added a predetermined number of times is performed. Step 5
Then, the travel area of the step is extracted. In step 6, a embarkation detection area is set. In step 7, a mischief detection area is set.

The process from step 1 to step 5 in FIG. 3, that is, the process of extracting the running area of the step by the area setting unit 84 will be described with reference to FIG.

FIGS. 4 (a) and 4 (b) are examples of images of the passenger conveyor, which are shown in FIGS. 4 (c), 4 (d) and 4 (d).
FIG. 4E shows an image subjected to the image processing in step 4. FIG. 4 shows images that have been binarized by the image processing unit 83.

Reference numeral 1 denotes a handrail, 2 denotes a step, 6 denotes an entrance floor, 11 denotes a demarcation line of the step (a yellow caution mark), and 12 denotes a traveling area of the extracted step.

FIG. 4B is an image after a predetermined time from when the image of FIG. 4A was taken. The predetermined time is set in the image memory 8
2B depends on the period of fetching the image data. For example, if the image memory 82 fetches the image data every 10 msec, FIG. 4B shows the image after 10 msec in FIG. 4A.

FIG. 4 (c) is an image obtained by calculating the exclusive OR of the difference between FIGS. 4 (a) and 4 (b). When exclusive OR is performed, only different portions of both images are extracted. The difference between the two images is that they move with time. In the case of FIG. 4, the step 2 is used, and in particular, the demarcation line 11 is used. Although the handrail 1 is also moving, it is usually formed in a single color, so even if it actually moves, it appears to stop on the image,
When exclusive OR is taken, it becomes 0 and does not appear in FIG.

In the image shown in FIG. 4C, the position of the demarcation line 11 differs depending on the moment when the image is taken, and therefore, the image differs depending on the time when the processing is performed. By adding (ORing) these exclusive OR images obtained at each time point, an image in which many lines exist in the running area of the step as shown in FIG. 4D is obtained. Can be.

By performing expansion (also called expansion or propagation) or contraction (also called erosion), which is a general technique of image processing, on the image shown in FIG. The gap can be filled, and the traveling area 12 of the step can be extracted as shown in FIG.

In FIG. 4D, even if the inside of the running area of the step is not painted as shown in FIG. 4E, the running of the step is performed based on the coordinates of both ends of each of the many lines in the running area of the step. Approximately obtain the coordinates of the hypotenuse of the trapezoid indicating the area,
The traveling area can be extracted.

A detailed flow of the process of step 6 in FIG. 3, that is, the process of setting the start detection area of the area setting section 84 will be described with reference to FIG.

In step 8, the length of the upper side of the step traveling area is measured based on the extraction of the step traveling area by the processing in step 5 of FIG. In step 9, the reduction ratio at the position of the upper side of the step traveling area is determined. Step 1
In the case of 0, the reduction value obtained in step 9 is multiplied by the set value of the preset start detection area. In step 11, the x coordinate of the left end point of the upper side of the step traveling area is calculated in step 10.
Find the coordinates obtained by subtracting the value obtained in. In step 12,
Step 1 from the x coordinate of the rightmost point on the upper side of the step running area
The coordinates obtained by adding the values obtained by 0 are obtained.

On the other hand, in step 13, the length of the lower side of the step traveling area is measured. In step 14, the reduction ratio at the position on the lower side of the step traveling area is determined. In step 15, the reduction value obtained in step 14 is multiplied by the preset set value of the embarkation detection area. In step 16, the coordinates obtained by subtracting the value obtained in step 15 from the x coordinate of the left end point on the lower side of the step traveling area are obtained. Step 17
Then, the coordinates obtained by adding the value obtained in step 15 from the x coordinate of the right end point on the lower side of the step running area are obtained.

Next, in step 18, an area surrounded by the left end point of the upper side and the left end point of the lower side of the step traveling area and the point of the coordinates obtained in steps 11 and 16 is defined as a left embarking area. In step 19, the right end point of the upper side and the right end point of the lower side of the step traveling area are compared with step 12, step 1
The area surrounded by the points of the coordinates obtained in step 7 is defined as the right embarking area.

FIGS. 5 (a) and 5 (b) are explanatory diagrams related to the process of setting the embargo detection area. Here, FIG.
The coordinates of the four points shown in FIG. 4A are an example of the coordinates of the four corners of a trapezoid indicating the step running area 12 in FIG.

In FIG. 5A, the coordinates of the four corners of this trapezoid are defined as upper sides (40, 50), (70, 50), lower sides (30, 170), and (80, 170). Since the size of the step can be known in advance, the size of the step on the image and the actual size can be compared. For example,
An example will be described in which the area of the tread is 1000 mm (long side) × 400 (short side) mm, and the detection area for embarking is set to 200 mm from the end of the step.

The length of the upper side of the trapezoid in FIG.
0 = 30, which corresponds to the length of the long side of the step. Since the actual length is 1000, it is 30 on the image, so that it is reduced to 30/1000 = 3/100, that is, 3/100. Therefore,
When the value of the x-coordinate is 40 and 70, the set value of the area for detection of embarking is set to 3/100 and 200 · (3 /
100) = 6. As a result, the value of the x coordinate is 3
4 and 76, and the xy coordinates are (34, 50), (76,
50).

Similarly, the length of the lower side of the trapezoid in FIG. 5 is 80-30 = 50. This length corresponds to the length of the long side of the step. Since the actual length is 1000, but it is 50 on the image, 50/1000 = 1 /
This means that the size has been reduced to 20 or 1/20. Therefore, when the value of the x-coordinate is 30 and 80, the set value of the area for detection of embarkation is also reduced to 1/20, and
= 10. As a result, the values of the x coordinate are 20 and 9
0, and the xy coordinates are (20, 170), (90, 17)
0).

FIG. 5B shows the setting of the detection area for boarding and boarding as described above. In FIG. 5B, reference numeral 12 denotes a step traveling area, and reference numeral 13 denotes an embarking detection area. 3, that is, the area setting unit 84
The detailed flow of the mischief detection area setting process will be described with reference to FIG.

In step 20, step 5 in FIG.
The length of the lower side of the step traveling area is measured based on the extraction of the step traveling area by the above processing. In step 21, a reduction ratio at a position on the lower side of the step traveling area is determined. In step 22, the set value of the mischief detection area set in advance is multiplied by the reduction ratio obtained in step 21. In step 23, the coordinates obtained by subtracting the value obtained in step 22 from the x coordinates of the left end point on the lower side of the step traveling area are obtained. Step 24
Then, the coordinates obtained by adding the value obtained in step 22 from the x coordinate of the right end point on the lower side of the step running area are obtained.

Next, in step 25, step 2
A straight line that passes through the point of the coordinates obtained in step 3 and is parallel to the left oblique side of the step running area is drawn. In step 26, step 24
Draw a straight line that passes through the point of the coordinates obtained in and is parallel to the right oblique side of the step traveling area. In step 27, the area surrounded by the lower side of the step running area, the straight line obtained in steps 25 and 26, and the lower end of the image is defined as a mischief area.

FIG. 5C is an explanatory diagram related to the processing of setting the mischief detection area according to the flow of FIG.

As shown in FIG. 5B, when the mischief detection area is set at 200 mm from the end of the step, the reduction ratio on the image with respect to the actual length of the lower side of the step traveling area becomes 1/20. When the value of the x coordinate is at the left end and the right end of the lower side of the step traveling area, the set value of the area for detecting tampering may be set to 1/20 and 200/20 = 10.

As a result, the values of the x coordinate are 20 and 90, and the xy coordinates are (20, 170) and (90, 170). Next, two straight lines are drawn from the coordinates of the two points so as to be parallel to the respective oblique sides of the trapezoid, and an area surrounded by these straight lines, the image edge, and the lower side of the trapezoid is defined as a mischief detection area.

A detailed flow of another processing example of step 7 in FIG. 3 will be described with reference to FIG.

In step 28, step 5 in FIG.
The length of the lower side of the step traveling area is measured based on the extraction of the step traveling area by the above processing. In step 29, a reduction ratio at a position on the lower side of the step traveling area is determined. In step 30, the reduction value obtained in step 29 is multiplied by the preset value of the mischief detection area. In step 31, coordinates obtained by subtracting the value obtained in step 30 from the x coordinate of the left end point on the lower side of the step traveling area are obtained. Step 32
Then, the coordinates obtained by adding the value obtained in step 22 from the x coordinate of the right end point on the lower side of the step running area are obtained.

Next, in step 33, step 3
1. A perpendicular line is drawn from the coordinate point obtained in step 32 toward the lower end of the image. In step 34, a square area surrounded by the lower side of the step running area, the two perpendicular lines obtained in step 33, and the lower end of the image is set as a mischief area.

FIG. 5D is an explanatory diagram relating to the processing for setting the mischief detection area according to the flow of FIG.

The difference from FIG. 5C is that the xy coordinates (20, 17) obtained as an area for detecting mischief are used.
0) and (90, 170) are drawn perpendicular to the lower end of the image. Others are the same as in the case of FIG.

In the present embodiment, the detection area is set directly from the travel area of the steps. However, based on the position and size of the travel area of the steps, the passenger conveyor such as the handrail 1 and the entrance / exit floor 6 is used. The same function can be realized by extracting each part of, and setting a detection area for embarking and a detection area for mischief therefrom.

Also, the detection area can be set after each part of the passenger conveyor is directly extracted. For example, the entrance / exit floor 6 can be extracted by texture matching since its material and surface design are predetermined. Alternatively, the shape of the entrance floor 6 is rectangular,
Since the size is also determined in advance, the entrance / exit floor 6 can be extracted based on such information. Or,
The handrails 1 are located on both sides of the passenger conveyor, and can be extracted from features such as being parallel to each other and having a known width dimension of the handrails 1. Alternatively, a marker is placed on the handrail 1 or the entrance / exit floor 6, and by detecting this marker, the handrail 1 or the like can be extracted.

At this time, any marker may be used as long as it has characteristics when an image is taken. For example, a material with high brightness such as a white tape or a light source may be used. Further, a marker having a special shape may be used. If the shape is registered in advance, a marker can be easily extracted by template matching.

The handrail 1 thus extracted
It is also possible to set a embarkation detection area and a mischief detection area based on each part of the passenger conveyor such as the floor and the entrance floor 6.

Next, the image recognition unit 86 in the present embodiment
Will be described with reference to FIG. 6, the same reference numerals as those in FIGS. 4 and 5 indicate the same elements.

First, detection of embarking will be described.
In the simplest detection method, the background image of the detection region 13 of the start is stored, compared with the current image, and when the difference is equal to or more than a predetermined value, the start is detected as the start. However, in this method, the recognition rate is low, for example, it is erroneously determined that a person stands alone in the detection area 13 to start. Therefore,
Improving the recognition rate by adding conditions such as that the object in the detection area 13 is moving in the same direction as the moving direction of the passenger conveyor, and moving at a speed higher than the moving speed of the passenger conveyor. Can be.

Next, detection of mischief will be described.
The simplest detection method is to store the background image of the mischief detection area 14, compare it with the current image, check whether the difference is greater than a predetermined value, and check if the difference is greater than a predetermined value. Is detected as mischief when has continued for a predetermined time or more. However, in this method, even when the passenger passes through the detection area without interruption, the recognition rate is low, such as misrecognition as mischief. Therefore, the recognition rate can be improved by judging the movement of the passenger, and adding a condition for judging the mischief when the passenger is moving in a direction different from the traveling direction of the passenger conveyor.

In this embodiment, the setting of the detection area for the embarking and mischief is shown in FIG. 5, but the setting of the detection area for detecting other events is not limited to the position or size of the traveling area 12 of the step. If it is set based on the degree, it can be realized. For example, the traveling area 12 of the step may be set as a detection area as it is and used as a detection area for detecting a passenger traveling backward on the passenger conveyor.

Next, the procedure for determining the detection area by the area position determination section 85 in the monitoring device 8 of the present embodiment will be described with reference to the flowchart shown in FIG.

At step 35, it is determined whether or not the area setting section 84 has set the area data. If the setting has not been successful, a notification is made to the manager room 15 in step 40. If it can be set, the processing of step 36 and thereafter is executed.

In step 36, it is determined whether or not the region has been set for the first time. In the first case, since there is no area data to be compared, the area data is stored in step 39 and the processing ends. In the case of the second time or later, the processing of step 37 and thereafter is executed.

In step 37, the current region data is compared with the stored previous region data to determine the difference between the region positions. As a method of calculating the difference, there is a method of calculating the difference between the center point and the center of gravity of each area, and a method of calculating the difference at the four corner points of each area.

In step 38, it is determined whether or not the difference in step 37 is equal to or greater than a predetermined value. If the difference is equal to or larger than the predetermined value, a notification is sent to the management room 15 in step 40. If the difference is less than the predetermined value, the area data set this time is stored in step 39.

[0066]

As described above, according to the present invention,
A monitoring device that monitors the passenger conveyor based on the image data automatically detects that the orientation or position of the imaging means such as a camera has shifted, and reports this to the management room or maintenance center. And the displacement can be quickly corrected.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is a block diagram showing an internal configuration of a monitoring device according to the present invention.

FIG. 3 is a flowchart illustrating a detection area setting procedure of an area setting unit according to the present invention.

FIG. 4 is an explanatory diagram illustrating a step traveling region extraction process of the region setting unit according to the present invention.

FIG. 5 is an explanatory diagram relating to a process of setting a embarkation and mischief detection area according to the present invention.

FIG. 6 is an explanatory diagram relating to a process of detecting embarrassing and mischief by the image recognition unit of the present invention.

FIG. 7 is a detailed flowchart of a process of setting a start detection area according to the present invention.

FIG. 8 is a detailed flowchart of a mischief detection area setting process according to the present invention.

FIG. 9 is a detailed flowchart of a mischief detection area setting process according to the present invention.

FIG. 10 is a flowchart illustrating an area position determination procedure of an area position determination unit according to the present invention.

[Explanation of symbols]

1 ... handrail, 2 ... step, 3 ... gear, 4 ... electric motor,
Reference numeral 5: control device, 6: entrance floor, 7: imaging means (camera, etc.), 8: monitoring device, 9: display means (display, etc.), 10
... voice output means (speaker etc.), 11 ... demarcation line of step 2, 12 ... step running area, 13 ... embarking detection area, 14 ... mischief detection area, 15 ... manager room, 16 ... report to manager room Means (display, speaker, etc.) 81 input unit 82 image memory 83 image processing unit 84 area setting unit 85 area position determination unit 86
... Image Recognition Unit, 87 ... Number Determination Unit, 88 ... Guide Unit, 89
... Output section.

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Hisao Chiba 1070 Ma, Hitachinaka-shi, Ibaraki F-term in Mito Plant of Hitachi, Ltd.F-term (reference) AA44 DD03 DD12 EE05 EE07 EE18 GG02 GG06 GG19 GG31

Claims (3)

[Claims] 1. An image showing a condition of a passenger conveyor and its surroundings is taken, and image processing is performed based on the taken image.
In a passenger conveyor monitoring device that automatically sets an area for monitoring based on image processed image data, compare the position of the previously set area for monitoring with the position of the currently set area And a means for notifying when the difference is greater than or equal to a predetermined value. 2. An image showing a condition of a passenger conveyor and its surroundings is taken, and image processing is performed based on the taken image.
In a passenger conveyor monitoring device for automatically setting an area for monitoring based on image-processed image data, a means for notifying when a monitoring area cannot be automatically set is provided. A passenger conveyor monitoring device, characterized in that: 3. A passenger conveyor monitoring device that captures an image showing a situation of a passenger conveyor and its surroundings, and monitors the passenger conveyor based on the captured image, automatically arranging an area for monitoring based on the captured image. Area setting means for setting the position, image recognition means for detecting a passenger using the passenger conveyor in an irregular state based on an image in the area set by the area setting means, and the image recognition means Means for determining whether or not the number of passengers using in an irregular state detected by the user exceeds a predetermined value;
Means for automatically setting an area for monitoring again based on the captured image when a predetermined value is exceeded, means for comparing the position of the previously set area with the position of the currently set area, A monitoring device for a passenger conveyor, comprising means for notifying when the difference is equal to or greater than a predetermined value.