JP2009086748A - Monitoring device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce a processing load by reducing computational complexity by easily and precisely monitoring a traveling object on a moving object by a monitoring device. <P>SOLUTION: An acquired photographic image G of an escalator ES is divided into a plurality of blocks B, and an optical flow is extracted from the reference photographic image, and an optical flow table T for a plurality of blocks B is preset. In monitoring, the blocks B to be compared are determined from the plurality of blocks of the photographic image G with a time difference on the basis of the vector W of each block B of the optical flow table T, and the degree of similarity between the blocks B is calculated. The number of the blocks B whose degree of similarity is lower than a prescribed threshold is counted, and the count value is compared with a prescribed threshold, so that the movement of a person P on the escalator ES is detected. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a monitoring device and a program for detecting and monitoring the movement of a moving body such as a person moving on a moving object.

  Wide range of monitoring devices that detect moving objects such as people moving on escalators and moving walkways, etc., with a surveillance camera that faces them, and monitor their movements to detect dangerous or violations It is popular. As such a monitoring device, a passenger conveyor safety device has been conventionally known in which a passenger conveyor is photographed by a surveillance camera, the photographed image is analyzed, and the safety of passengers who use it is improved. (See Patent Document 1).

  In this conventional safety device, the number of passengers on the passenger conveyor is detected by processing the captured image, and whether the number of passengers is large or small is monitored, and optimal boarding guidance is given to the passengers by voice. Etc. to improve safety. However, this conventional device does not have means for specifically identifying the dangerous behavior of the passenger based on the image taken by the surveillance camera, and may not be able to recognize the dangerous behavior on the passenger conveyor and overlook it. There is a problem that the monitoring accuracy is not sufficient.

  Also, conventionally, a photographed image by a camera is compared with a reference image, and a person is cut out from the photographed image, and the movement position of the person appearing in the next frame is predicted from the movement direction of the person, and the appearance is predicted in the region. On the other hand, a tracking device that tracks a person by associating frames with each other is also known (see Patent Document 2).

  However, in this conventional tracking device, when monitoring a moving object on a moving object, such as when monitoring an escalator, the moving object on the moving object is cut out, for example, both of them are cut out. Is difficult, and the tracking (monitoring) accuracy may be lowered. In addition, this conventional tracking device requires processing for obtaining a movement prediction area for each frame with respect to moving objects and individual movements of the moving body that occur on the entire shooting screen, and the processing amount increases. There is also a problem that the load becomes large.

  On the other hand, conventionally, the surroundings of the vehicle are detected and monitored by, for example, photographing the periphery of the vehicle with a camera mounted on the automobile, extracting a predetermined optical flow in the predetermined region, and monitoring the direction thereof. A vehicle monitoring device is also known (see Patent Document 3).

  However, in this conventional monitoring device, since the detection is performed in a limited situation around the vehicle, the monitoring area displayed in the screen and the monitoring vector in the monitoring area can be input in advance as known information. In the monitoring device that does not know whether the image is captured below, information on the monitoring area and the monitoring vector cannot be input in advance. Therefore, this conventional monitoring device has a problem that its applicable range is narrow, it cannot cope with each situation where a general monitoring device is placed, and monitoring by optical flow extraction cannot be performed properly.

  By the way, the optical flow is to detect the movement of a moving body from luminance information in an image or the like, and to express the movement and motion such as the direction and degree of movement of the target moving body by a velocity vector (motion vector). . As an optical flow extraction method, there are a gradient method and a block matching method. Conventionally, a captured image is analyzed by using these, and the movement of a moving object on a moving object, for example, the movement of a person on an escalator is related. An optical flow is extracted, and based on the extracted optical flow, a person's movement on the escalator is detected and a dangerous action or a violation is monitored.

11A and 11B are schematic diagrams for explaining a conventional method for monitoring a person on an escalator by optical flow. FIG. 11A schematically shows an image taken by a monitoring camera, and FIG. 11B schematically shows an image obtained by optical flow processing. Is shown.
In addition, in FIG. 11, the example of the image which image | photographed the whole ascending escalator ES which moves toward the upper floor (it moves upward in the figure) with the monitoring camera from the front is shown. Here, a plurality of people P board the step S of the escalator ES, and one of them (indicated by lattice hatching in FIG. 11A) walks up on the step S toward the pedestrian P1 ( FIG. 11A shows an example in which the movement is represented with a figure superimposed.

  When such a photographed image (monitoring image) (see FIG. 11A) is subjected to optical flow processing to extract the optical flow of the moving body, as shown in FIG. 11B, the motions of step S, person P, and pedestrian P1 are represented by vector V. , V1 (arrow in the figure), an optical flow processed image is obtained. That is, in this optical flow processed image, for the step S of the escalator ES and a normal person P who is not walking on the escalator ES, the upward fixed vector V ( The black arrow in the figure appears. However, the whole does not become a constant value, and on the lower side (boarding gate side) close to the monitoring camera, each movement amount in the image becomes relatively large, so the vector V also becomes large. The vector V becomes smaller on the upper side (the exit) side far from the camera. In addition, a noise-like vector V also appears when the person P moves slightly, but a great difference does not occur compared to the surrounding vector V.

  On the other hand, in the part of the pedestrian P1, a vector V1 (open arrow in the figure) that is larger than the surroundings appears according to the movement amount and speed. Therefore, if the optical flow of the photographed image is compared with the optical flow processing data (background data) relating to the movement in step S or the like in a state where the person P is not on board, the vector V1 corresponding to the pedestrian P1 is compared. Thus, it is possible to detect the movement of the pedestrian P1 (here, walking). In this manner, the movement of a person on the escalator ES is detected, and a dangerous action or a violation is determined and monitored.

FIG. 12 is a schematic diagram for explaining another example of a method for monitoring a person on an escalator by a conventional optical flow. FIG. 12A is an image taken by a monitoring camera, and FIG. 12B is an optical flow process. Each image is shown schematically.
Note that FIG. 12 shows an example of an image obtained by photographing the entire ascending escalator ES on which a plurality of people P are boarded, with the monitoring camera, similarly to FIG. 11 described above. In addition, here, an example is shown in which one person on the step S (indicated by grid-like hatching in FIG. 12A) is a dangerous actor P2 who leans outward from the handrail H of the escalator ES (right side in FIG. 12A). Yes.

  In the optical flow processed image of such a captured image (see FIG. 12A), as shown in FIG. 12B, in addition to the vector V in the normal state, an outer region J10 of the handrail H where no vector appears in the background data, A vector V2 (open arrow in the figure) corresponding to the dangerous actor P2 appears. As described above, when it is determined that the vector V2 appears in a region where the vector is zero in the background data of the optical flow, the escalator is detected in the same manner as described above by detecting the movement (in this case, the embarking action). It detects the movement of people on the ES, and monitors and monitors dangerous acts and violations.

  Here, when the above-described optical flow extraction is performed by the above-described block matching method, it is possible to increase the detection accuracy with respect to the movement of the moving body, such as detecting a large movement or rotation of a moving body such as a person. . However, in this method, the captured image is divided into a plurality of vertical and horizontal blocks, and each block of one image captured with a predetermined time difference is searched for, for example, by comparing with all the blocks of the other image. There is a need. Therefore, in the case of the block matching method, there is a problem that the amount of calculation for extracting the optical flow for each image becomes very large and the processing load increases. On the other hand, when the optical flow is extracted by the gradient method described above, there is a problem that the calculation amount necessary for the extraction can be relatively reduced, but the accuracy with respect to a large movement of the moving body is lowered.

JP-A-9-301664 JP 2003-346159 A JP-A-6-314340

  The present invention has been made in view of the above-described conventional problems, and an object of the present invention is to enable a monitoring device to monitor a moving object on a moving object easily and accurately, and to reduce the amount of calculation at that time. It is to reduce the load.

The invention according to claim 1 is a monitoring device for monitoring a moving body on a moving object, the means for dividing a captured image of a monitoring area captured by a monitoring camera into a plurality of blocks, and the moving object or the moving object And a means for extracting an optical flow from a reference photographed image of a moving body, a means for setting an optical flow table for each of the plurality of blocks from the extracted optical flow, and a time difference based on the optical flow table. Based on the means for determining the blocks to be compared from the plurality of blocks of the captured image, the means for calculating the similarity between the determined blocks, and the calculated similarity for each of the plurality of blocks, Means for monitoring the movement of the moving body on the moving object.
According to a second aspect of the present invention, in the monitoring apparatus according to the first aspect, the monitoring means includes the means for comparing the calculated similarity with a predetermined threshold value of the similarity. Based on the result of comparing the count number, means for counting the number of blocks whose similarity is lower than the threshold value, means for comparing the count number of the block with a predetermined threshold value of the count number, Means for detecting movement of the moving body on the moving object.
A third aspect of the present invention is the monitoring apparatus according to the first or second aspect, further comprising means for setting a plurality of areas for the captured image, and the means for monitoring is the moving object for each of the plurality of areas. The movement of the moving body is monitored.
According to a fourth aspect of the present invention, in the monitoring device according to the third aspect, the counting means counts the number of blocks having the calculated similarity lower than the threshold value for each of the plurality of areas, and the counting The means for comparing numbers compares the count number of the block for each of the plurality of areas with a predetermined threshold value of the count number of the area.
The invention according to claim 5 is a program for causing a computer of a monitoring device to function as each means of the monitoring device according to any one of claims 1 to 4.

  According to the present invention, a moving body on a moving object can be easily and accurately monitored by the monitoring device, and the processing load can be reduced by reducing the amount of calculation at that time.

Hereinafter, an embodiment of a monitoring device of the present invention will be described with reference to the drawings.
This monitoring device is a device for monitoring a moving body such as a person or an animal on a moving object such as an escalator moving at a predetermined speed or a moving sidewalk, and monitors the movement of the moving body on the moving object. Detecting dangerous acts and violations.
In this embodiment, similarly to the above-described conventional example (see FIGS. 11 and 12), a person who is one or a plurality (here, a plurality) of moving bodies that rides on an ascending escalator that is a moving object is monitored. The monitoring apparatus will be described as an example.

FIG. 1 is a schematic diagram illustrating a schematic configuration of a monitoring system including the monitoring device of the present embodiment, and also schematically illustrates an installation state on an escalator that is a monitoring target.
As shown in the figure, the monitoring system 1 includes a monitoring device 10 that also serves as a control device for the entire system, a monitoring camera 2 that is connected to the monitoring device 10 via an interface or the like, and a warning device 3.

  The monitoring camera 2 is an imaging means for monitoring, and is installed obliquely above the entire ascending escalator ES, and includes the escalator ES that is the monitoring target and its surroundings, and a person P who has boarded the escalator ES. Take an image of the surveillance area from the front. The surveillance camera 2 is a camera used for surveillance of a commercially available digital camera, digital video camera, etc., and is controlled by the surveillance device 10 to take an image of the escalator ES at a predetermined interval, timing, etc. Are output to the monitoring apparatus 10 and transmitted.

  The warning device 3 is controlled by the monitoring device 10 to detect, for example, an alarm for the person P on the escalator ES and its surroundings when a dangerous action or violation of the person P boarding the escalator ES described later is detected. Or sound, or an audible or visual alarm such as a warning light to warn. Here, the warning device 3 is an alarm generation device arranged toward the escalator ES, and generates a warning alarm toward the escalator ES when a control signal (warning generation command) is received from the monitoring device 10. .

  The monitoring apparatus 10 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12 for temporarily storing a program, data, and the like for processing of the CPU 11, and driving the CPU 11 to perform calculations. It is composed of a computer having various memories such as a ROM (Read Only Memory) 13 storing a program for executing processing and the like, and controls each part of the connected device to execute an operation for monitoring and the like. . The monitoring apparatus 10 performs various processes necessary for monitoring by performing image processing and analysis on the captured image received from the monitoring camera 2 based on a predetermined program such as a monitoring program and preset conditions. It is also an image processing device and a monitoring processing device.

FIG. 2 is a functional block diagram showing a schematic configuration of the monitoring device 10.
As shown in the figure, the monitoring device 10 includes an input unit 23 and a display unit 24 in addition to an interface unit 21 to which external devices such as a monitoring camera 2 and a warning device 3 are connected, and a control unit 22 that controls the entire system. In addition to the background data setting unit 25, the image processing unit 26, and various programs necessary for the control (monitoring) processing and image processing described above, various data including image data 27A such as captured images and processed images are stored. A storage unit 27 is provided, and these units (means) 21 to 27 are connected to each other via a bus 29.

  The input unit 23 is, for example, a mouse, a keyboard, or a touch panel for inputting various data including setting data necessary for image processing and monitoring, such as instructions and operations for the monitoring device 10, and setting and updating of various thresholds described later. Etc. The display unit 24 is a display unit such as a monitor for displaying data input via the input unit 23, an image taken by the monitoring camera 2, or a processed image after various processes, and monitoring and setting of the escalator ES. Used when entering data.

  The background data setting unit 25 is a means for setting background data as a reference necessary for monitoring processing for detecting the movement of the person P on the escalator ES, which will be described later, and an optical flow extraction unit 25A and an optical flow table setting. The background data for determining a region (here, a block) to be compared between the captured images to be compared is created.

FIG. 3 is a schematic diagram for explaining the setting of the background data. Like FIG. 11 described above, FIG. 3A shows an image taken by the surveillance camera 2 (see FIG. 1), and FIG. 3B shows an optical flow. Each processed image is shown schematically.
Here, when detecting and monitoring the movement of the person P on the moving escalator ES, it is not necessary to detect and grasp the strict movement vector of each person P, and the movement moves at a predetermined speed and direction. It is only necessary to detect other movements in consideration of background movements such as step S and handrail H of the escalator ES. Therefore, in this embodiment, an optical flow such as an escalator ES as a reference is acquired in advance, an optical flow table that is a background data table of the optical flow is created, and block matching is performed only at a flow position estimated based on this. Like to do.

  Specifically, as a reference image, an image in a normal state where no person P is on the escalator ES or in a normal state where the person P is riding without walking, dangerous acts, or violations, a predetermined time difference is obtained by the monitoring camera 2. Take multiple shots. As described above, the reference photographic image of the escalator ES or the escalator ES and the person P is acquired. Here, as shown in FIG. 3A, a reference photographic image in a state where a plurality of persons P are on the escalator ES is acquired.

  From this reference photographed image, the optical flow extraction unit 25A obtains the motion vector V (see FIG. 3B) of each part in the image by the block matching method or the like, and extracts the optical flow as a reference. Also, an optical flow background data table (optical flow table) is created from the extracted optical flow by the optical flow table setting unit 25B. At this time, in the monitoring apparatus 10, the captured image captured by the monitoring camera 2 is divided into a plurality of blocks by the block dividing unit 26A of the image processing unit 26, and an optical flow table is set for each of the plurality of blocks.

FIG. 4 is a schematic diagram for explaining the setting of the optical flow table.
As shown in FIG. 4A, the block dividing unit 26A divides the captured image G in the monitoring area into blocks B each including a predetermined number of pixels in the vertical and horizontal directions (XY directions), and the entire captured image G is divided into a plurality of blocks. Partition into blocks B. The extracted optical flow vector V (see FIG. 3B) is made to correspond to the plurality of blocks B divided in this way, and each vector for each block B is calculated. As shown in FIG. 4B, the optical flow table T Set.

  In FIG. 4B, a part of the optical flow table T of the captured image G is extracted, and the direction and length of the vector W set for each block B are schematically shown. In addition, here, the optical flow table T is visually shown for explanation, but actually, the optical flow table T to be set is data related to a vector corresponding to the block B, and is associated with the block B, respectively. And stored in the optical flow table 27B of the storage unit 27. Furthermore, in the illustrated example, the optical flow represents the vector direction from the current captured image G (frame) to the previous captured image G that is captured with a predetermined time difference, and the actual moving direction is indicated. It is a vector W in the opposite direction to the vector V shown (see FIG. 3B).

FIG. 5 is a schematic diagram for explaining the optical flow table T set as described above.
As shown in the figure, the optical flow table T indicates in which direction and position the block B of the current captured image corresponds to the block B in the previous captured image in the X and Y directions, respectively. It is represented by a vector W composed of components.

  Specifically, when attention is paid to one block B1 (lattice hatching area in the figure) of the optical flow table T, the vector W at the same position is directed diagonally downward to the left. Therefore, the block B1 at the same position in the current photographed image G1 corresponds to the block B2 adjacent in the diagonally lower left direction in the previous photographed image G2 according to the direction and length indicated by the vector W. That is, it is assumed that the captured image G2 in the block B2 moves to the block B1 of the next captured image G1, and those blocks B1 and B2 are basically the same or corresponding ones. Will be. Using this, the monitoring apparatus 10 of the present embodiment compares the blocks B of the two images G1 and G2 by the image processing unit 26 (see FIG. 2) based on the optical flow table T set in advance as described above. The image processing load or the like is reduced by performing each of the predetermined blocks B1 and B2 only. Next, the image processing unit 26 will be described in detail.

  In addition to the block division unit 26A described above, the image processing unit 26 includes a comparison block determination unit 26B, a similarity calculation unit 26C, a similarity comparison unit 26D, a count unit 26E, a count number comparison unit 26F, and a determination / detection unit 26G. Have

  Based on the optical flow table T, the comparison block determination unit 26B determines the blocks B to be compared from the plurality of blocks B of the captured image G captured with a predetermined time difference. At that time, the comparison block determination unit 26B reads a predetermined optical flow table T set in advance from the data stored in the optical flow table 27B of the storage unit 27, and based on the data related to the vector W, etc. The blocks B to be compared in the image G are estimated and determined.

FIG. 6 is a schematic diagram for explaining the determination of the blocks B to be compared, and shows new photographed images G in order from the top. In the figure, each captured image G is schematically represented by a plurality of blocks B, and a vector W of each block B acquired from the optical flow table T is also shown in each block B.
In the same manner as described with reference to FIG. 5, the comparison block determination unit 26B is a block to be compared between the two captured images G divided into a plurality of blocks B (see FIG. 4A) by the block division unit 26A. B is estimated.

  Specifically, as shown in the drawing, for example, an object such as a person P who was in the block B10 in the captured image G (t) at the time t from the position vector W10 at the previous time t−1. It is presumed that the position was in the position of the block B11 in the captured image G (t-1) at that time. Similarly, the block B20 in the photographed image G (t) corresponds to the block B21 in the photographed image G (t-1) at time t-1 from the position vector W20. In the same manner as described above, the blocks B11 and B21 in the captured image G (t-1) at time t-1 correspond to the captured image G (t-2 at the previous time t-2, respectively. ) In block B12 and block B22. Further, the blocks B12 and B22 in the captured image G (t-2) at the time t-2 correspond to the blocks B13 and B23 in the captured image at the previous time t-3, respectively. .

  In this way, the comparison block determination unit 26B, from the vector W for each block B of the optical flow table T, for each of the plurality of blocks B of the captured image G having a predetermined time difference, each corresponding block position (flow The position B) is estimated and the blocks B to be compared between the two images G are determined. Based on the determination by the comparison block determination unit 26B, the image processing unit 26 (see FIG. 2) uses the similarity calculation unit 26C to acquire the current captured image G (t) acquired and the previous captured image G ( From t−1), the degree of similarity between the determined blocks B is calculated.

  The similarity calculation unit 26C is a means for calculating the similarity by comparing each block B of one captured image G (t) with the corresponding block B of the other captured image G (t-1). Then, the respective similarities are sequentially calculated over a plurality of blocks B of the captured image G (t). That is, the similarity calculation unit 26C indicates that the current block B and the data in the optical flow table T at the block position exist in the previous captured image G (t-1). Search for block B at the position where it would be. In this way, the similarity between the blocks B is calculated only at the flow position estimated based on the optical flow table T, respectively. At this time, here, block matching is performed on both blocks B to be compared, and the similarity is calculated and acquired.

  Here, a method of calculating the similarity between the blocks B will be described by taking, as an example, SAD (Sum of Absolute Difference) for calculating the sum of the absolute difference values of general pixel values. First, if the number of pixels of the entire screen of the captured image G is M × N pixels and each block B to be extracted is m × n pixels, M / m × N / n blocks B are formed on the entire screen. When block matching is performed on each of these blocks B, the matching coefficient R between the corresponding blocks B of the captured image G at time t and time t−1 that is a predetermined time before that is expressed by the following equation (several 1).

In this formula, the tone value of each pixel in _{I t} is time t (e.g., luminance _values), a gradation value of each pixel in _{I t-1} at time _{t-1, x i, y} j is This is the coordinates of each pixel in the target block B within the same block. The smaller the matching coefficient R expressed in this way, the higher the similarity between the blocks B, and the larger the matching coefficient R, the lower the similarity between the blocks B.

  The image processing unit 26 according to the present embodiment, based on the similarity calculated for each of the plurality of blocks B as described above, determines the moving escalator ES based on the overall height, the height distribution in the captured image G, and the like. The movement of the person P above is monitored. Specifically, here, the means for monitoring includes the above-described similarity comparison unit 26D, count unit 26E, count number comparison unit 26F, and determination / detection unit 26G.

  The similarity comparison unit 26D is a means for thresholding each similarity between the blocks B calculated by the similarity calculation unit 26C, and sequentially calculates the calculated similarity and a predetermined similarity threshold. Compare. Thereby, the similarity comparison unit 26D determines the block B that cannot be regarded as the same as the corresponding block B before a predetermined time from the plurality of blocks B of the current photographed image G, and is irregular except for the movement by the escalator ES. A block B indicating motion is extracted.

FIG. 7 is a schematic diagram for explaining the extraction of the block B by the threshold value processing of the similarity of the block B, and similarly shows each captured image G and the like corresponding to FIG. 6 described above.
Also, the figure shows an example in which block matching is performed with one block B11 of the captured image G (t) at time t and the corresponding block B12 of the captured image G (t-1) at time t-1. Yes.

  As shown in the figure, the similarity comparing unit 26D performs the threshold processing of the similarity between both the blocks B11 and B12, and the similarity is determined to be lower than the predetermined threshold and the two blocks B11 and B12 are not similar. Sometimes, the block B11 of the photographed image G (t) is extracted as a block candidate in which a violation or a dangerous action has been made. Similarly, the similarity comparison unit 26D determines whether to extract all blocks B of the captured image G (t) by threshold processing.

  Note that this similarity threshold value is input and set in advance by the user or the like via the input unit 23 (see FIG. 2) before monitoring, and is stored in the setting data 27C of the storage unit 27. It is read and used by the similarity comparison unit 26D. Further, this threshold value is appropriately set according to each situation of the monitoring area. In this case, for example, when the block B is composed of a total of 16 pixels of 4 × 4 pixels, and the gradation value takes a value from 0 to 255, the matching coefficient R described above has a value from 0 to 4080. It can take. Normally, when the difference between the gradation values is 20 or more, the same color cannot be considered, and the total value in the block B is 20 × 16 = 320, so the threshold value of the matching coefficient R is set to 320. In this case, when the matching coefficient R indicating the similarity is 320 or more, the similarity of the block B is extracted as being lower than a predetermined similarity threshold.

FIG. 8 is a schematic diagram illustrating an example of an image obtained by extracting the block B having a low degree of similarity as described above, and photographing in a state where there is a pedestrian P1 ascending on the step S of the escalator ES illustrated in FIG. 11 described above. The example which extracted the block B from which the similarity degree is lower than a threshold value from the image G is shown.
In the photographed image G, as shown in the figure, since the change in the image is large in the region J1 (the region indicated by the oblique lines in the figure) where the pedestrian P1 is located, the similarity of the block B in the region becomes low, and the It is determined and extracted as a block B having a movement such as an action or a dangerous action.

  From the comparison result by the similarity comparison unit 26D, the image processing unit 26 counts the number of blocks B whose calculated similarity is lower than a predetermined threshold by the counting unit 26E (see FIG. 2). The number of blocks B having the irregular vector of Further, the image processing unit 26 compares the count number of the block B (the number of blocks B having low similarity) by the count unit 26E with a threshold value of a predetermined count number by the count number comparison unit 26F. Further, the image processing unit 26 detects the movement of the person P on the escalator ES by the determination / detection unit 26G based on the comparison result of the count number, and when the count number exceeds the threshold, It is determined that violations or dangerous acts have occurred. In addition, according to the determination result of the determination / detection unit 26G, for example, the warning device 3 is operated via the interface unit 21 to generate a warning alarm toward the escalator ES.

  At that time, the determination / detection unit 26G determines whether an action requiring a warning is performed based on the count number of the extracted block B (see FIG. 8). You may warn when it is more than the predetermined ratio (for example, 50%) of the number of the whole blocks B, or you may judge by a different standard (threshold) according to monitoring time etc., and may warn. As described above, the threshold value of the count number is appropriately set according to the monitoring situation and the like, and is stored in advance in the setting data 27C of the storage unit 27. However, in actuality, the person P moves slightly on the escalator ES, and the corresponding block B is extracted like noise. Therefore, the threshold value of the count number is the number of blocks corresponding to the noise during normal operation in advance. Are detected as a noise amount, and the noise amount including them is set to be equal to or greater than the noise amount.

Next, procedures and operations for monitoring the person P on the escalator ES by the monitoring system 1 and the monitoring device 10 (see FIG. 1) described above will be described.
FIG. 9 is a flowchart showing a procedure related to the monitoring process and operation of the monitoring system 1 and the monitoring device 10.

  Prior to the start of monitoring, as shown in the figure, first, a reference image such as an escalator ES is captured and acquired by the monitoring camera 2, and an optical flow (from the reference captured image is extracted by an optical flow extraction unit 25A of the background data setting unit 25). (See FIG. 3) is extracted (S101). Subsequently, the optical flow table setting unit 25B creates an optical flow table T (see FIG. 4) for each of the plurality of blocks B divided by the block dividing unit 26A from the extracted optical flow (S102), and as background data It is set and stored in the optical flow table 27B of the storage unit 27. Further, the captured image G during normal operation of the escalator ES is subjected to image processing or analysis processing by the image processing unit 26, and the above-described noise amount and the like in that state are detected, and determined and set based on them. The threshold TH1 of the similarity of the block B (here, the matching coefficient R) and the threshold TH2 of the count number are stored in the setting data 27C (S103).

  After the start of monitoring, first, captured images G with a predetermined time difference in the monitoring area are sequentially acquired by, for example, capturing images including the escalator ES by the monitoring camera 2 at predetermined intervals (S104), and image data 27A. To remember. Next, based on the determination of the comparison block determination unit 26B using the optical flow table T from these captured images G, the similarity calculation unit 26C performs the previous one for each block B of the current captured image G. Block matching (see FIG. 7) with the block B to be compared is performed. Thereby, here, the matching coefficient R is calculated as the similarity of each block B of the current photographed image G (S105), the matching coefficient R of each block B is acquired, and the person on the escalator ES is based on them. The movement of P is monitored.

  Specifically, in the present embodiment, first, the calculated matching coefficient R representing the similarity of each block B is compared with a preset threshold TH1 by the similarity comparing unit 26D, and the matching coefficient R is set to the threshold. It is determined whether or not it is greater than TH1 (S106). As a result, when the matching coefficient R is smaller than the threshold value TH1 (S106, NO), it is determined whether or not the processing for all the blocks B of the captured image G has been completed (S109). ) Shifts to the next block B (S112), and calculates a matching coefficient R of the block B (S105). On the other hand, when the matching coefficient R is equal to or higher than the threshold TH1 and the similarity of the block B is lower than the predetermined threshold (S106, YES), the block B is counted by the counting unit 26E as a violation act block candidate here ( S107).

  Next, the count number comparison unit 26F compares the count number of the block B with a threshold value TH2 of a predetermined count number. As a result, when the ratio of the violation block B is equal to or greater than the threshold value TH2 (S108, YES), the determination / detection unit 26G determines that a violation has been made on the escalator ES, etc. Is detected (see FIG. 8), and a warning alarm is issued by the warning device 3 (S111). On the other hand, when the ratio of the violation block B is smaller than the threshold value TH2 (S108, NO), it is determined whether or not all the blocks B have been processed (S109). ) Move to the next block B (S112), and repeat the above-described processes and the like until the processes of all the blocks B are completed (S109, YES). As described above, when the processing of all the blocks B of one photographed image G is completed (S109, YES), the process proceeds to the process of the photographed image at the next time (S110), and the image is acquired (S104). And monitoring the person P on the escalator ES continuously or at predetermined time intervals.

  At the time of the monitoring process described above, the monitoring apparatus 10 does not calculate the optical flow for each captured image G each time, but based on the preset optical flow table T, and performs block matching only at the estimated flow position. The similarity of each block B is calculated, for example, and monitoring processing is executed based on the similarity. Therefore, the monitoring device 10 can relatively accurately detect the movement of the person P on the escalator ES with a small amount of calculation while maintaining the monitoring accuracy. In connection with this, when there are pedestrians such as a person P walking at a speed higher than the speed set on the escalator ES, or when there is a person P who leans outside from the handrail, violations or dangerous actions on the escalator ES, etc. Can be detected by simple image processing with a small processing load, and the safety of the escalator ES can be improved. Therefore, according to this embodiment, the moving body on the moving object can be easily and accurately monitored by the monitoring device 10, and the amount of calculation at that time can be reduced to reduce the processing load for monitoring.

  Here, in the present embodiment, the entire captured image G is divided into blocks B to calculate the similarity and the like. However, a part of the set area of the captured image G is divided into blocks B, and only that area is calculated. For example, a part of the captured image G may be monitored. The block B having a similarity lower than the predetermined threshold may be counted for the entire captured image G. The captured image G is divided into a plurality of areas and counted for each area. It may be compared with the threshold value of each different count number set.

FIG. 10 is a schematic diagram showing an example of setting areas for the captured image G.
Here, as shown in the figure, the photographed image G is divided into a central area K1 including the entire escalator ES and its periphery (near the handrail H) and an outer area K2 located on both sides thereof, and photographed. A plurality of areas K1 and K2 are set in the image G.

  Thereby, according to the situation of each area K1, K2, in the central area K1 where noise is likely to be generated by the movement of the person P, the threshold value of the count number of the block B is set high. In the outer area K2, it is possible to perform appropriate monitoring for each of the areas K1 and K2, for example, by setting the threshold value low. At that time, for example, the noise amount of the count number generated in the normal operation is about 25% of the number of blocks B in the central area K1, and about 8% of the number of blocks B in the outer area K2. In some cases, the threshold value for each count number is set to 30% of the number of blocks in the area in the central area K1, 10% of the number of blocks in the area in the outer area K2, and the like. In this way, when a person P (see FIG. 12) enters the outside area K2 after getting out of the handrail H, an unexpected or high-risk action or movement is accurately and reliably detected. And detection accuracy can be increased.

  In this case, the image processing unit 26 (see FIG. 2) of the monitoring apparatus 10 is provided with a means for setting a plurality of areas for the captured image G, and thereby for each of the plurality of areas set in advance. The movement of the person P on the escalator ES is monitored. In addition, the count unit 26E counts the number of blocks B whose calculated similarity is lower than the threshold value for each of a plurality of areas, and the count number comparison unit 26F calculates the count number of blocks B for each of the plurality of areas and the area. The threshold value of the count number of the corresponding area is compared among the threshold values of the count number determined in advance.

  As described above, the apparatus for monitoring the escalator ES has been described as an example. However, the present invention is applied to an apparatus for monitoring a moving body on another moving object capable of setting the optical flow table T, such as a moving sidewalk or a conveyor. You can also. The present invention can also be realized as a program for causing the computer of the monitoring apparatus 10 to function as each unit of the monitoring apparatus 10 described above.

It is a schematic diagram which shows schematic structure of the monitoring system containing the monitoring apparatus of this embodiment. It is a functional block diagram which shows schematic structure of the monitoring apparatus of this embodiment. It is a schematic diagram for demonstrating the setting of background data. It is a schematic diagram for demonstrating the setting of an optical flow table. It is a schematic diagram for demonstrating an optical flow table. It is a schematic diagram for demonstrating determination of the blocks which should be compared. It is a schematic diagram for demonstrating extraction of the block by the threshold value process of a block similarity. It is a schematic diagram which shows the example of the image which extracted the block with low similarity. It is a flowchart which shows the procedure regarding the monitoring process and operation | movement of the monitoring system and monitoring apparatus of this embodiment. It is a schematic diagram which shows the example of a setting of the area with respect to a picked-up image. It is a schematic diagram for demonstrating the monitoring method of the person on the escalator by the conventional optical flow. It is a schematic diagram for demonstrating the other example of the monitoring method of the person on the escalator by the conventional optical flow.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Surveillance system, 2 ... Surveillance camera, 3 ... Warning device, 10 ... Surveillance device, 11 ... CPU, 12 ... RAM, 13 ... ROM, 21 ... Interface unit, 22 ... control unit, 23 ... input unit, 24 ... display unit, 25 ... background data setting unit, 25A ... optical flow extraction unit, 25B ... optical flow table setting , 26... Image processing unit, 26 A... Block division unit, 26 B... Comparison block determination unit, 26 C .. similarity calculation unit, 26 D .. similarity comparison unit, 26 E. , 26F... Count number comparison unit, 26G... Determination / detection unit, 27... Storage unit, 27A... Image data, 27B. ···bus, ... Block, G ... captured image, ES ... escalator, P ... man, S ... step, T ... optical flow table, H ... handrail, V ... vector.

Claims (5)

A monitoring device for monitoring a moving object on a moving object,
Means for dividing the captured image of the surveillance area captured by the surveillance camera into a plurality of blocks;
Means for extracting an optical flow from a reference captured image of the moving object or the moving object and the moving object;
Means for setting an optical flow table for each of the plurality of blocks from the extracted optical flow;
Means for determining blocks to be compared from a plurality of blocks of the captured image with a time difference based on the optical flow table;
Means for calculating the degree of similarity between the determined blocks;
Means for monitoring the movement of the moving object on the moving object based on the calculated similarity for each of the plurality of blocks;
A monitoring device comprising: The monitoring device according to claim 1,
The monitoring means compares the calculated similarity with a predetermined threshold of the similarity; and counts the number of blocks with the calculated similarity lower than the threshold; Means for comparing the count number of the block with a predetermined threshold value of the count number, and means for detecting movement of the moving body on the moving object based on the comparison result of the count number A monitoring device characterized by that. In the monitoring device according to claim 1 or 2,
Means for setting a plurality of areas for the captured image;
The monitoring device, wherein the monitoring means monitors the movement of the moving body on the moving object for each of the plurality of areas. The monitoring device according to claim 3, wherein
The means for counting counts the number of blocks whose calculated similarity is lower than the threshold value for each of the plurality of areas, and the means for comparing the number of counts counts the number of blocks for each of the plurality of areas. A monitoring apparatus that compares a predetermined threshold value of the count number of the area.   The program for functioning the computer of a monitoring apparatus as each means of the monitoring apparatus as described in any one of Claim 1 thru | or 4.

Images