JP4783898B2 - Abnormal event detection method at intersections by image processing - Google Patents

Description

  The present invention relates to a method for processing a time-series image at an intersection, tracking a moving object in the image, and determining the presence or absence of an abnormal event.

  By capturing a moving object with a video camera, performing image processing, and tracking the moving object on the image, it is possible to automatically detect a traffic state or a traffic accident.

  In Patent Document 1 below, when vehicles are crowded, images taken with a TV camera are overlapped, so the number and speed of the vehicle cannot be measured microscopically. Measure the congestion level by measuring fast and slow, plot the congestion level in a plane, and judge that something abnormal has occurred near the place where the congestion level has changed significantly compared to the surrounding conditions. Is described. In addition, it is described that an abnormality is determined when the travel line is shifted by a certain portion.

However, since this is a macro-statistical method, when this is applied to a complex area such as an intersection, the degree of congestion is averaged and abnormal events cannot be detected. Furthermore, this patent document does not disclose how to implement a macroscopic or statistical method.
JP-A-6-76195

  In view of the above problems, an object of the present invention is to detect an abnormal event at an intersection by image processing that can simplify the handling of an intersection as a combination of a plurality of moving object paths and determine the presence or absence of an abnormal event. It is to provide a method.

In a first aspect of the present invention, in a method for detecting an abnormal event by processing a time-series image at an intersection stored in a storage means,
(A) Preliminarily set data of all routes predicted to pass by vehicles or pedestrians on the intersection image;
(B) Find the position and velocity vector of each vehicle,
(C) Determine whether the vehicle is stopped,
(D) Based on the route data and the position of the vehicle, determine which route the vehicle is on;
(G) If it is determined in step (c) that the vehicle is stopped, it is determined whether the vehicle stop is a normal event in relation to a vehicle on another route or a traffic signal;
(H) If the vehicle cannot be determined to be any normal event, it is determined that the vehicle stop is an abnormal event.
Has steps.

In the second aspect of the method for detecting an abnormal event at an intersection by image processing according to the present invention, in the first aspect, between the step (d) and the step (g),
(E) When an affirmative determination is made in step (c), it is determined whether or not the vehicle is the stop vehicle located on the most front side in the route for the route of step (d);
(F) If it is determined in step (e) that the vehicle is at the foremost stop side, it is determined whether or not there is a vehicle in front of the stop vehicle in the route.
The method further includes a step, and when it is determined in step (f) that there is no vehicle on the front side, the process proceeds to step (g).

In the third aspect of the abnormal event detection method at an intersection by image processing according to the present invention, in the first aspect,
A set of trajectories that are time series of vehicle positions obtained in step (b) is stored in the storage means,
In step (a), all the routes are obtained based on the set.

In the fourth aspect of the abnormal event detection method at an intersection by image processing according to the present invention, in any one of the first to third aspects,
In the step (g), it is determined whether the traffic signal of the pedestrian route intersecting the route of the vehicle is a progress signal or whether it changes to a progress signal within a set time. If a positive determination is made, it is determined that the event is a normal event.

In the fifth aspect of the method for detecting an abnormal event at an intersection by image processing according to the present invention, in the second aspect,
In step (g), if the route crossing the route of the vehicle also proceeds to step (g), it is determined that there is no abnormal event.

In a sixth aspect of the abnormal event detection method at an intersection by image processing according to the present invention, in any one of the first to fifth aspects, in step (d), a past route of the vehicle is also stored. ,
(I) If it is determined that the vehicle is stopped in step (c), if there is a vehicle whose route has been changed from the route of the stopped vehicle to another route within the set range from the stopped vehicle, To determine an event,
It further has a step.

In the seventh aspect of the abnormal event detection method at an intersection by image processing according to the present invention, in the fourth aspect,
(J) An average value of vehicle speeds existing in a region of the vehicle route before the pedestrian route is obtained, and based on a step change of the average value, a state change of a traffic signal signal is recognized.
And further comprising steps
In step (g), the state change of the signal is used as the traffic signal of the pedestrian route.

According to the configuration of the first aspect,
(A) Preliminarily set data of all routes predicted to pass by vehicles or pedestrians on the intersection image;
(B) Find the position and velocity vector of each vehicle,
(C) Determine whether the vehicle is stopped,
(D) Based on the route data and the position of the vehicle, determine which route the vehicle is on;
(G) If it is determined in step (c) that the vehicle is stopped, it is determined whether the vehicle stop is a normal event in relation to a vehicle on another route or a traffic signal;
(H) If any normal event cannot be determined, it is determined that the vehicle stop is an abnormal event, so that complicated events at the intersection are simplified and the processing is simplified. There is an effect that the presence or absence of an abnormal event at the intersection can be determined relatively easily.

  Other objects, configurations and effects of the present invention will become apparent from the following description.

  Embodiment 1 of the present invention will be described below with reference to the drawings. In the drawings, the same or similar elements are denoted by the same or similar reference numerals.

  FIG. 1 is a schematic functional block diagram of an abnormal event detection apparatus 20 according to a first embodiment of the present invention that processes an image captured by a video camera (for example, an ITV camera) video camera 10 to track a moving object.

  The abnormal event detection device 20 other than the storage unit can be configured by computer software, dedicated hardware, or a combination of computer software and dedicated hardware.

  The time-series images taken by the video camera 10 are sampled at a rate of 30 frames / second, for example, and stored in the image memory 21, and the oldest frame image is rewritten with a new frame image.

  In the present invention, not only when image data stored in the image memory 21 is processed in real time by the blocks 23 to 29 but also when stored in an external storage device (not shown) and only necessary portions are processed later. Is also applicable.

  Further, even if the image (original image) stored in the image memory 21 is directly processed in the blocks 23 to 29, it is converted into a spatial difference frame image by applying a filter such as a Laplacian filter or two-dimensional Fourier transform. A configuration may be used in which blocks 23 to 29 are processed after inverse Fourier transform (transformed image) after emphasizing components of a predetermined frequency or higher. Hereinafter, “image” means an original image or a converted image.

  The background image generation unit 22 includes a storage unit and a processing unit. The processing unit accesses the image memory 21, and for example, the pixel values of the corresponding pixels of the frame image of all or thinned out objects in the past one minute. A histogram is created, and an image having the mode value (mode) as the pixel value of the pixel is generated as a background image in which no moving object exists, and this is stored in the storage unit. The background image is updated by performing this process periodically.

  In the ID generation / annihilation unit 23, area data of entrance slits 31A and 31B and exit slits 32A and 32B arranged in the frame image 30 as shown in FIG. In FIG. 2, these slits and block boundary lines indicated by dotted lines are superimposed on the frame image 30. One block is, for example, 8 × 8 pixels.

  The slit is set by a setting means including an input device IN such as a pointing device such as a mouse or a keyboard, a display device OUT for confirming the setting, and an input processing program included in the ID generation / erasure unit 23. Performed based on the operation. In FIG. 2, the depth of the slit is 2 blocks wide, but may be an integer multiple of 1 block width.

  The ID generation / annihilation unit 23 reads the image data in the entrance slits 31A and 31B from the image memory 21, and determines whether or not there is a moving object inside each block. Whether there is a moving object in a block is determined by whether the sum of absolute values of differences between the pixels in the block and the corresponding pixels in the background image is greater than or equal to a threshold value.

  When the ID generation / annihilation unit 23 determines that there is a moving object in the block, the ID generation / annihilation unit 23 assigns a new moving object identification code (ID) to the block. If the ID generation / annihilation unit 23 determines that a moving object exists in a block adjacent to the ID-assigned block, the ID generation / annihilation unit 23 assigns the same ID as the assigned block to this adjacent block. This ID-added block includes a block adjacent to the slit. Hereinafter, a block of blocks to which an ID is assigned is referred to as a cluster.

  The ID is assigned to the corresponding block of the object map in the storage unit object map storage unit 24. The object map is for storing the moving object information of each block of 60 × 80 blocks in the above-described case. The moving object information includes a flag indicating whether or not an ID is assigned, and the ID is assigned. In addition, the ID and the block motion vector (MV) are included. This information can be represented by an array M (i, j, n). Here, (i, j) indicates a block in the i-th row and j-th column, and n = 0, n = 1, and n = 2 indicate that M is a flag, ID, and MV, respectively. Instead of using the flag, it may be determined that no ID is assigned when ID = 0. The most significant bit of the ID may be used as a flag. The MV of a block whose MV cannot be obtained by block matching is set equal to, for example, the average value of MVs of blocks having the same ID around it.

  For the cluster that has passed through the entrance slit 31A or the entrance slit 31B, the moving object tracking unit 25 performs tracking processing by the method described in Patent Document 1, for example. For example, for high-speed processing, the approximate movement range of the cluster at time t is estimated based on the MV at time t-1 (frame number), and the above-mentioned moving object existence determination is performed for each block within this range. ID is assigned to the block on the moving direction side, and ID is erased on the block on the opposite direction to the movement, and the cluster at time t is determined. Further, the MV of the block to which the ID is assigned at time t is obtained by block matching between the frame images at time t−1 and t. For high-speed processing, the range in which block matching is performed is determined based on the MV at time t-1. As described in Patent Document 1, the ID and MV of each block can be determined simultaneously using an evaluation function. The ID and MV are updated on the object map in the storage unit object map storage unit 24.

  The tracking processing by the moving object tracking unit 25 is performed up to the exit slit 32A or the exit slit 32B for each cluster.

  The ID generation / annihilation unit 23 further checks whether an ID is assigned to the blocks in the exit slits 32A and 32B based on the object map in the storage unit object map storage unit 24, and if so, the cluster is exited. When passing through the slit, the ID is extinguished.

  FIG. 4 shows the ID object map at time t0 superimposed with the moving objects 33 and 34 on the image in FIG. 2 for easy understanding, and the boundaries of these clusters 33C and 34C are indicated by bold lines. It is.

  The cluster inscribed rectangle determining unit 26 in FIG. 1 determines rectangles 33D and 34D that are inscribed in the clusters 33C and 34C for each ID on 35, as shown in FIG. In FIG. 5, the cluster 34C and the rectangle 34D are the same.

  Next, how to obtain the representative points and measurement points of clusters for each ID will be described.

  It is assumed that the moving objects 33 and 34 that were at the position shown in FIG. 2 on the frame image 30 have moved to the positions shown in FIG. As shown in FIG. 7, since the distance between the moving object 33 and the video camera 10 is different, and the angle between the straight line from the video camera 10 to the road surface and the road surface is different depending on the position of the moving object 33, The length (reduction ratio) on the image with respect to the length of the image differs depending on the position on the image. Since the image on the imaging plane of the video camera 10 is proportional to the actual projection image on the plane projection plane SC perpendicular to the optical axis of the video camera 10, the reduction ratio is δS on the projection plane SC and this. Is proportional to the ratio μ = δS / δR with δR on the road surface corresponding to. This proportionality constant is 1, that is, the reduction ratio is μ. A product obtained by multiplying each minute line segment of the locus on the image by μ at the position corresponds to the line segment of the actual locus.

  On the other hand, when the representative point of the moving object is obtained as the geometric center of gravity of the region, the height differs for each moving object. For this reason, if the speed of the moving object is measured based on the representative point, an error occurs. The camera side end of the moving object, for example, the moving object rear end when the moving object moves away from the video camera 10, and the moving object front end when moving in the opposite direction are closest to the road surface, and the video If it becomes a position that can be seen from the camera 10, and this is used as a measurement point, the speed measurement error can be reduced.

  However, when the camera angle is lowered to widen the field of view for moving objects, or when the vehicle is congested, multiple moving objects overlap in the traveling direction on the image, and this measurement point is hidden. Problems arise. Further, when a monochrome grayscale image is used to perform real-time processing at high speed, a moving object and its shadow are recognized as a moving object, the measurement point is not stable, and the error becomes relatively large.

  Thus, in the first embodiment, this problem is solved as follows.

  When the ID generation / annihilation unit 23 determines in FIG. 2 and FIG. 4 that the rear end 33C1 of the cluster 33C has completely passed the rear end 31A1 of the entrance slit 31A, it notifies the initial value determination unit 27 of this. This time is set as t0.

  In response to this, the initial value determination unit 27 obtains a representative point of the moving object region based on the object map 35 at time t0. For example, in the object map 35 shown in FIG. 4, the geometric gravity center position of the cluster 33 </ b> C is obtained as a representative point of the moving object 33. The block to which the ID of the cluster 33C is assigned can be found in a short time by scanning the block in the rectangle obtained in block 26.

  Here, the representative point of the moving object 33 at time t is the representative point RP (t), the offset vector from the representative point to the measurement point is the offset vector offset vector, and the motion vector of the representative point (representative motion vector) is the representative motion vector representative. This is expressed as a motion vector. FIGS. 8A and 8B show these at times t0 and tk, respectively. Further, the reduction ratio of the block in the i-th row and the j-th column is expressed as μ (i, j).

  On the other hand, the motion vector average value calculation unit 28 in FIG. 1 obtains the average value of the motion vectors MV of all the blocks included in each moving object region as a representative motion vector representative motion vector in the object map of the motion vector MV.

  The initial value determination unit 27 further acquires the representative motion vector RMV (t0) from the motion vector average value calculation unit 28, and in FIG. 8A, the representative motion vector RMV (t0) starting from the representative point RP (t0). ) Is scanned on the frame image 30 in the direction approaching the video camera 10, and the outermost edge position in the cluster 33C is obtained as the initial measurement point MP (t0).

  The representative point tracking unit 29 in FIG. 1 cumulatively adds the representative motion vector sequentially obtained by the motion vector average value calculating unit 28 to the representative point RP (t0) obtained by the initial value determining unit 27, thereby representing the representative point. The trajectory representative point RP (t) is obtained. FIG. 9 shows the positions of representative points at times (frame numbers) t0 to t5 when the moving object 33 moves linearly.

  The trajectory of the representative point thus obtained is stable because it is not affected by the block quantization error. Also, even if a part of the moving object is concealed by another moving object on the image, the representative point is hardly shifted (a little because the motion vector to be averaged decreases). The trajectory can be obtained more accurately. Further, for example, by processing a color image only within the cluster 33C only at the time t0, when the shadow is determined to be a moving object, the shadow is removed from the cluster 33C, and the representative point RP (t0) and the representative motion vector RMV are removed. If (t0) is obtained, then it is possible to obtain the representative point RP (t) accurately and at high speed without being affected by the shadow by processing only the monochrome image.

  Returning to FIG. 1, the reduction ratio map 2A includes a storage unit and a processing unit. The storage unit stores the above-described reduction rate map (array) μ, and the processing unit stores the initial representative point RP (t0). ) And the representative point RP (t) obtained by the representative point tracking unit 29, the coordinates (i, j) of the block to which the representative point belongs are obtained, and the reduction ratio μ (i, j) is traced to the measurement point. Supply to part 2B.

  The measurement point tracking unit 2B has an offset vector OFV (t) = − | offset vector OFV (t0) | · μ (i, j) / starting from the representative point RP (t) obtained by the representative point tracking unit 29. μ (i0, j0) · representative motion vector RMV (t) / | representative motion vector RMV (t) | However, this sign is reversed for a moving object approaching the video camera 10 side.

  That is, the length of the offset vector offset vector is obtained by multiplying the absolute value of the offset vector OFV (t0) obtained by the initial value determination unit 27 by μ (i, j) / μ (i0, j0). Here, (i, j) and (i0, j0) are block coordinates to which the representative points at times t0 and t belong, respectively. Further, the camera side of the straight line through which the representative motion vector RMV (t) obtained by the motion vector average value calculation unit 28 passes, in the case of FIG. 7, the moving object rear end side is obtained as the direction of the offset vector.

  The locus of the measurement point MP (t) is obtained by connecting the leading ends of the offset vector OFV (t) thus obtained. FIG. 9 also shows the locus of the measurement point MP (t) when the moving object 33 moves linearly.

  FIG. 10 shows the trajectories of representative points and measurement points obtained as described above when the moving object 33 turns to the right.

  By obtaining the measurement points in this way, the measurement points can be obtained accurately even if they are hidden by other moving objects on the image.

  The features of the first embodiment are the blocks 2C to 2I in FIG. 1 and the relationship between these blocks 29 and 2B, which will be described in detail below.

  For each measurement point MP obtained by the measurement point tracking unit 2B, the speed measurement unit 2C obtains an average value of motion vectors of the block including the measurement point MP and its surrounding blocks, and divides this by μ. The vehicle speed V is obtained. When the measurement point MP is concealed by another moving object, the vehicle speed V is obtained by expanding the averaging range.

  On the other hand, before the abnormality determination process, the process shown in FIG. 11 is performed by the route determination / storage unit 2D. In the following, the step identification codes in the figure are shown in parentheses.

  (S0) Based on the past time-series object map in which the contents of the object map storage unit 24 are stored in the external storage device, a cluster passage area is obtained as a path distinguished by the combination of the entrance slit and the exit slit. Before entering the entrance slit, the cluster passing region is obtained by tracing the moving object from the entrance slit. For example, an area as shown in FIG. 6 through which the cluster 33C of FIG. 4 has passed is obtained. That is, 1 is added to the cluster passing block. Such processing is repeated, and the vehicle route is obtained by binarizing the count sequence along each crossing line of the route with a threshold value.

  In FIG. 13, the vehicle routes AS, AR, and AL that enter the intersection from the road A, go straight, turn right, and turn left are obtained as described above. The same applies to the roads B to D, and a combination of these roads is as shown in FIG. Further, pedestrian routes AW, BW, CW and DW on the roads A to D are obtained. Further, among these routes, areas AC, BC, CC, and DC (dot areas) where the vehicle stops with a stop signal for each of the pedestrian paths AW, BW, CW, and DW are obtained for signal determination described later.

  (S1) The obtained route is displayed on the display device OUT of FIG.

  (S2) A dialog box indicating whether or not the route and area need to be corrected is displayed, and the user waits for selection from the input device IN. When the user selects correction necessity, this is corrected based on an instruction from the input device IN, and the process returns to step S1 to display the corrected one. If the user has selected that correction is unnecessary, the process proceeds to step S4.

  (S4) Store the data of the determined vehicle route and signal determination area in the storage unit.

  FIG. 14 is a combination of all routes, and is complicated. However, in this embodiment, as shown in FIG. 15, the vehicle is divided into 12 vehicle routes that enter the intersection from four directions and branch into three and four pedestrian routes, and the same for each vehicle route. Since processing is performed, processing is simplified.

  Here, a traffic signal is required to determine whether or not the stop of the vehicle is an abnormal event. However, depending on the installation location and angle of the video camera 10, it may be costly if the traffic signal goes out of view and the signal state cannot be determined by image processing, or the signal information is supplied to the abnormal event detection device 20 by wire or wirelessly. May be high.

  Therefore, the signal determination unit 2E obtains the average value of the representative speeds of the vehicles included in the areas AC and CC in FIG. 13, and similarly obtains the average value of the representative speeds of the vehicles included in the areas BC and DC. FIG. 17 shows a time chart of these average values. By averaging, the influence of the vehicle speed neglecting the signal is almost eliminated, and the attention signal is divided on both sides of the step change. The signal determination unit 2E detects and stores the on / off signal period and the change reference point, and when the step change amount is less than or equal to the threshold value, that is, the traffic amount decreases, and the traffic flow is detected. When the traffic signal change cannot be detected, the signal change is estimated based on the stored signal period and the change reference point. Next, when the step change amount exceeds the threshold value, the state of the traffic signal is determined based on the waveform, and the signal period and the change reference point are updated.

  The belonging route determination unit 2F determines to which route it belongs for each moving object ID, and the same ID is given to each rectangle determined by the cluster inscribed rectangle determination unit 26. If an area (cluster) is acquired from the object map storage unit 24 and this cluster is included in the path for a given path, the cluster is included in the path. The route identification code FP is associated with the moving object ID. FIG. 16 illustrates in an image that each of the moving objects with ID = 1 to 6 has the route identification code FP as an attribute. The value of each identification code FP is associated with the moving object in the overlapped part of the route.

  The feature quantity storage unit 2G for each moving object stores the feature quantity as a table for each moving object ID. This table can be expressed by defining a structure, further defining its array, and using the array subscript as a moving object ID. The table items (structural variables) are the inscribed rectangle diagonal point coordinates and representative points obtained by the cluster inscribed rectangle determining unit 26, representative point tracking unit 29, speed measuring unit 2C, and belonging route determining unit 2F. , The velocity vector, and the belonging route identification code FP.

  Next, processing in the abnormality determination unit 2H will be described.

  Here, if a collision accident occurs at the intersection, the vehicle stops after that, the vehicle in the path ahead does not exist, the rear vehicle is blocked by the passage and stops or bypasses the accident vehicle (Change route) and proceed. The same applies when a vehicle breaks down at an intersection. On the other hand, there is a case where the vehicle stops while waiting for a signal, or a straight-ahead vehicle and a right turn vehicle are stopped in a deadlock state.

  Therefore, all the normal events that are not abnormal events such as accidents and failures are examined, and if they do not correspond to any normal event, they are determined to be abnormal events.

  First, for each route, if there is a stop vehicle, it is checked whether there is a stop vehicle in front of it, and it is determined whether there is a vehicle ahead of the first stop vehicle in the route. This process can be simplified as follows. That is, by referring to the feature quantity storage unit 2G for each moving object, it is checked whether or not there is a stop vehicle (which may be regarded as a stop when the speed is equal to or less than a set value) regardless of the route. Check if there is more than one on a route, and if there are more than one, determine which is the first stop vehicle in the route and confirm that there are no vehicles ahead in that route To do.

  The abnormality determination unit 2H obtains the processing shown in FIG. 12 for the frame image sampled at time t by the moving object feature amount obtained by the moving object feature amount storage unit 2G and the route determination / storage unit 2D. And the traffic signal state obtained by the signal determination unit 2E.

  In the following, it is assumed that the moving object ID is assigned from 1 to imax in the ID object map at time t. Also, assume that there are 12 routes at the intersection and these are identified by numbers 1-12.

  (S10) The initial value 1 is substituted for the variable i of the moving object ID, and ST (j), j = 1 to 12 indicating the head stop vehicle ID in the route j is initialized to 0.

  (S11) If i is not imax, the process proceeds to step S12. Otherwise, the process proceeds to step S17.

  (S12) Referring to the feature quantity storage unit 2G for each moving object, it is determined whether or not the vehicle with ID = i is stopped. If it is stopped, the process proceeds to step S13. Otherwise, the process proceeds to step S16. .

  (S13) With reference to the moving object feature quantity storage unit 2G, the route j to which the vehicle with ID = i belongs is acquired. If it does not belong to any route, it is route 0. If ST (j) = 0, that is, if it is the first stop car found on the route j, the process proceeds to step S15, and if it is the second stop car or later, the process proceeds to step S14.

(S14) The representative point coordinates stored in the moving object feature quantity storage unit 2G and the route determination / storage unit 2D indicate which of the vehicles with ID = i or ID = ST (j) precedes the route j. And the route stored in the route. For example, the center line of the route is obtained in advance, the representative point coordinates are projected perpendicularly onto the center line, and it is determined which precedes the center line. Further, on the ID object map, the main scanning in the path direction may be performed while performing the sub-scanning in the direction orthogonal to the path, and the front or back may be determined depending on which ID is found later.

  If the vehicle with ID = i is ahead of the vehicle with ID = ST (j), the process proceeds to step S15, and if vice versa, the process proceeds to step S16.

  (S15) Substitute i for ST (j).

  (S16) Increment i by 1 and return to step S11.

  By such processing, it is possible to find the first stop vehicle for each route on the frame image at time t.

  (S17) The initial value 1 is substituted into the route variable j.

  (S18) If j <13, the process proceeds to step S19. Otherwise, the process in FIG. 12 is terminated.

  (S19) ST (j) = 0, that is, if the stop vehicle does not exist on the route j, the process proceeds to step S20, and if not, the process proceeds to step S21.

  (S20) Increment j by 1 and return to step S18.

(S21) In the path j, it is checked whether or not there is a vehicle ahead of the stopped vehicle of ID = ST (j). If there is no vehicle, the process proceeds to step S22, and if it exists, the process returns to step S18. For example, on the ID object map, main scanning is performed in the path direction while sub-scanning in the direction orthogonal to the path, and it is determined that the ID does not exist if no ID is found. The reason why the process of step S21 is performed is to prevent a case where a group of vehicles stopped in series start in order from the front is not determined as an abnormal event.

  (S22) It is checked whether or not the stop of the vehicle with ID = ST (j) corresponds to a possible normal event.

  For example, if there is a pedestrian crossing crossing the route j in front of the vehicle, the traffic signal of this pedestrian crossing is a progress signal or changes to a progress signal within a set time based on the information of the signal determination unit 2E. It is determined whether or not the prediction can be made. If the determination is affirmative, it is determined that there is no abnormal situation, and the process returns to step S18.

  Further, it is checked whether or not there is a stop vehicle for a route that intersects with the route j (a right-turn route and a straight-ahead route that intersect each other). If it exists, if the distance between both representative points is within the set value, it is determined that there is a deadlock condition and there is no abnormal situation, and the process returns to step S18.

  (S23) The abnormal event such as an accident or failure is determined, and this is notified to the abnormal memory / notification unit 2I.

  In response to this, the abnormality storage / notification unit 2I receives the image of the plurality of frames before and after the point determined to be abnormal, the abnormality determination time, and the feature amount data regarding the plurality of frames of the moving object determined to be abnormal. The data is stored in an external storage device and transmitted to a traffic center via a communication medium.

  For example, since the moving object with ID = 6 shown in FIG. 16 has FP = 1, 2 and 3, steps S13 to S15 in FIG. 12 are processed for j = 1, 2, 3 respectively. However, only one of them may be performed. In this case, the routes 1 to 3 are regarded as having no overlapping portion (for example, the length of the straight traveling route is not changed, but the right turn route and the left turn route are shorter because there is no overlapping portion).

  Further, the past belonging route of the moving object or the presence / absence of route change is also stored in the feature amount storage unit 2G for each moving object as the feature amount of the moving object, and between step S12 and step S13 or between step S22 and step S23. In the meantime, if there is a vehicle whose route is changed from the route of the stopped vehicle to another route within the set range from the stopped vehicle, the configuration may be such that it is determined as an abnormal event of an accident or failure. Alternatively, a configuration may be adopted in which moving objects whose routes have been changed are counted, and when the count exceeds a set value within a set time, it is determined that the event is an abnormal event of an accident or failure.

  According to the first embodiment, the intersection is simplified by dividing it into a plurality of routes through which the vehicle passes, and it is determined whether the vehicle stop is a normal event in consideration of other routes, and corresponds to any normal event. If it is not, it is determined as an abnormal event, so that a complicated event at the intersection can be analyzed comparatively easily, which greatly contributes to the practical use of the abnormal event automatic detection device at the intersection.

  Note that the present invention includes various other modifications.

  For example, instead of using the signal determination unit 2E, the signal of the traffic signal is received and the signal information of the traffic signal is obtained from the traffic center via the Internet, or the signal information is obtained by processing the image of the traffic signal. Of course, it may be a configuration.

  Further, the motion vector of the vehicle representative point may be used as the vehicle speed.

  Further, the route may be configured in advance by the user.

  Alternatively, the vehicle route may be a vehicle representative point route and the stop of the representative point may be handled as a vehicle stop. In this case, the representative point may be a geometric centroid.

  It may be possible to easily determine the preceding of a vehicle using a list structure in which IDs are linked in the order of progression of vehicles in the route.

  In the above embodiment, the average value of all the motion vectors in the cluster is obtained as the initial representative motion vector. However, the average value of the motion vectors of a plurality of blocks included in the moving object region may be used, and the vicinity of the representative point. The average value of the motion vectors of the blocks, for example, the block to which the representative point belongs and the eight surrounding blocks may be used.

  Further, in the above embodiment, the geometric center of gravity of the cluster is obtained as the geometric center of gravity of the moving object region to be obtained as the initial representative point. However, the geometrical center of the moving object region on the frame image corresponding to the cluster is obtained. The target center of gravity may be obtained.

  In the above embodiment, the end point of the initial offset vector starting from the initial representative point is obtained on the frame image. However, since the block size is relatively small, the point in the end block on the object map or the vicinity thereof A point may be obtained as the end point of the initial offset vector.

  Further, for example, in FIG. 3, the accuracy is higher when the initial representative point and the measurement point are determined on the side of the exit slit 32B than on the side of the entrance slit 31B far from the video camera. It is also possible to obtain the above and track the moving object by going back in time as described in Japanese Patent Application Laid-Open No. 2004-207786. This is when real-time processing is not required as in the analysis of past video. It is effective for. Further, a line may be used instead of the entrance slit or the exit slit, and ID may be given / disappeared in relation to the line such as line passing.

  In addition, the processing start time in the initial value determination unit 27 is the time when the set line has a predetermined relationship with the moving object (cluster), for example, the front end, the rear end, or the representative point of the moving object is the line. It may be the point of coincidence or passing through the line. This is because the predetermined relationship can be changed depending on where the line is set, and the moving object trajectory before entering the entrance slit can be obtained by going back in time. This line may be a side of the region.

  Further, in the above embodiment, the case where the reduction rate at the representative point is obtained has been described. However, this is an approximate one. For example, after obtaining the measurement point as in the above embodiment, the measurement point and the representative point are obtained. For example, the accuracy of the measurement point may be improved by obtaining the reduction ratio of the block to which the central point belongs and multiplying this by the initial offset length to determine the absolute value of the offset vector. Such processing may be repeated a plurality of times or until the absolute value converges. The degree of approximation also depends on the camera arrangement height, zoom magnification, camera angle, and the like.

It is a schematic functional block diagram of the abnormal event detection apparatus of Example 1 of this invention. It is explanatory drawing which overlap | superposed the slit of an entrance and an exit, and the block boundary line on the frame image of the time t0. It is explanatory drawing which overlap | superposed the slit of the entrance and exit, and the block boundary line on the frame image at other time. FIG. 3 is an explanatory diagram of an ID object map in which the moving object in FIG. 2 is overlaid on the object map of the moving object ID at time t0 for easy understanding, and the boundaries of these clusters are indicated by bold lines. It is a cluster inscribed rectangle explanatory drawing. It is explanatory drawing which shows the method of calculating | requiring a path | route automatically. It is a figure explaining the reduction ratio which is ratio of the length on the image with respect to actual length, and the cause which a measurement error produces. (A) and (B) are diagrams showing a representative point RP, a representative motion vector RMV, and an offset vector OFV at times t0 and tk, respectively. It is explanatory drawing of locus | trajectory RP (t) of the representative point which uses time t as a parameter, and locus | trajectory MP (t) of a measurement point when a moving object moves linearly. It is locus | trajectory explanatory drawing of the representative point and measurement point in case a moving object turns right. It is a flowchart which shows a route determination process. It is a flowchart which shows an abnormality determination process. It is explanatory drawing of a part of path | route in an intersection. It is route explanatory drawing which overlap | superposed all the routes in an intersection. . It is route explanatory drawing which divides and shows all the routes in an intersection into a some layer. It is a figure which shows with an image that each of the moving objects of ID = 1-6 has the path | route identification code FP as an attribute. It is a time chart which shows a vehicle speed average value corresponding to a traffic signal state.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Video camera 20 Abnormal event detection apparatus 21 Image memory 22 Background image generation part 23 ID production | generation / annihilation part 24 Object map memory | storage part 25 Moving object tracking part 26 Cluster inscribed rectangle determination part 27 Initial value determination part 28 Motion vector average value calculation Unit 29 representative point tracking unit 2A reduction rate map 2B measurement point tracking unit 2C speed measurement unit 2D route determination / storage unit 2E signal determination unit 2F belonging route determination unit 2G feature amount storage unit for each moving object 2H abnormality determination unit 2I abnormality storage / Notification unit 30 Frame image 31A, 31B Entrance slit 32B, 32A Exit slit 33, 34 Moving object 33C, 34C Cluster IN Input device OUT Display device RP Representative point RMV Representative motion vector OFV Offset vector MP Measurement point SC Projection plane

Claims (5)

In a method for detecting an abnormal event by processing a time-series image at an intersection stored in a storage means,
(A) Preliminarily set data of all routes predicted to pass by vehicles or pedestrians on the intersection image;
(B) Find the position and velocity vector of each vehicle,
(C) Determine whether the vehicle is stopped,
(D) Based on the route data and the position of the vehicle, determine which route the vehicle is on;
(E) When an affirmative determination is made in step (c), it is determined whether or not the vehicle is the stop vehicle located on the most front side in the route for the route of step (d);
(F) If it is determined in step (e) that the vehicle is located at the foremost side, it is determined whether or not there is a vehicle in front of the stopped vehicle in the route;

(G) If it is determined in step (f) that there is no vehicle on the front side, it is determined whether the stop of the vehicle is a normal event in relation to a vehicle on another route or a traffic signal. ,
(H) If the vehicle cannot be determined to be any normal event, it is determined that the vehicle stop is an abnormal event.
An abnormal event detection method at an intersection by image processing, characterized by comprising steps. In the step (g), it is determined whether the traffic signal of the pedestrian route intersecting the route of the vehicle is a progress signal or whether it changes to a progress signal within a set time. If a positive determination is made, it is determined that the event is a normal event.
The method for detecting an abnormal event at an intersection by image processing according to claim 1 . In step (g), if the route crosses the route of the vehicle also proceeds to step (g), it is determined as a normal event .
The method for detecting an abnormal event at an intersection by image processing according to claim 1 or 2 . In this step (d), the past route of the vehicle is also stored,
(I) If it is determined that the vehicle is stopped in step (c), if there is a vehicle whose route has been changed from the route of the stopped vehicle to another route within the set range from the stopped vehicle, To determine an event,
The method for detecting an abnormal event at an intersection by image processing according to any one of claims 1 to 3 , further comprising a step. (J) An average value of vehicle speeds existing in a region of the vehicle route before the pedestrian route is obtained, and based on a step change of the average value, a state change of a traffic signal signal is recognized.
And further comprising steps
The method for detecting an abnormal event at an intersection by image processing according to claim 2 , wherein in step (g), a change in state of the signal is used as a traffic signal of the pedestrian route.

Images