JP2006330942A - Traffic condition deciding system, image processor, computer program, and traffic condition deciding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a traffic condition deciding system for deciding such detail traffic conditions as traffic congestion, traffic abnormality, a traffic offense and traffic accident, an image processor configuring the traffic condition deciding system, a computer program for operating the image processor, and a traffic condition deciding method. <P>SOLUTION: A CPU starts clocking of a decision time T1 and a calculation time T2, and executes frame image processing, and decides presence/no presence of the lapse of the calculation time T2. When the calculation time T2 does not elapse, the CPU continuously executes the frame image processing. When the calculation time T2 elapses, the CPU executes traffic congestion decision processing and traffic abnormality decision processing. The CPU decides presence/no presence of the traffic abnormality, and when any traffic abnormality is present, the CPU decides presence/no presence of the lapse of the decision time T1. When the decision time T1 does not elapse, the CPU resets the calculation time T2, and repeats the frame image processing, the traffic congestion decision processing and the traffic abnormality decision processing. When the decision time T1 elapses, the CPU notifies a control center device (3) of the traffic abnormality. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a traffic situation determination system for determining a wide range of traffic conditions based on a captured image obtained by imaging an area including a road with an imaging apparatus, an image processing apparatus constituting the traffic situation determination system, and the image The present invention relates to a computer program for realizing a processing device and a traffic situation determination method.

  Road traffic systems aimed at improving the road traffic environment are being used in various fields, such as those that limit the inflow of vehicles to reduce traffic congestion, those that provide traffic-related information and optimize traffic flow, There are some which process captured images captured by a video camera and collect information such as the number of passing vehicles and traveling speed.

  As a method for detecting a traveling vehicle on a road, an image of only a road where no vehicle exists is created in advance, and a difference between an image when the vehicle is traveling and an image of only a road is calculated and calculated. There is one that extracts information on only the vehicle based on the information. However, this method sometimes erroneously detects a shadow on a road, a road marking, or the like as a traveling vehicle due to the influence of changes in the external environment such as sunshine changes and weather changes.

  On the other hand, a method for detecting a vehicle by calculating a motion vector of a moving body by calculating a difference or correlation information of captured images between a plurality of images at minute time intervals is affected by changes in the external environment. It is difficult to obtain information such as the speed and moving direction of the vehicle by calculating the motion vector.

For example, if the optical axis of the video camera is placed above the road with the road surface and the movement of each vehicle on the image captured by the video camera is recognized as a motion vector, even if the inter-vehicle distance is small, A traffic information measuring device capable of measuring traffic information such as the speed, direction, and number of vehicles has been proposed (see Patent Document 1).
JP-A-10-334395

  However, in the apparatus of Patent Document 1, traffic information (traffic parameters) such as the speed, direction, and number of vehicles can be measured simultaneously, but traffic jams, traffic violations, traffic accidents, abnormal driving, etc. The traffic situation cannot be judged. In order to effectively realize smooth traffic or accident prevention, not only traffic information such as the speed, direction and number of vehicles, but also more detailed traffic information is measured, and traffic conditions are accurately determined. A wide range of judgments were required.

  The present invention has been made in view of such circumstances, and calculates a motion vector of a pixel block based on a change in the position of the corresponding pixel block specified by a captured image at a different time of imaging, and the direction of the calculated motion vector is By calculating a motion vector of a connected block obtained by connecting adjacent pixel blocks that substantially match each other, and determining the traffic situation based on the calculated motion vector of the connected block or the distribution density of the motion vector calculated based on the motion vector, Provided are a traffic situation determination system capable of determining a traffic situation without being affected by environmental changes, an image processing device constituting the traffic situation judgment system, a computer program for realizing the image processing device, and a traffic situation judgment method. For the purpose.

  Another object of the present invention is to calculate a statistical value including at least one of an average value, a mode value, or a median value based on a motion vector distribution density and a motion vector calculated a plurality of times in a predetermined time. An object of the present invention is to provide an image processing apparatus that can determine the traffic situation more accurately by calculating.

  Another object of the present invention is an image that can accurately determine the degree of traffic congestion by determining the traffic congestion status based on the distribution density of motion vectors or the size of motion vectors of connected blocks. It is to provide a processing apparatus.

  Another object of the present invention is to determine a traffic abnormality when the angle difference between the direction of the motion vector of the connected block and the standard direction that determines the direction of traffic flow is approximately 180 degrees, thereby determining the direction of travel of the vehicle. It is an object of the present invention to provide an image processing apparatus that can easily determine the abnormality.

  Another object of the present invention is that the angle difference between the direction of the motion vector of the connected block and the standard direction that determines the direction of traffic flow is greater than or equal to the first threshold until the timed determination time exceeds a predetermined time. In some cases, it is an object of the present invention to provide an image processing apparatus that can easily determine that avoidance traveling for avoiding a parked vehicle, an accident vehicle, an obstacle, or the like is continued by determining a traffic abnormality. .

  Another object of the present invention is to provide determination images corresponding to a plurality of lanes in the same traveling direction in a captured image, and to connect one determination region until a measured determination time becomes a predetermined time or more. When the magnitude of the motion vector of the block is smaller than the second threshold value and the magnitude of the motion vector of the connected block in the other determination area is larger than the second threshold value, it is determined that the traffic is abnormal, thereby It is an object of the present invention to provide an image processing apparatus that can determine that a traffic abnormality has occurred.

  Another object of the present invention is to determine a traffic abnormality when the motion vector distribution density of the one determination area is larger than a third threshold until the measured determination time reaches a predetermined time or more. Accordingly, an object of the present invention is to provide an image processing apparatus capable of determining a traffic abnormality with higher accuracy.

  Another object of the present invention is to provide a captured image with determination areas corresponding to a plurality of lanes in the same traveling direction, and connect them in one determination area until the measured determination time becomes a predetermined time or more. Image processing that can determine a traffic abnormality with higher accuracy by determining a traffic abnormality when a block is not specified and the motion vector size of a connected block in another determination area is larger than the second threshold. To provide an apparatus.

  Another object of the present invention is to provide a captured image with a restriction area corresponding to traffic restriction, and the direction of the motion vector of the connected block that has entered the restriction area substantially coincides with the restriction direction of the traffic flow. In this case, it is desirable to provide an image processing device capable of determining traffic violations such as entering an entry prohibition area or traveling right when prohibiting a right turn by determining a traffic violation.

  Another object of the present invention is between a first imaging time point when the number of pixel blocks in an overlapping region where connected blocks overlap each other is a fourth threshold value and a second imaging time point before the first imaging time point. And providing an image processing apparatus that can easily determine the occurrence of a collision accident by determining a traffic accident when the difference in luminance value of the pixels in the overlap region is equal to or greater than the fifth threshold. There is.

  Another object of the present invention is to provide a difference between the first imaging time point and the second imaging time point of a luminance value of each pixel in the region where the connected block is present at the second imaging time point being a sixth threshold value or more. An object of the present invention is to provide an image processing apparatus capable of determining the occurrence of a collision accident with higher accuracy by determining a traffic accident in some cases.

  Another object of the present invention is to transmit a motion vector calculated by an image processing device to a determination device, which determines a motion vector distribution density or a connected block motion vector based on the received motion vector. An object of the present invention is to provide a traffic condition determination system that can easily determine a traffic condition at a remote place by calculating and determining the traffic condition.

  In addition, another object of the present invention is to provide an image pickup apparatus that picks up different areas including a road and three or more image processing apparatuses that process picked-up images picked up by the image pickup apparatus along the road. Traffic situation determination capable of determining a wide range of traffic jam conditions based on a captured image captured by the apparatus and captured images captured by the two most recent imaging apparatuses installed with the one imaging apparatus in between To provide a system.

  In addition, another object of the present invention is to determine the traffic jam situation at one time point and a plurality of other time points different from the one time point, thereby reducing misjudgment and accurately determining the traffic jam situation. An object of the present invention is to provide a traffic situation determination system that can do this.

  Another object of the present invention is to provide a traffic situation determination system that can determine a traffic jam change point based on a traffic jam situation determined from captured images captured by a plurality of continuous imaging devices. is there.

  An image processing apparatus according to a first aspect of the present invention is an image processing apparatus that processes an acquired captured image to determine a traffic situation, and that specifies a pixel block corresponding to a captured image at a different imaging time point, and the position of the pixel block Means for calculating a motion vector of the pixel block based on a change, means for specifying a connected block connecting adjacent pixel blocks having substantially the same direction of the calculated motion vector, and each pixel block of the connected block Vector calculation means for calculating a motion vector of the connected block based on the motion vector of the connected block; density calculation means for calculating a distribution density of the motion vector based on the number of pixel blocks of the connected block; and the distribution density or connected block. And determining means for determining a traffic situation based on the motion vector.

  The image processing apparatus according to a second invention is the image processing apparatus according to the first invention, wherein the density calculating means and the vector calculating means calculate the distribution density and the motion vector of the connected block a plurality of times in a predetermined time. On the basis of the distribution density and the motion vector of the connected block, a statistical value including at least one of the average value, the mode value, and the median value is calculated.

  An image processing apparatus according to a third invention is characterized in that, in the second invention, the determination means determines a traffic jam situation based on the distribution density or the size of a motion vector of a connected block. To do.

  The image processing apparatus according to a fourth aspect of the present invention is the image processing apparatus according to the first aspect, wherein storage means for storing a standard direction for determining the direction of traffic flow, and an angle for calculating an angle difference between the direction of the motion vector of the connected block and the standard direction. And a difference calculating means, wherein the determining means determines that the traffic is abnormal when the calculated angle difference is approximately 180 degrees.

  An image processing apparatus according to a fifth invention is the image processing apparatus according to any one of the first to fourth inventions, a storage unit that stores a standard direction that determines a direction of traffic flow, a direction of a motion vector of the connection block, and the standard direction. An angle difference calculating means for calculating an angle difference between the first angle threshold value and a means for measuring a determination time. The determination means is configured such that the calculated angle difference is equal to or greater than a first threshold until the determination time is equal to or greater than a predetermined time. In some cases, a traffic abnormality is determined.

  The image processing apparatus according to a sixth aspect of the present invention is the image processing apparatus according to any one of the first to fourth aspects, wherein the captured image includes determination areas corresponding to a plurality of lanes in the same traveling direction, and the magnitude of the motion vector of the connected block And a second time threshold, and a means for measuring a determination time, wherein the determination means is a magnitude of a motion vector of a connected block in one determination area until the determination time reaches a predetermined time or more. Is smaller than the second threshold and the motion vector of the connected block in the other determination area is larger than the second threshold, it is determined that the traffic is abnormal.

  An image processing apparatus according to a seventh aspect of the present invention is the image processing apparatus according to the sixth aspect, further comprising means for comparing the distribution density with a third threshold value, wherein the determination means is configured to perform the determination until the determination time reaches a predetermined time or more. When the distribution density of the determination area is larger than the third threshold value, it is determined that the traffic is abnormal.

  An image processing apparatus according to an eighth aspect of the present invention is the image processing apparatus according to any one of the first to fourth aspects, wherein a determination region corresponding to a plurality of lanes in the same traveling direction is provided in the captured image, and the magnitude of the motion vector of the connected block And a second time threshold, and a means for measuring a determination time, wherein the determination means does not identify a connected block in one determination area until the determination time reaches a predetermined time, When the magnitude of the motion vector of the connected block in the other determination area is larger than the second threshold, it is determined that the traffic is abnormal.

  An image processing device according to a ninth invention is the image processing device according to any one of the first to eighth inventions, wherein a restriction region corresponding to traffic restriction is provided in the captured image, and the restriction direction of traffic flow in the restriction region is stored. And a means for detecting entry of the connected block into the restricted area, and a means for comparing the direction of the motion vector of the connected block with the restricted direction, wherein the determining means is provided in the restricted area. It is characterized in that a traffic violation is determined when the direction of the motion vector of the connected block that has entered substantially coincides with the regulation direction.

  The image processing apparatus according to a tenth aspect of the present invention is the image processing apparatus according to any one of the first to ninth aspects, wherein means for identifying an overlapping area between the connected blocks based on a motion vector of the connected block; and a pixel block of the overlapping area A means for comparing the number and a fourth threshold value, wherein the determining means includes a first imaging time point at which the number of pixel blocks in the overlapping region is equal to or greater than the fourth threshold value, and a second time point before the first imaging time point. It is characterized in that a traffic accident is determined when the difference in luminance value of the pixels in the overlapping area from the imaging time point is not less than the fifth threshold value.

  An image processing apparatus according to an eleventh aspect of the present invention is the image processing apparatus according to the tenth aspect, wherein the determination means includes the first imaging time point and the second imaging point of the luminance value of each pixel in the region where the connected block exists at the second imaging time point. It is characterized in that a traffic accident is determined when the difference at the time is equal to or greater than the sixth threshold.

  A traffic situation determination system according to a twelfth aspect of the present invention is the traffic situation determination system in which an area including a road is imaged by an imaging device, and a captured image obtained by imaging is processed by an image processing device to determine the traffic situation. The image processing apparatus includes: a unit that identifies a corresponding pixel block of a captured image at a different time of imaging; a unit that calculates a motion vector of the pixel block based on a change in the position of the pixel block; Means for identifying a connected block that connects adjacent pixel blocks having substantially the same direction; vector calculating means for calculating a motion vector of the connected block based on a motion vector of each pixel block of the connected block; and Density calculating means for calculating the distribution density of the motion vector based on the number of pixel blocks of the block, and the distribution density or concatenation Based on the lock of the motion vector, characterized in that it comprises a determination means for determining traffic situation.

  A traffic situation determination system according to a thirteenth aspect of the present invention is provided with an imaging apparatus that captures an area including a road, an image processing apparatus that processes a captured image obtained by imaging, and a determination apparatus that determines a traffic situation. In the determination system, the image processing device includes: a unit that identifies a corresponding pixel block of a captured image at a different time of imaging; a unit that calculates a motion vector of the pixel block based on a change in the position of the pixel block; And a means for transmitting the motion vector transmitted to the determination device, wherein the determination device includes a means for receiving the motion vector transmitted by the image processing device and an adjacent pixel block whose directions of the received motion vectors substantially coincide. Based on the means for identifying the connected block connected, and the motion vector of each pixel block of the connected block, the motion of the connected block. Vector calculation means for calculating a vector, density calculation means for calculating a distribution density of a motion vector based on the number of pixel blocks of the connected block, and determining a traffic situation based on the distribution density or a motion vector of the connected block And determining means for performing the processing.

  According to a fourteenth aspect of the present invention, in the thirteenth aspect of the present invention, there are provided three image pickup devices for picking up different areas including roads, and three image processing devices for processing picked-up images picked up by the image pickup devices along the road. The determination device is configured to determine the traffic congestion status based on the distribution density or the motion vector of the connected block, and the first traffic congestion determined by the captured image captured by one imaging device. When the situation is different from the second traffic congestion situation determined by the captured images taken by the two most recent imaging apparatuses installed with the one imaging apparatus in between, it is the second traffic congestion situation It is characterized in that it is determined.

  According to a fifteenth aspect of the present invention, in the fourteenth aspect of the present invention, there is provided the traffic situation determination system according to the fourteenth aspect, in which the determination device determines whether the first traffic congestion situation determined from the captured image captured by the one imaging device If the second traffic congestion situation determined by the captured images captured by the two most recent imaging devices installed with the imaging device in between is different from the one time point, the one When the traffic jam situation determined by the captured image captured by the imaging device is the second traffic jam situation, it is determined that the traffic jam situation is the second traffic jam situation.

  The traffic situation determination system according to a sixteenth aspect of the invention is the fourteenth aspect or the fifteenth aspect of the invention, wherein the determination device has the same traffic jam situation determined by captured images captured by a plurality of continuous imaging devices. The traffic congestion change point is determined based on the position of the imaging device at the downstream end of the traffic flow in the imaging device.

  A computer program according to a seventeenth aspect of the present invention is a computer program for causing a computer to process an acquired captured image and determine a traffic situation, and to connect the computer to adjacent pixel blocks whose motion vector directions substantially coincide with each other. A means for identifying a block; a vector calculating means for calculating a motion vector of the connected block based on a motion vector of each pixel block of the connected block; and a motion vector distribution density based on the number of pixel blocks of the connected block And a density calculating means for calculating a traffic condition based on the distribution density or the motion vector of the connected block.

  A traffic situation determination method according to an eighteenth aspect of the present invention is the traffic situation determination method in which an area including a road is imaged by an imaging device, and the captured image obtained by imaging is processed to determine the traffic situation. Identifying a connected block that connects adjacent pixel blocks that substantially match, calculates a motion vector of the connected block based on a motion vector of each pixel block of the connected block, and based on the number of pixel blocks of the connected block The distribution density of the motion vector is calculated, and the traffic situation is determined based on the distribution density or the motion vector of the connected block.

  In the first invention, the twelfth invention, the seventeenth invention, and the eighteenth invention, the image processing apparatus divides the captured image into desired pixel blocks (for example, 8 × 8 pixel blocks), and one imaging time point. A pixel block corresponding to each pixel block of the captured image is identified by a captured image at an imaging time point different from the one imaging time point. When a corresponding pixel block is specified between different imaging time points, the image processing device is a pixel block that is a vector that connects the centers of the corresponding pixel blocks based on a positional change in the captured image of the specified pixel block. The motion vector of is calculated. As a result, it is possible to eliminate shadows on the road, road surface markings, and the like in which the position of the pixel block does not change between different imaging points. The image processing apparatus identifies pixel blocks in which the calculated motion vector directions of the respective pixel blocks substantially coincide with each other and connected blocks obtained by connecting adjacent pixel blocks. Thereby, the vehicle which drive | works a road is specified.

  The vector calculation means calculates the motion vector of the connected block based on the motion vector of each pixel block constituting the connected block (for example, the average value of the motion vector of each pixel block). Based on the number of pixel blocks of the connected block (for example, the ratio of the number of pixel blocks constituting the connected block (or all connected blocks when there are a plurality of connected blocks) based on the number of pixel blocks in the captured image) ) To calculate the motion vector distribution density. The determination means, based on the temporal distribution and the spatial distribution of the vehicle, calculates the motion vector distribution density or the motion vector of the connected block based on the obtained captured image when the imaging device repeatedly captures images. Determine traffic conditions.

  In the second invention, the density calculating means and the vector calculating means calculate the motion vector distribution density and the motion vector of the connected block a plurality of times over a predetermined time, and calculate the calculated motion vector distribution densities and concatenation. Based on the motion vector of the block, a statistical value including at least one of the average value, the mode value, and the median value is calculated. As a result, when the motion vector distribution density and the motion vector fluctuation of the connected block suddenly occur, the influence is eliminated and a highly reliable value is calculated.

  In the third invention, the determination means determines the traffic congestion status based on the motion vector distribution density calculated from the statistical value or the size of the motion vector of the connected block. For example, according to the combination of the magnitude of the motion vector distribution density, the magnitude of the motion vector of the connected block (or the magnitude of the speed converted from the magnitude of the motion vector in the captured image, and the magnitude of the converted speed), Determining the degree of traffic jams such as light, normal flow, traffic jams, and heavy traffic jams.

  In the fourth invention, the image processing apparatus stores in advance a standard direction that determines the direction of the traffic flow in which the vehicle travels based on the position and direction of the road or lane on the captured image. The angle calculation means calculates an angle difference between the direction of the motion vector of the connected block and the standard direction, and the determination means determines that the traffic is abnormal when the calculated angle difference is approximately 180 degrees. That is, the determination unit determines that the vehicle is abnormal in traffic in the reverse direction.

  In the fifth invention, the image processing apparatus stores in advance a standard direction that determines the direction of traffic flow in which the vehicle travels based on the position and direction of the road or lane on the captured image. The angle difference calculation means calculates an angle difference between the calculated motion vector direction and the standard direction. The determination means determines that the traffic is abnormal when the angle difference is equal to or greater than the first threshold until the measured determination time is equal to or greater than a predetermined time. As a result, it is determined that a traffic abnormality such as a traffic accident or avoiding traveling avoiding an obstacle on the road continues to occur.

  In the sixth invention, determination areas corresponding to a plurality of lanes in the same traveling direction are provided in the captured image in advance. The determination means is such that the magnitude of the motion vector of the connected block in one determination area (area corresponding to one lane in the captured image) is smaller than the second threshold until the measured determination time is equal to or greater than a predetermined time. When the magnitude of the motion vector of the connected block in another determination area (area corresponding to another lane in the captured image) is larger than the second threshold, it is determined that the traffic is abnormal.

  For example, when it is determined that the magnitude of the motion vector (or vehicle speed) in the lane corresponding to one determination area is smaller than the second threshold and the degree of traffic congestion is congested or very congested, When the magnitude of the motion vector (or vehicle speed) in the lane corresponding to the judgment area is larger than the second threshold and the degree of congestion is determined to be quiet or normal, traffic congestion in both lanes The difference in degree is large, and it is determined that there is a traffic abnormality due to a traffic fault such as a traffic accident or parking violation in the lane corresponding to one determination area. When the magnitude of the motion vector in the lane corresponding to one determination area and the magnitude of the motion vector in the lane corresponding to another determination area are both larger or smaller than the second threshold value, It is determined that there is no traffic abnormality because the difference in the degree of congestion is small in both lanes.

  In the seventh invention, the determination means determines that a traffic abnormality is detected when the motion vector distribution density of the one determination region is greater than a third threshold until the measured determination time reaches a predetermined time or more. judge. For example, when it is determined that the degree of congestion in the lane corresponding to one determination area is very congested, it is a traffic abnormality due to a traffic fault such as a traffic accident or parking violation in the lane corresponding to one determination area. Is determined. Thereby, the traffic abnormality due to traffic troubles such as traffic accidents and parking violations is determined with higher accuracy.

  In the eighth invention, determination areas corresponding to a plurality of lanes in the same traveling direction are provided in the captured image in advance. The determination means does not specify a connected block in one determination area (area corresponding to one lane in the captured image) until the determined determination time is equal to or greater than a predetermined time, and other determination areas (captured image) If the magnitude of the motion vector of the connected block in the area corresponding to the other lane in (1) is larger than the second threshold, it is determined that the traffic is abnormal. For example, there is no vehicle in the lane corresponding to one determination area, the magnitude of the motion vector (or vehicle speed) in the lane corresponding to the other determination area is greater than the second threshold, and the degree of congestion is When it is determined that the vehicle is quiet or normal, it is determined that there is a traffic abnormality due to a traffic obstacle that prevents the vehicle from traveling in the lane corresponding to one determination area. Thereby, the traffic abnormality due to traffic troubles such as traffic accidents and parking violations is determined with higher accuracy.

  In the ninth invention, the image processing apparatus includes a restriction area corresponding to traffic restrictions on the road (for example, prohibition of entry in front of a stop line, prohibition of right turn at an intersection) in the captured image. (For example, indicating a travel prohibition direction) is stored. When the image processing apparatus detects entry of the connected block into the restricted area, the image processing apparatus compares the motion vector of the connected block with the restricted direction of the restricted area. The determination unit determines that a traffic violation has occurred when the direction of the motion vector of the connected block that has entered the restriction area substantially matches the restriction direction.

  For example, if the direction of the motion vector of the connected block and the restriction direction substantially coincide with each other, the determination unit determines that the determination unit has traveled in the direction in which the vehicle is prohibited, Judged as a traffic violation such as a right turn with no right turn.

  In the tenth invention, when the overlapping area where the connected blocks overlap is specified, the image processing apparatus compares the number of pixel blocks in the overlapping area with the fourth threshold value. That is, when the number of pixel blocks in the overlapping area is equal to or greater than the fourth threshold, there is a high possibility of collision between vehicles corresponding to the connected blocks, and the number of pixel blocks in the overlapping area is equal to or less than the fourth threshold. The possibility of collision between vehicles corresponding to the connection block is low. The determination means includes a first imaging time point (that is, a collision estimation time point) at which the number of pixel blocks in the overlapping region is equal to or greater than the fourth threshold value, and a second imaging time point before the first imaging time point (that is, before the collision). If the difference in luminance value between the pixels in the overlapping area is greater than or equal to the fifth threshold value, the traffic accident is determined.

  That is, the determination unit is configured such that the difference in luminance value between the overlapping areas is equal to or greater than the fifth threshold value between the second imaging time point (time point before the collision) and the first imaging time point (collision estimation time point). Since the vehicle has moved to the overlapping area between the second imaging time and the first imaging time, it is determined that a traffic accident has occurred as a collision. On the other hand, when the difference between the brightness values of the overlapping area is equal to or less than a fifth threshold between the second imaging time point (time point before the collision) and the first imaging time point (collision estimation time point), Since the vehicle has not moved to the overlapping area between the second imaging time and the first imaging time, it is determined that a collision is avoided and there is no traffic accident.

  In the eleventh aspect of the invention, the determination means includes the first imaging time point (collision estimation time point) of the luminance value of each pixel in the region where the connected block exists at the second imaging time point (time point before the collision). And when the difference in the 2nd imaging time is more than a 6th threshold value, it determines with a traffic accident. That is, when the difference between the luminance values of the pixels in each of the regions where the connected block exists at the second imaging time point is equal to or more than the sixth threshold value at the first imaging time point and the second imaging time point, Since the vehicle corresponding to the connection block is moving without stopping between the second imaging time and the first imaging time, it is determined that a traffic accident has occurred as a collision. On the other hand, when the difference between the luminance values of the pixels in each of the regions where the connected block exists at the second imaging time point is less than or equal to the sixth threshold value at the first imaging time point and the second imaging time point, Since the vehicle corresponding to the connection block is stopped between the second imaging time and the first imaging time, it is determined that the collision is avoided and there is no traffic accident.

  In the thirteenth aspect, the image processing apparatus divides the captured image into desired pixel blocks (for example, 8 × 8 pixel blocks), and pixel blocks corresponding to the respective pixel blocks of the captured image at the time of one imaging. Is identified by a captured image at an imaging time point different from the one imaging time point. When the corresponding pixel block is specified between different imaging time points, the motion vector of the pixel block, which is a vector connecting the centers of the corresponding pixel blocks, is calculated based on the position change in the captured image of the specified pixel block. To do. As a result, it is possible to eliminate shadows on the road, road surface markings, and the like in which the position of the pixel block does not change between different imaging points. The image processing apparatus transmits the calculated motion vector to the determination apparatus.

  The determination device receives the motion vector, and identifies a pixel block in which the directions of the received motion vectors substantially coincide with each other and connects adjacent pixel blocks. Thereby, the vehicle which drive | works a road is specified. The vector calculation means calculates the motion vector of the connected block based on the motion vector of each pixel block constituting the connected block (for example, the average value of the motion vector of each pixel block). The density calculation means calculates a ratio of the number of pixel blocks constituting a connected block (all connected blocks when there are a plurality of connected blocks) to the number of pixel blocks in the captured image as a motion vector distribution density. The determination apparatus determines a traffic situation based on the calculated motion vector distribution density or a connected block motion vector.

  In the fourteenth aspect, the picked-up images picked up by three or more image pick-up devices picking up different areas including the road are processed by the image processing device, and the determination device is a motion vector transmitted by each of the image processing devices. Based on the above, the motion vector distribution density or the motion vector of the connected block is calculated, and the traffic jam situation in each imaging device is determined. Of the three or more imaging devices positioned along the road, the first traffic congestion situation determined by the captured image captured by one imaging device is the latest two installed with the one imaging device in between. When it is different from the second traffic congestion situation determined by the captured image captured by each of the two imaging devices, it is determined that it is the second traffic congestion situation.

  For example, the traffic congestion situation determined by the captured image captured by one imaging device is normal, and determined by the captured images captured by the two most recent imaging devices installed with the one imaging device in between. When the traffic congestion situation is congested, the determination device determines that there is a density wave with a degree of congestion, and determines that the traffic congestion status determined by the captured image captured by the one imaging device is also congested. Thus, the same traffic congestion situation is determined for a wide area of the road.

  In the fifteenth aspect, the determination device determines a first traffic congestion situation based on a captured image captured by the one imaging device at one time point, and is installed with the one imaging device in between. The second traffic congestion situation is determined based on the captured images captured by the two most recent imaging devices. When the first traffic congestion situation and the second traffic congestion situation are different, the determination device makes a determination based on a captured image captured by the one imaging device at a plurality of other time points different from the one time point. When the traffic jam situation is the second traffic jam situation, it is determined that the first traffic jam situation determined at the one time point is a false judgment and the second traffic jam situation is determined.

  For example, at one time point, the traffic jam situation determined by the captured image captured by the one imaging device is congested, and images are captured by the two most recent imaging devices installed with the one imaging device in between. When the traffic congestion situation determined by the captured image is not congested but quiet, the traffic congestion situation determined by the captured image captured by the one imaging device is quiet at a plurality of other time points different from the one time point. In some cases, the determination result determined to be crowded at the one time point is an error, and it is determined to be quiet.

  In the sixteenth aspect of the invention, the determination device is an image pickup device at the downstream end of the traffic flow among the image pickup devices when the traffic congestion situation determined by the picked-up images picked up by a plurality of continuous image pickup devices is the same. Based on the position of, the traffic jam change point is determined. For example, when the traffic jam situation determined by the captured images captured by a plurality of continuous imaging devices is congested, the imaging device at the downstream end of the traffic flow (that is, the region determined to be congested) is selected among the imaging devices. Among the plurality of continuous imaging devices to be imaged, the position of the imaging device that is the most downstream in the traffic flow is determined as a traffic jam change point (for example, a congestion bottleneck).

  In the first invention, the twelfth invention, the seventeenth invention, and the eighteenth invention, a motion vector of a connected block in which adjacent pixel blocks are connected by a change in position of the corresponding pixel block specified by a captured image at a different imaging time point. And the traffic situation is determined based on the calculated motion vector or the distribution density of the motion vector calculated based on the motion vector. Can determine the traffic situation.

  In the second invention, a statistical value including at least one of an average value, a mode value, or a median value is calculated based on the motion vector distribution density and the motion vector calculated a plurality of times in a predetermined time. Thus, the traffic situation can be determined more accurately.

  In the third invention, the degree of traffic congestion such as light, normal flow, traffic jam, and heavy traffic jam is determined by determining the traffic jam status based on the motion vector distribution density or the size of the motion vector of the connected block. Can be accurately determined.

  In the fourth invention, when the angle difference between the direction of the motion vector of the connected block and the standard direction that determines the direction of traffic flow is approximately 180 degrees, the vehicle is reversed in the opposite direction by determining that the traffic is abnormal. Abnormal running can be easily determined.

  In the fifth invention, when the angle difference between the direction of the motion vector of the connected block and the standard direction that determines the direction of traffic flow is equal to or greater than the first threshold until the measured determination time is equal to or greater than a predetermined time. By determining that the traffic is abnormal, it is possible to easily determine that avoidance traveling for avoiding a parked vehicle, an accident vehicle, or an obstacle is continued.

  In the sixth invention, determination areas corresponding to a plurality of lanes in the same traveling direction are provided in the captured image, and the connected blocks of one determination area are connected until the measured determination time becomes a predetermined time or more. When the magnitude of the motion vector is smaller than the second threshold and the magnitude of the motion vector of the connected block in the other determination area is larger than the second threshold, it is determined that the traffic is abnormal, and the degree of congestion in both lanes Accordingly, it can be determined that a traffic fault has occurred in one of the same lanes.

  In the seventh invention, until the timed determination time is equal to or greater than a predetermined time, when the motion vector distribution density of the one determination region is greater than a third threshold, it is determined as a traffic abnormality, Furthermore, it is possible to determine traffic abnormality with high accuracy.

  In the eighth invention, the connected block is not specified in one determination region until the measured determination time is equal to or longer than a predetermined time, and the size of the motion vector of the connected block in the other determination region is the first value. When it is larger than 2 threshold values, it is possible to determine the traffic abnormality with higher accuracy by determining the traffic abnormality.

  In the ninth invention, when a restriction area corresponding to traffic restriction is provided in the captured image, and the direction of the motion vector of the connected block that has entered the restriction area substantially coincides with the restriction direction of the traffic flow. By determining the traffic violation, it is possible to determine a traffic violation such as entering the prohibited entry area or traveling right when the right turn is prohibited.

  In the tenth invention, between the first imaging time when the number of pixel blocks in the overlapping region where the connected blocks overlap each other is equal to or greater than the fourth threshold, and the second imaging time before the first imaging time, The occurrence of a collision accident can be easily determined by determining a traffic accident when the difference in luminance value of the pixels in the overlapping region is equal to or greater than the fifth threshold value.

  In the eleventh aspect of the invention, the difference in luminance value between the first imaging time point and the second imaging time point of each pixel in the region where the connected block exists at the second imaging time point is equal to or greater than the sixth threshold value. In this case, it is possible to determine the occurrence of a collision accident with higher accuracy by determining a traffic accident.

  In the thirteenth invention, the motion vector calculated by the image processing device is transmitted to the determination device, and the determination device calculates the motion vector distribution density and the motion vector of the connected block based on the received motion vector. By determining the traffic situation based on the calculated motion vector distribution density or the motion vector of the connected block, it is possible to easily determine the traffic situation at a remote location and reduce the load on the image processing apparatus, thereby simplifying Therefore, the cost of the entire system can be reduced.

  In the fourteenth aspect of the present invention, the first traffic jam situation determined by the captured image captured by one imaging device among the three or more imaging devices positioned along the road is a gap between the one imaging device. If it is different from the second traffic congestion situation determined by the captured images taken by the two most recent imaging devices installed as described above, it is determined that the traffic congestion situation is the second traffic congestion situation. It is possible to determine a wide range of traffic congestion conditions at a plurality of remote locations.

  In the fifteenth aspect of the present invention, the first traffic jam situation determined by the captured image captured by the one imaging device at one time point and the latest two installed between the one imaging device. When the second traffic congestion situation determined by the captured image captured by each of the imaging devices is different, the traffic congestion situation by the captured image captured by the one imaging device at a plurality of other time points different from the one time point By determining this, it is possible to reduce erroneous determination.

  In the sixteenth aspect of the invention, the determination device is an image pickup device at the downstream end of the traffic flow among the image pickup devices when the traffic congestion situation determined by the picked-up images picked up by a plurality of continuous image pickup devices is the same. By determining the traffic jam change point based on the position of the traffic bottleneck, it is possible to detect a bottleneck point of traffic jam (for example, congestion).

Embodiment 1
Hereinafter, the present invention will be described with reference to the drawings illustrating embodiments. FIG. 1 is a schematic diagram showing an outline of a traffic situation determination system according to an embodiment of the present invention. In the figure, reference numeral 1 denotes a video camera that captures an area including a road. For example, the video camera 1 has an optical axis arranged along the road direction at a position of about 10 m on the road near an intersection, and picks up the traffic situation of an imaging range 6 of several tens of meters on a road 5 of four lanes on one side. .

  An image processing device 2 is connected to the video camera 1, and the image processing device 2 processes a captured image obtained by imaging with the video camera 1 to determine a traffic situation. The image processing apparatus 2 is connected to a communication line 4 such as the Internet, and a control center apparatus 3 is connected to the communication line 4. The traffic situation determined by the image processing apparatus 2 is transmitted to the control center apparatus 3 via the communication line 4, and the control center apparatus 3 receives the traffic situation transmitted from the image processing apparatus 2 and receives the received traffic situation. The control center operator performs necessary traffic control. The image processing apparatus 2 controls the signal display of a traffic light (not shown) according to the determined traffic situation, and displays necessary traffic information on a traffic display board (not shown), etc. Prevent accidents.

  FIG. 2 is a block diagram showing the configuration of the image processing device 2 and the control center device 3. In the figure, reference numeral 1 denotes a video camera. The video camera 1 outputs a captured image obtained by imaging to the image input unit 21 as a video signal (analog signal). The image input unit 21 outputs the acquired video signal to the A / D conversion unit 22, and the A / D conversion unit 22 converts the input video signal into a digital signal, which is converted under the control of the CPU 28. The obtained digital signal is stored in the image memory 23 as image data. The CPU 28 uses the captured image input from the video camera 1 via the image input unit 21 as image data in synchronization with the frame rate of the video camera 1 (interval at the time of imaging, for example, 30 frames per second). The data is stored in the image memory 23 in units of frames (for example, 640 × 480 pixels).

  The auxiliary storage unit 26 stores the computer program PG recorded on the CD-ROM 29 in the RAM 24 by inserting the CD-ROM 29 on which the computer program PG of the present invention is recorded. The CPU 28 executes the computer program PG stored in the RAM 24. The image processing apparatus 2 reads the computer program PG recorded in the CD-ROM 29 into the RAM 24, and executes the read computer program PG with the CPU 28, whereby frame image processing, traffic congestion determination processing, and traffic abnormality determination processing described later are performed. The process for judging traffic conditions such as is realized.

  The image memory 23 stores the captured image acquired via the image input unit 21 as image data for each frame, and stores the calculated result corresponding to each frame in accordance with the processing of the CPU 28. .

  The storage unit 27 stores the determination result of the traffic situation obtained by executing the computer program PG by the CPU 28. The storage unit 27 stores a traffic jam determination table 271 for determining the degree of traffic jam and a standard direction for determining the traveling direction of the vehicle. The standard direction determines the direction of traffic flow of vehicles traveling on the road, and the direction of the traffic flow on the captured image differs depending on the form of the road, the installation location of the video camera 1, etc. A standard direction that defines a required direction can be stored in accordance with the installation location of 1 or the like. The standard direction can be stored in a numerical table indicating the relationship of the y coordinate to the x coordinate in the captured image.

  The communication unit 25 transmits the determination result of the traffic situation determined by the image processing device 2 to the control center device 3 via the communication line 4. In addition, the communication unit 25 transmits a control signal based on the determination result to a traffic light, a traffic display board (both not shown), etc. under the control of the CPU 28, and so on. Can also be done.

  The communication unit 31 of the control center device 3 receives the determination result of the traffic situation transmitted from the image processing device 2 and displays the received determination result on the display unit 33 having a liquid crystal display under the control of the CPU 36. At the same time, it is stored in the storage unit 34.

  The CPU 36 executes a computer program stored in the RAM 32 to perform processing such as display of traffic condition determination results and aggregation.

  The operation unit 35 includes a keyboard, a mouse, and the like, and can perform an input operation on the control center device 3. By receiving the operation of the control center operator, the control center device 3 displays a traffic condition determination result. Perform processing, aggregation processing, etc.

  Next, a processing procedure of the traffic situation determination system according to the present invention will be described. FIG. 3 is a flowchart showing a processing procedure for determining traffic conditions. The CPU 28 starts counting the determination time T1 by setting a built-in timer (S1), and starts counting the calculation time T2 (T2 <T1) (S2).

  The CPU 28 performs frame image processing (S3), and determines whether the calculation time T2 has elapsed (S4). If the calculation time T2 has not elapsed (NO in S4), the CPU 28 continues to repeat the frame image processing in step S3.

  On the other hand, when the calculation time T2 has elapsed (YES in S4), the CPU 28 performs traffic congestion determination processing (S5) and traffic abnormality determination processing (S6). The CPU 28 determines whether or not there is a traffic abnormality (S7). If it is determined that there is a traffic abnormality (YES in S7), the CPU 28 determines whether or not the determination time T1 has elapsed (S8).

  When the determination time T1 has not elapsed (NO in S8), the CPU 28 resets the calculation time T2 by setting the calculation time T2 to zero (S9), and continues the processing from step S2 onward, thereby performing frame image processing, traffic Repeat the congestion determination process and the traffic abnormality determination process.

  When the determination time T1 has elapsed (YES in S8), the CPU 28 notifies the traffic control center device 3 of a traffic abnormality via the communication unit 25 (S10). The CPU 28 determines whether or not there is an end command from the control center device 3 (S11). If there is no end command (NO in S11), the determination time T1 and the calculation time T2 are calculated by setting the determination time T1 and the calculation time T2 to zero. The time T2 is reset (S12), and the processing after step S1 is continued.

  On the other hand, when it determines with there being no traffic abnormality (it is NO at S7), CPU28 continues the process after step S11. CPU28 complete | finishes a process, when there exists completion | finish instruction | command (it is YES at S11).

  Thereby, the image processing apparatus 2 repeatedly performs the frame image processing for the calculation time T2, and performs the traffic jam determination process and the traffic abnormality determination process when the calculation time T2 has elapsed. That is, based on the result repeatedly calculated during the calculation time T2, the traffic jam determination process and the traffic abnormality determination process are performed every time the calculation time T2 elapses. If it is determined that there is a traffic abnormality at the time when the calculation time T2 has elapsed, it is determined whether or not the traffic abnormality continues during the determination time T1 longer than the calculation time T2, and continues for the determination time T1. If there is a traffic abnormality, notification of the traffic abnormality is performed.

FIG. 4 is a flowchart showing a processing procedure of frame image processing. CPU28 acquires imaging point different two captured images (e.g., frames f _k of the imaging time t _k, the frame f _{k + 1} of the imaging time t _{k + 1)} from the image memory 23 (S301).

  The CPU 28 scans and matches the other captured image for each pixel block (for example, 8 × 8 pixels) of the acquired captured image based on the luminance value (or gradation value) of the pixel in the pixel block. The pixel block to be specified is specified (S302). When specifying a matching pixel block between two captured images, a parameter indicating the degree of coincidence such as the average luminance value, standard deviation value, and absolute value of the difference luminance value of the pixels in the pixel block can be used.

The CPU 28 calculates the motion vector of each pixel block based on the change in coordinates (position) in the captured image of the pixel block that matches in the two captured images (S303). In this case, the coordinates of the pixel block are the coordinates of the center of the pixel block. Note that the coordinates of the upper left corner of the pixel block may be used. The coordinates of the pixel block P _{j in} the captured image at the imaging time t _k are (x _k , y _k ), and the coordinates of the pixel block P _{j in} the captured image at the imaging time t _{k + 1} are (x _{k + 1} , y _{k + 1).} ), The motion vector PV _j is (x _{k + 1} -x _k , y _{k + 1} -y _k ). When there is no change in coordinates, the motion vector is zero, and such a pixel block is excluded in the processing after step S303. If the calculated motion vector is smaller than a predetermined threshold value, the motion vector of the pixel block may be excluded as being zero.

  The CPU 28 identifies pixel blocks in which the calculated motion vector directions substantially coincide with each other and connects adjacent pixel blocks (S304). Thereby, a connection block respond | corresponds to a traveling vehicle. In addition, when specifying a connection block, when the number of the pixel blocks which comprise a connection block is smaller than a predetermined threshold value (for example, 3), it can also remove as noise.

  The CPU 28 determines whether or not there is a specified connected block (S305). If there is a connected block (YES in S305), the CPU 28 uses the average value of the motion vectors of the pixel blocks constituting the connected block as the motion of the connected block. It calculates as a vector (S306). The CPU 28 reads the standard direction from the storage unit 27 and calculates the angle difference between the direction of the motion vector of the connected block and the standard direction (S307).

  The CPU 28 determines whether or not the calculated angle difference is approximately equal to 180 ° (S308). If the angle difference is approximately equal to 180 ° (YES in S308), the control center device 3 determines that there is reverse travel. Notification is made (S309). When the angle difference is not approximately equal to 180 ° (NO in S308), the CPU 28 moves the ratio of the number of pixel blocks constituting the connected block specified by the captured image to the number of pixel blocks included in the captured image for one frame. The vector distribution density is calculated (S310).

  The CPU 28 converts the magnitude of the motion vector of the connected block into the speed of the vehicle corresponding to the connected block using a predetermined conversion formula, calculates the speed of the vehicle (S311), and calculates the distribution density of the calculated motion vector. The speed and the number of connected blocks are recorded (S312), and the frame image processing is terminated. On the other hand, when there is no connected block (NO in S305), the CPU 28 records that there is no connected block (S313), and ends the frame image processing. In addition, since the magnitude | size of the motion vector of a connection block and the speed of the vehicle corresponding to a connection block are determined uniquely, it can be made equivalent.

FIG. 5 is an explanatory diagram showing a motion vector of a pixel block. As shown in the figure, the coordinates of the pixel block P _{j in} the captured image at the imaging time t _k are (x _k , y _k ), and the coordinates of the pixel block P _{j in} the captured image at the imaging time t _{k + 1} are (x _{k + 1} , y _{k + 1} ), the motion vector PV _j is (x _{k + 1} −x _k , y _{k + 1} −y _k ).

  FIG. 6 is an explanatory diagram showing an example of a connected block. As shown in the figure, the captured image is divided into predetermined pixel blocks, the motion vector of each pixel block is calculated, and the calculated motion vector directions are substantially the same, and only adjacent pixel blocks are connected. By doing so, the connected block is specified. Even if the pixel blocks have substantially the same motion vector direction, the separated pixel blocks are removed as noise. Further, even when the number of connected pixel blocks is equal to or less than a predetermined threshold (for example, 3), the traveling vehicle can be accurately identified by the connected blocks by removing the noise blocks.

  FIG. 7 is an explanatory diagram illustrating an example of a captured image. As described above, the video camera 1 is arranged so as to capture an area including a road on four lanes on one side. In the captured image, determination areas 30a, 30b, 30c, and 30d are set for determining traffic conditions for each lane. That is, each of the determination areas 30a, 30b, 30c, and 30d indicates each lane of four lanes on one side. A vehicle traveling on the road is specified by a motion vector. The frame image processing described above is processed for each determination area.

FIG. 8 is an explanatory diagram showing calculation information calculated by frame image processing. As shown in FIG. 8A, the captured image of one frame is divided into pixel blocks P ₁ , P ₂ ,... P _n composed of 8 × 8 pixels, for example. As shown in FIG. 8B, motion vectors PV ₁ , PV ₂ ,... PV _n calculated for each pixel block are recorded. The motion vector of the pixel block in which no motion is detected is zero. As shown in FIG. 8C, the number of connected blocks for each captured image of one frame, the motion vector CB _{1 of} the connected block for each specified connected block, the vehicle speed Cv ₁ corresponding to the connected block, ... Calculation information such as motion vector distribution density D is calculated and recorded.

  The conversion from the motion vector of the connected block to the speed of the vehicle corresponding to the connected block can be performed based on, for example, an expression represented by Equation 1.

  Here, the coordinate system in the real space is (X, Y, Z), and the coordinate system on the captured image is (x, y). In real space, Z is the vertical direction, Y is the direction in which the road extends, and X is the direction that crosses the road. The x-axis direction on the captured image plane is parallel to the X direction in the real space, the y-axis direction on the captured image plane is the Y direction in the real space, and the video camera depression angle (the optical axis and the Y axis of the video camera) The angle formed by the direction is set to α. Further, it is assumed that the focal length of the video camera lens is F, the installation height of the video camera is H, the size (dimension) in the y-axis direction of the captured image plane is Qy, and the number of pixels in the y-axis direction of the captured image plane is Ny. From experience values such as detection points from the road surface of the traveling vehicle, for example, Y is calculated by substituting the average height of the vehicle 50 cm into Equation 1, and the speed of the vehicle is calculated from the moving distance and moving time of the Y coordinate. be able to. Note that the speed may be calculated from the magnitude of the motion vector by using a conversion table, or may be obtained by arithmetic processing based on a conversion formula.

  FIG. 9 is a flowchart showing a processing procedure for determining traffic congestion. The CPU 28 calculates an average value of speeds from the speed and motion vector distribution densities calculated a plurality of times during the calculation time T2 (S501), and calculates an average value of motion vector distribution densities (S502). The CPU 28 reads the traffic jam determination table 271 from the storage unit 27 and determines whether or not the speed is 10 km or less (S503).

  When the speed is 10 km or less (YES in S503), the CPU 28 determines whether or not the motion vector distribution density is 75% or less (S504). If the distribution density is 75% or less (YES in S504), the CPU 28 determines that it is congested (S505), notifies the control center device 3 of the determination result (S510), and ends the process. If the distribution density is 75% or more (NO in S504), the CPU 28 determines that the congestion is heavy (S506), notifies the control center device 3 of the determination result (S510), and ends the process.

  On the other hand, when the speed is 10 km or more (NO in S503), the CPU 28 determines whether or not the motion vector distribution density is 50% or less (S507). If the distribution density is 50% or less (YES in S507), the CPU 28 determines that it is quiet (S508), notifies the control result to the control center device 3 (S510), and performs processing. If the distribution density is 50% or more (NO in S507), the CPU 28 determines that the flow is normal (S509), notifies the control center device 3 of the determination result (S510), and ends the process. The traffic jam determination process is performed for each of the above-described determination areas (for example, determination areas 30a and 30b).

  Thereby, the degree of traffic congestion can be determined based on the average value of the motion vector calculated multiple times and the distribution density of the motion vector over the calculation time T2.

  FIG. 10 is a record layout of the traffic jam determination table. As shown in the figure, according to the combination of speed and motion vector distribution density, the degree of traffic congestion can be determined by classifying it into light, normal flow, congestion, and heavy congestion. Note that the magnitude of the motion vector may be combined instead of the speed.

  FIG. 11 is a flowchart showing a traffic abnormality determination processing procedure. In the figure, one region (region for determining the presence or absence of traffic abnormality) and another region (other regions other than the one region, for example, if there are four cars on one side, any of the other three lanes Is the same direction of travel of the vehicle. The CPU 28 determines whether or not the degree of congestion in the other area is normal flow or quiet (second threshold) (S601). When it is normal flow or quiet (YES in S601), the CPU 28 determines whether or not the congestion degree of one area is large congestion (third threshold) (S602).

  If one area is heavily congested (YES in S602), the CPU 28 records that there is a traffic abnormality because the degree of congestion is greatly different in the same direction lane (S603), and ends the process. When one area is not very crowded (NO in S602), the CPU 28 determines whether or not there is a connected block in the one area (S604).

  If there is no connected block in one area (YES in S604), the CPU 28 records that there is a traffic abnormality because the degree of congestion is greatly different in the same direction lane (S603), and ends the process. When there is a connected block in one area (NO in S604), the CPU 28 calculates an angle difference between the direction of the motion vector and the direction of the standard vector read from the storage unit 27 (S605).

  The CPU 28 determines whether or not the calculated angle difference is larger than the threshold angle (first threshold) (S606). If the angle difference is larger than the threshold angle (YES in S606), it is assumed that the traveling direction of the vehicle is abnormal. Then, it is recorded that there is a traffic abnormality (S603), and the process is terminated. When the angle difference is smaller than the threshold angle (NO in S606), the CPU 28 records that there is no traffic abnormality (S607) and ends the process.

  As a result, the difference in the degree of traffic congestion between the same traveling lanes is determined based on the average values of the motion vectors calculated multiple times and the distribution density of the motion vectors over the calculation time T2. When an abnormality (for example, a traffic accident due to a traffic accident, a parking violation vehicle, an obstacle, or the like) is recorded and the state of the traffic abnormality continues for the determination time T1, it can be determined as a traffic abnormality. Also, based on the average value of the motion vector, the angle difference from the standard direction indicating the normal traveling direction of the vehicle is determined and the traffic abnormality is recorded, and the traffic abnormality state continues for the determination time T1. Can be determined as a traffic abnormality.

  FIG. 12 is a time chart showing an example of traffic abnormality determination. As shown in the figure, the image processing apparatus 2 determines the degree of traffic congestion in the determination areas 30a and 30b corresponding to adjacent lanes every time the calculation time T2 elapses. The determination area 30a is determined to be quiet, and the determination area 30b is determined to be a normal flow. In the next determination cycle, when it is determined that the determination area 30a is quiet and the determination area 30b is determined to be very crowded, a traffic abnormality is recorded because the difference in the degree of congestion in both determination areas is large. Timing of the determination time T1 is started. Thereafter, the image processing apparatus 2 determines that the traffic is abnormal when the determination time T1 elapses when the determination area 30a determines that the area is quiet and the determination area 30b continues to determine that the area is very crowded.

  In the above example, the traffic abnormality is determined for the two lanes corresponding to the determination area 30a and the determination area 30b. However, the determination of the traffic abnormality is not limited to the comparison between the two lanes. For example, when determining a traffic abnormality in one lane (for example, the determination region 30b) of four lanes on one side, a comparison between the determination region 30b and the determination region 30a and the determination region 30c, a determination region 30b and the determination region 30a, and a determination region 30c and the determination area 30d may be compared, or the determination area 30b may be compared with the determination area 30c and the determination area 30d. Thereby, it is possible to determine a traffic abnormality not only in two lanes but also in three or more lanes.

Embodiment 2
In the first embodiment, an image of a range of several tens of meters of the inflow path or the outflow path of the intersection is captured, but the traffic situation in the intersection can also be determined. FIG. 13 is a schematic diagram showing an outline of the traffic condition determination system according to the second embodiment. As shown in the figure, the installation location of the video camera 1 is different from that of the first embodiment. For example, the video camera 1 has an optical axis arranged along a road direction at a position of about 10 m on the road at an intersection angle, and the traffic situation of the imaging range 6 including each road near the intersection including the inside of the intersection 7. Image. Note that the image processing device 2 and the control center device 3 are the same as those in the first embodiment, and thus description thereof is omitted.

  Next, a processing procedure of the traffic situation determination system according to Embodiment 2 will be described. FIG. 14 is a flowchart showing a traffic condition determination processing procedure. The CPU 28 performs frame image processing for calculating a motion vector (S20), performs traffic violation determination processing for determining whether there is a traffic violation (S21), and traffic accident for determining whether there is a traffic accident. A determination process is performed (S22). For example, the CPU 28 determines whether or not there is an end command from the control center device 3 (S23). If there is no end command (NO in S23), the CPU 28 continues the processing after step S20 and there is an end command ( The process is terminated.

FIG. 15 is a flowchart showing a processing procedure of frame image processing. The description of the same part as the frame image processing of the first embodiment is simplified. CPU28 has two captured images having different imaging times (e.g., acquires frame f _k of the imaging time t _k, the frame f _{k + 1} of the imaging time t _{k + 1} from the image memory 23 (S201).

  The CPU 28 identifies a matching pixel block between the two acquired captured images (S202), and calculates a motion vector of each pixel block based on a change in coordinates (position) in the captured image of the matching pixel block (S203). ). The CPU 28 identifies pixel blocks in which the calculated motion vector directions substantially coincide with each other and connects adjacent pixel blocks (S204).

  The CPU 28 determines whether or not there is a connected block (S205). If there is a connected block (YES in S205), the CPU 28 calculates the average value of the motion vectors of the pixel blocks constituting the connected block as the motion vector of the connected block. (S206). If there is no connected block (NO in S205), the CPU 28 continues the processing from step S201. The CPU 28 records the coordinates of the connected block (for example, coordinates in the captured image near the four corners of the connected block) (S207), and ends the process.

  FIG. 16 is a flowchart showing a traffic violation determination processing procedure. The CPU 28 determines whether or not the connected block is within the restricted area based on the recorded coordinates of the connected block in the captured image in which the restricted area corresponding to the traffic restriction on the road is set in advance (S211). .

  When the connection block is within the restriction area (YES in S211), the CPU 28 restricts the control information related to the restriction area (for example, the traffic flow restriction direction such as the travel prohibition direction in the restriction area, or the restriction direction). (Including a restriction vector determined based on the magnitude corresponding to the speed limit) is read from the storage unit 27 (S212), and the direction of the motion vector of the connected block is compared with the restriction direction (S213).

  The CPU 28 determines whether or not the direction of the motion vector of the connected block substantially coincides with the restriction direction (S214). If the direction substantially coincides (YES in S214), the CPU 28 determines that it is a traffic violation (S215). The control center device 3 is notified. The CPU 28 determines whether or not there is another restriction area (S216), and if there is another restriction area (YES in S216), the processing after step S211 is continued. On the other hand, if they do not substantially match (NO in S214), the CPU 28 continues the processing after step S216.

  FIG. 17 is an explanatory diagram illustrating an example of a restriction region of a captured image. Restriction areas 50a, 50b, 50c, and 50d are set at the corners of the intersection and in front of the stop lines before the intersection, and a restriction area 50e is set at the center of the intersection. The regulation areas 50a, 50b, 50c, 50d, and 50e regulate the travel prohibition direction according to the signal display of the intersection, and the regulation area 50e regulates the travel prohibition direction such as a right turn prohibition according to the time. In addition, the regulation areas 50a, 50b, 50c, and 50d regulate the speed limit. For the restriction information such as the travel prohibition direction and the speed limit, a restriction information table 272 associated with the restriction area is stored in the storage unit 27.

  FIG. 18 shows a record layout of the restriction information table. For each restriction area, the signal display for the road in the y-axis direction (up and down direction) on the captured image, the direction of the restriction vector indicating the travel prohibition direction (regulation direction), the restriction time when restricting the restriction area by time zone, It is composed of fields such as a direction of a restriction vector indicating a travel prohibition direction in the restriction time zone, a size of a restriction vector indicating a speed limit, and the like.

  When comparing the direction of the motion vector and the restriction direction, it is also possible to detect the signal display at the intersection and compare it with the restriction direction according to the signal display. Further, when comparing the direction of the motion vector and the regulation direction, it is also possible to detect the current time, determine whether the current time is in the regulation time zone, and compare it with a predetermined regulation direction. . Furthermore, it is possible to determine whether the motion vector is larger than the size of the restriction vector by determining whether the magnitude of the motion vector is larger than the size of the restriction vector that defines the speed limit.

  FIG. 19 is a flowchart showing a traffic accident determination processing procedure. The CPU 28 determines whether or not there are a plurality of connected blocks in the captured image (S221). When there are a plurality of connected blocks (for example, connected blocks A and B) (YES in S221), the CPU 28 records the imaging time t (S222). The areas where the connected blocks A and B at the imaging time t exist are defined as At and Bt. Based on the motion vector, the CPU 28 predicts the movement destination areas of the connected blocks A and B at the imaging time point t + 1 (S223). The areas where the connected blocks A and B exist at the imaging time t + 1 are defined as At + 1 and Bt + 1. When there are not a plurality of connected blocks (NO in S221), the CPU 28 ends the process.

  The CPU 28 determines whether or not there is an overlapping area Ct + 1 of the areas At + 1 and Bt + 1 (S224). If there is an overlapping area Ct + 1 (YES in S224), the number of pixel blocks constituting the overlapping area Ct + 1 is a predetermined threshold value (first number). It is determined whether it is larger than (4 threshold) (S225). If there is no overlapping area Ct + 1 (NO in S224), the CPU 28 ends the process.

  When the number of pixel blocks is larger than the threshold (YES in S225), the CPU 28 calculates the luminance difference of the overlapping area Ct + 1 at the imaging time points t and t + 1 (S226). In this case, it is determined that the vehicles corresponding to the regions At + 1 and Bt + 1 are highly likely to collide at the imaging time t + 1. If the number of pixel blocks is smaller than the threshold (NO in S225), the CPU 28 ends the process.

  The CPU 28 determines whether or not the calculated luminance difference is larger than a threshold value (fifth threshold value) (S227). When the luminance difference of the overlapping region Ct + 1 is larger than the threshold (YES in S227), the CPU 28 calculates the luminance difference of the region At at the imaging time t and t + 1 and the luminance difference of the region Bt at the imaging time t and t + 1 ( S228). When the luminance difference of the overlapping area Ct + 1 is smaller than the threshold value (NO in S227), the CPU 28 ends the process. In this case, in this case, it is determined that one or both of the vehicles corresponding to the connection blocks A and B have stopped. The CPU 28 determines whether or not the luminance difference of the region At at the imaging times t and t + 1 and the luminance difference of the region Bt at the imaging times t and t + 1 are larger than a threshold (sixth threshold) (S229).

  If the luminance difference between each of the areas At and Bt is larger than the threshold (YES in S229), the CPU 28 determines that there is an accident (S230), notifies the control center device 3 of the determination result, and ends the process. If the luminance difference of either region At or Bt is smaller than the threshold value (NO in S229), the CPU 28 ends the process. If the luminance difference between each of the areas At and Bt is larger than the threshold value, it is determined that each vehicle corresponding to the connected blocks A and B has entered the collision possibility area on the road corresponding to the overlapping area, and a traffic accident occurs. The presence / absence determination can be more accurately performed. Note that it is also possible to determine the presence or absence of an accident based only on the determination result in step S227 without performing the determination in step S229. In this case, the labor required for processing can be reduced.

  FIG. 20 is an explanatory diagram illustrating an example of an overlapping region of captured images. As shown in the figure, when the connection blocks A and B corresponding to the vehicles traveling toward the intersection on the roads intersecting each other on the captured image are specified, the area where the connection block A exists at the times t and t + 1 is Similarly, let At and At + 1 be the same, and the regions where the connected block B was present at times t and t + 1 are Bt and Bt + 1, respectively. At time t + 1, the overlapping region Ct + 1 where the connected blocks At and Bt are expected to overlap is a portion where the regions At + 1 and Bt + 1 overlap.

Embodiment 3
In the above-described embodiment, the configuration is such that the captured image captured by the video camera installed near the intersection is processed by the image processing device to determine the traffic situation, but the traffic situation is determined by the control center device 3. You can also.

  FIG. 21 is a schematic diagram showing an outline of the traffic condition determination system according to the third embodiment. As shown in the figure, the video cameras 1, 1,..., The image processing devices 2, 2,... Are installed in the vicinity of the intersection 7, respectively, and take an image of the intersection 7 or the road 5 near the intersection. The devices 2, 2,... Are connected to the control center device 3 via the communication line 4.

  The image input unit 21 of the image processing apparatus 2 acquires image data obtained by imaging with the video camera 1 as an NTSC signal, and the A / D conversion unit 22 converts the acquired analog signal into a digital signal, and the CPU 28 Stores the converted digital signal in the image memory 23 as image data. The CPU 28 encodes the image data stored in the image memory 23 based on a predetermined method (for example, MPEG, MPEG2, MPEG4, H.264, etc.), and the encoded data is transmitted from the communication unit 25 to the control center. Transmit to device 3. The CPU 28 calculates a motion vector based on the image data when performing the encoding process, and encodes the calculated motion vector.

  The control center device 3 receives the encoded data transmitted from the image processing devices 2, 2,... By the communication unit 31 and stores the received data in the storage unit 34. The CPU 36 reads the encoded data from the storage unit 34, restores the original image data from the read data, and extracts a motion vector and the like. The control center device 3 reads a predetermined computer program into the RAM 32, and executes the read computer program with the CPU 36, so that the control center device 3 can determine the traffic situation performed by the image processing device 2 in the first and second embodiments. Realize the process.

  The traffic condition determination process realized by the CPU 36 is the same as the process procedure shown in FIGS. 3, 9, 11, 14, 16, and 19, but the frame image processes S3 and S20 are different. Hereinafter, frame image processing in the case of performing traffic congestion determination and traffic abnormality determination in the present embodiment will be described.

  FIG. 22 is a flowchart showing a processing procedure of frame image processing. The CPU 36 receives the encoded image data transmitted from the image processing device 2 (S331). The CPU 36 extracts a motion vector from the received encoded image data (S332).

  The CPU 36 identifies a connected block that is a pixel block in which the directions of the extracted motion vectors substantially match and connects adjacent pixel blocks (S333). The CPU 36 determines whether or not there is a specified connected block (S334). If there is a connected block (YES in S334), the CPU 36 uses the average value of the motion vectors of each pixel block constituting the connected block as the motion of the connected block. It calculates as a vector (S335). The CPU 36 reads the standard direction from the storage unit 34, and calculates the angle difference between the direction of the motion vector of the connected block and the standard direction (S336).

  The CPU 36 determines whether or not the calculated angle difference is substantially equal to 180 ° (S337). If the angle difference is substantially equal to 180 ° (YES in S337), the CPU 36 notifies that reverse running is present (S338). . When the angle difference is not approximately equal to 180 ° (NO in S337), the CPU 36 moves the ratio of the number of pixel blocks constituting the connected block specified by the captured image to the number of pixel blocks included in the captured image for one frame. The vector distribution density is calculated (S339).

  The CPU 36 converts the magnitude of the motion vector of the connected block into the speed of the vehicle corresponding to the connected block using a predetermined conversion formula, calculates the speed of the vehicle (S340), and calculates the distribution density of the calculated motion vector. The speed and the number of connected blocks are recorded (S341), and the frame image processing is terminated. On the other hand, if there is no connected block (NO in S334), the CPU 36 records that there is no connected block (S342), and ends the frame image processing. When a plurality of image processing apparatuses 2 are installed, encoded image data from each image processing apparatus 2 is received, and the same processing is repeated.

  Next, frame image processing when performing traffic violation determination and traffic accident determination will be described. FIG. 23 is a flowchart showing a processing procedure of frame image processing. The CPU 36 receives the encoded image data transmitted from the image processing device 2 (S231). The CPU 36 extracts a motion vector from the received encoded image data (S232).

  The CPU 36 identifies pixel blocks in which the directions of the extracted motion vectors substantially match and connect adjacent pixel blocks (S233). The CPU 36 determines whether or not there is a connected block (S234). If there is a connected block (YES in S234), the CPU 36 calculates an average value of motion vectors of each pixel block constituting the connected block as a motion vector of the connected block. (S235). If there is no connected block (NO in S234), the CPU 36 continues the processing from step S231. The CPU 36 records the coordinates of the connected block (for example, coordinates in the captured image near the four corners of the connected block) (S236), and ends the process. When a plurality of image processing apparatuses 2 are installed, encoded image data from each image processing apparatus 2 is received, and the same processing is repeated.

  As a result, the control center device 3 can determine traffic conditions at multiple points in a wide range and simultaneously. In addition, the control center device 3 can transmit control information based on the determination result to traffic lights, traffic information display boards and the like arranged at multiple points, and can control traffic lights, display traffic information, and the like.

Embodiment 4
By arranging a plurality of video cameras and image processing devices along the road, it is possible to determine the traffic jam situation in more detail over a wide range. FIG. 24 is a schematic diagram showing an outline of the traffic condition determination system according to the fourth embodiment. As shown in the figure, the video cameras 1, 1,..., The image processing devices 2, 2,... Are arranged along the road with an appropriate separation distance (for example, several hundred meters, several kilometers, etc.). is doing. For convenience, in the figure, five video cameras and image processing apparatuses are shown, and in order to simplify the following description, the symbols of the video camera 1 are denoted by C1, C2, C3, C4 from upstream to downstream of the traffic flow. , And C5.

  The control center device 3 determines the traffic jam situation of the areas imaged by the video cameras C1,..., C5 based on the captured images captured by the video cameras C1,. First, a description will be given of a process for detecting a traffic jam in a traffic flow and determining a traffic jam condition in a wide range with high accuracy.

  FIG. 25 is a flowchart showing a traffic jam determination processing procedure. The CPU 36 sets the index n for indicating the code of the video camera 1 to 1 (S521). The CPU 36 determines traffic congestion based on the captured image captured by the video camera Cn (S522), and based on the captured image captured by the video camera Cn + 1 that is installed next to the video camera Cn at the same time point. A traffic jam is determined (S523).

  The CPU 36 determines whether or not the degree of congestion determined by the video camera Cn is the same as the degree of congestion determined by the video camera Cn + 1 (S524), and if it is determined that the degree of congestion is not the same (NO in S524), The CPU 36 determines traffic congestion at the same time point based on the captured image captured by the video camera Cn + 2 that is installed adjacent to the video camera Cn + 1 (S525).

  The CPU 36 determines whether or not the degree of congestion determined by the video camera Cn + 2 and the degree of congestion determined by the video camera Cn are the same (S526), and if it is determined that the degree of congestion is the same (YES in S526). Then, the degree of congestion determined by the video camera Cn + 1 is corrected to the degree of congestion determined by the video camera Cn (S527).

  The CPU 36 determines whether or not the traffic jam determination has been completed based on the captured images captured by all the video cameras (S528), and the traffic jam determination has been completed based on the captured images captured by all the video cameras. If not (NO in S528), n is replaced with n + 1 (S529), and the processing after step S523 is continued.

  When the traffic jam determination is completed based on the captured images captured by all the video cameras (YES in S528), the CPU 36 records the determined traffic jam (S530). If it is determined in step S526 that the degree of congestion is not the same (NO in S526), the CPU 36 continues the processing from step S528. On the other hand, if it is determined in step S524 that the degree of congestion determined by the video camera Cn is the same as the degree of congestion determined by the video camera Cn + 1 (YES in S524), the CPU 36 continues the processing from step S528.

  In step S531, the CPU 36 determines whether or not there is an instruction to end the process (S531). If there is no instruction to end the process (NO in S531), the process from step S521 is continued. If there is an instruction to end the process (YES in S531), the CPU 36 ends the process.

  For example, at a certain time t, the congestion status determined by the video cameras C1, C2,..., C5 is congestion (C1), congestion (C2), normal flow (C3), congestion (C4), and congestion (C5). In this case, it is determined that the traffic wave is a rough and dense wave, and the normal flow determined by the video camera C3 is corrected to be congested. Thereby, it is possible to determine the traffic jam condition with higher accuracy compared to the case where the congestion wave of the traffic jam degree is detected over a wide range of the road and the traffic jam situation is determined by the video camera C3 alone.

  The above-described processing procedure is an example, and the number of video cameras 1 used for determination can be appropriately set according to the road condition. When the number of video cameras 1 used for determination is increased, the degree of congestion determined by a plurality of video cameras continuously installed on both sides is used without being limited to the two video cameras installed on both sides. You can also. Thereby, it is possible to accurately determine the degree of traffic congestion over a wider range.

  Next, the traffic jam situation determined by the video cameras C1, C2,..., C5 at a certain time is compared with the traffic jam situation similarly judged at a plurality of other times, thereby making an erroneous determination of the traffic jam. The process to reduce is demonstrated.

  FIG. 26 is a flowchart showing a traffic jam determination processing procedure. It is assumed that the traffic jam determined by the video cameras C1, C2,..., C5 at a certain time t is St. For example, the traffic jam St can be expressed as St = (low, low, normal, low, low) corresponding to the video cameras C1, C2,..., C5.

  The CPU 36 determines the traffic jam St + 1 at the time t (S541), and determines the traffic jam St + 1 at the time t + 1 having an appropriate distance from the time t (S542). The CPU 36 determines whether or not the traffic jam St and St + 1 are the same (S543). When the traffic jam St and St + 1 are not the same (NO in S543), the CPU 36 determines the traffic jam St-1 at the time t-1 having a proper separation distance from the time t (S544).

  The CPU 36 determines whether or not the traffic jams St + 1 and St-1 are the same (S545). When the traffic jam St + 1 and St-1 are the same (YES in S545), the CPU 36 determines the traffic jam St + 2 at time t + 2 (S546). The CPU 36 determines whether or not the traffic jam St + 1 and St + 2 are the same (S547).

  When the traffic jam St + 1 and St + 2 are the same (YES in S547), the CPU 36 determines the traffic jam St-2 at time t-2 (S548). The CPU 36 determines whether or not the traffic jams St-1 and St-2 are the same (S549). If the traffic jam St-1 and St-2 are the same (YES in S549), the CPU 36 corrects the traffic jam St at the time t to be the traffic jam St + 1 at the time t + 1, assuming that the traffic jam St at the time t is an erroneous determination (S550).

  The CPU 36 determines whether or not there is an instruction to end the process (S551). If there is no instruction to end the process (NO in S551), the time t is replaced with the time t + 1 (S552), and the processes after step S542 are continued. When there is an instruction to end the process (YES in S551), the CPU 36 ends the process.

  On the other hand, if the traffic jam St and St + 1 are the same in step S543 (YES in S543), the CPU 36 continues the processing from step S551. If the traffic jam St + 1 and St-1 are not the same in step S545 (NO in S545), the CPU 36 continues the processing from step S551.

  On the other hand, when the traffic jam St + 1 and St + 2 are not the same (NO in S547), the CPU 36 continues the processing from step S551. If the traffic jams St-1 and St-2 are not the same (NO in S549), the CPU 36 continues the processing from step S551.

  For example, the traffic jam St at time t is St = (light, light, crowded, light, quiet), and traffic jams at time t + 1, t + 2, t−1, t−2, St + 1, St + 2, St−1, St−2. When -2 is all (low, low, low, low, low), the traffic jam at time t (congestion determined by the video camera C3) is erroneously determined and St is (low, low, low) , Quiet, quiet). Thereby, it is possible to reduce erroneous determination and accurately determine traffic congestion.

  The above processing procedure is an example, and it is possible to increase or decrease the number of times for determining traffic congestion according to road conditions. Further, the time interval can be set as appropriate.

  Next, a description will be given of a process of determining a traffic jam occurrence position (for example, a congestion start position) as to which point on the road is a traffic jam bolt neck when a traffic jam occurs.

  FIG. 27 is a flowchart showing a traffic jam determination processing procedure. The CPU 36 sets the count number K to zero (S561), and sets the index n for indicating the sign of the video camera 1 to 1 (S562). The CPU 36 determines traffic congestion based on the captured image captured by the video camera Cn (S563), and determines whether the degree of congestion is congested (S564).

  When it is determined that the traffic is congested (YES in S564), the CPU 36 adds 1 to the counter number K (S565), and determines whether the count number K is equal to or greater than a threshold (S566). When the count number K is equal to or greater than the threshold (YES in S566), the CPU 36 sets the position of the video camera Cn as the congestion start position (S567).

  The CPU 36 determines whether or not the traffic jam determination has ended based on the captured images captured by all the video cameras (S568). If the traffic jam determination has not ended (NO in S568), the CPU 36 determines that n Is replaced with n + 1 (S569), and the processing after step S563 is continued. When the traffic jam determination is completed (YES in S568), it is determined whether or not there is an instruction to end the process (S570).

  If there is no instruction to end the process (NO in S570), the CPU 36 continues the processes after step S561. When there is an instruction to end the process (YES in S570), the CPU 36 ends the process. On the other hand, if it is determined that the traffic is not congested (NO in S564), the CPU 36 continues the processing from step S568. If the count number K is smaller than the threshold value (NO in S566), the CPU 36 continues the processing from step S568.

  For example, when the threshold is set to 2 and when congestion is determined by three video cameras installed in succession, the vicinity of the video camera at the downstream end of the traffic flow is the traffic jam in the three video cameras. It can be determined that it is a bottleneck, that is, a congestion start position. When the number of video cameras installed along the road is changed according to the road conditions, the threshold value can be changed as appropriate.

  The above-described processing is an example, and instead of determining the presence or absence of congestion, the presence or absence of heavy congestion may be determined. In addition, since the number of continuous video cameras that have been determined to be crowded or crowded according to the size of the finally counted number is known, the start position of the crowded (or crowded) and the crowded (or crowded) The trailing position can also be obtained. Further, based on the separation distance of each video camera, it can be determined how much range (distance) the congestion (or large congestion) has occurred.

  Since the congestion range does not suddenly occur or disappears, it is possible to smooth the congestion range at a plurality of points in time and increase the accuracy. As a smoothing method, for example, the average, mode, or median position of the congestion start position at several times before and after time t can be set as the congestion start position at time t. The trailing position can also be obtained.

  As described above, in the present invention, a motion vector of a connected block obtained by connecting adjacent pixel blocks is calculated based on a change in the position of a matching pixel block specified by different captured images at the time of imaging, and the calculated motion Depending on the vector or the motion vector distribution density calculated based on the motion vector, not only traffic information such as the number of vehicles traveling, speed, and direction, but also determination of the degree of traffic jam, traffic accidents, parked vehicles, or Based on the temporal and spatial distribution of vehicles, such as judgment of traffic abnormalities due to obstacles such as obstacles, judgment of traffic violations against traffic regulations according to places on the road, judgment of occurrence of traffic accidents, etc. The traffic situation can be determined simultaneously and in a wide range.

  In the above-described embodiment, the video camera 1 and the image processing device 2 are configured as separate devices. However, the video camera 1 and the image processing device 2 are integrated into a single device. Also good.

  In the above-described embodiment, the video camera 1 is arranged near the intersection. However, the arrangement location of the video camera 1 or the image processing device 2 is not limited to this, and the traffic situation is determined. Needless to say, it may be placed in any place as long as it is necessary to do so.

  In the above-described embodiment, the motion vector distribution density is calculated based on the ratio of the number of pixel blocks constituting the connected block specified by the captured image to the number of pixel blocks included in the captured image for one frame. However, the motion vector distribution density is not limited to this, and any index may be used as long as it is an index that can evaluate the spatial distribution of the connected blocks in the captured image.

  In the above-described embodiment, the traffic jam is determined based on the average value of the motion vector (speed) calculated every time the calculation time T2 and the average value of the motion vector distribution density. The determination is not limited to this. For example, the calculation time can be changed according to the time zone, or may be changed according to the determination result. It goes without saying that a desired time is appropriately used as the calculation time according to the road conditions.

  In the above-described embodiment, whether or not the traffic abnormality continues for the determination time T1 based on the average value of the motion vector (speed) calculated every calculation time T2 and the average value of the distribution density of the motion vectors. However, the present invention is not limited to this, and any method can be used as long as it can be confirmed that the traffic abnormality continues for the required time. Good.

  In the embodiment described above, the average value of the motion density of the connected blocks and the distribution density of the motion vectors is calculated. However, the present invention is not limited to this, and it corresponds to traffic conditions such as the mode value and the median value. Alternatively, a desired statistical value may be used.

  In the above-described embodiment, the traffic jam determination table and the restriction information table are examples, and are not limited thereto. The threshold value of velocity or distribution density can be changed as appropriate, and other information can be added. In addition, the control center device 3 can update the image processing devices 2, 2,... To different desired traffic congestion determination tables and restriction information tables, respectively, according to traffic conditions. In addition, the direction of the restriction vector specifies the travel prohibition direction, but a travel allowable direction can also be used.

  In the above-described embodiment, when acquiring captured images at different imaging time points, the time interval at the imaging time point can be set as appropriate according to the traffic situation of the road. The time interval can be shortened for roads where vehicles travel at relatively high speeds, and the time interval can be lengthened for roads with many low-speed traveling vehicles.

  In the above-described embodiment, the motion vector of each pixel block is calculated from the captured image obtained by capturing with the video camera. However, the motion vector of each pixel block is captured with the video camera. The motion vector information of each pixel block included in the video data converted into the encoded video format such as H.264 may be used. Here, MPEG, MPEG2, MPEG4, H.264. The encoded video such as H.264 compresses video data by detecting and storing only moving parts in an image. When converting to this format, the pixel block is specified and each The motion vector of the pixel block is calculated.

It is a schematic diagram which shows the outline | summary of the traffic condition determination system which concerns on embodiment of this invention. It is a block diagram which shows the structure of an image processing apparatus and a control center apparatus. It is a flowchart which shows the process sequence of traffic condition determination. It is a flowchart which shows the process sequence of a frame image process. It is explanatory drawing which shows the motion vector of a pixel block. It is explanatory drawing which shows the example of a connection block. It is explanatory drawing which shows the example of a captured image. It is explanatory drawing which shows the calculation information calculated by frame image processing. It is a flowchart which shows the process sequence of traffic congestion determination. It is a record layout of a traffic jam determination table. It is a flowchart which shows the process sequence of traffic abnormality determination. It is a time chart which shows the example of traffic abnormality determination. It is a schematic diagram which shows the outline | summary of the traffic condition determination system which concerns on Embodiment 2. FIG. It is a flowchart which shows the process sequence of traffic condition determination. It is a flowchart which shows the process sequence of a frame image process. It is a flowchart which shows the processing procedure of traffic violation determination. It is explanatory drawing which shows the example of the control area | region of a captured image. It is a record layout of a regulation information table. It is a flowchart which shows the process sequence of traffic accident determination. It is explanatory drawing which shows the example of the overlap area | region of a captured image. It is a schematic diagram which shows the outline | summary of the traffic condition determination system which concerns on Embodiment 3. FIG. It is a flowchart which shows the process sequence of a frame image process. It is a flowchart which shows the process sequence of a frame image process. It is a schematic diagram which shows the outline | summary of the traffic condition determination system which concerns on Embodiment 4. It is a flowchart which shows the process sequence of traffic congestion determination. It is a flowchart which shows the process sequence of traffic congestion determination. It is a flowchart which shows the process sequence of traffic congestion determination.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Video camera 2 Image processing apparatus 3 Control center apparatus 4 Communication line 5 Road 6 Imaging range 7 Intersection 21 Image input part 22 A / D conversion part 23 Image memory 24 RAM
25 Communication Unit 26 Auxiliary Storage Unit 27 Storage Unit 28 CPU
29 CD-ROM
31 Communication unit 32 RAM
33 Display unit 34 Storage unit 35 Operation unit 36 CPU
37 Auxiliary storage

Claims (18)

In the image processing apparatus that processes the acquired captured image and determines the traffic situation,
Means for identifying corresponding pixel blocks of captured images at different imaging points;
Means for calculating a motion vector of the pixel block based on a change in position of the pixel block;
Means for identifying a concatenated block obtained by concatenating adjacent pixel blocks in which the directions of the calculated motion vectors substantially match;
Vector calculation means for calculating a motion vector of the connected block based on a motion vector of each pixel block of the connected block;
Density calculating means for calculating a motion vector distribution density based on the number of pixel blocks of the connected block;
An image processing apparatus comprising: a determination unit that determines a traffic situation based on the distribution density or a motion vector of a connected block. The density calculating means and vector calculating means are:
The distribution density and the motion vector of the connected block are calculated a plurality of times in a predetermined time,
A statistical value including at least one of an average value, a mode value, and a median value is calculated based on the calculated plurality of distribution densities and motion vectors of connected blocks. The image processing apparatus according to 1. The determination means includes
The image processing apparatus according to claim 2, wherein a traffic congestion situation is determined based on the distribution density or the size of a motion vector of a connected block. Storage means for storing a standard direction for determining the direction of traffic flow;
An angle difference calculating means for calculating an angle difference between the direction of the motion vector of the connected block and the standard direction;
The determination means includes
The image processing apparatus according to claim 1, wherein when the calculated angle difference is approximately 180 degrees, it is determined that the traffic is abnormal. Storage means for storing a standard direction for determining the direction of traffic flow;
An angle difference calculating means for calculating an angle difference between the direction of the motion vector of the connected block and the standard direction;
Means for measuring the judgment time, and
The determination means includes
5. The traffic abnormality is determined when the calculated angle difference is equal to or greater than a first threshold until the determination time reaches a predetermined time or more. An image processing apparatus according to 1. Judgment areas corresponding to multiple lanes in the same traveling direction are provided in the captured image,
Means for comparing the magnitude of the motion vector of the connected block with a second threshold;
Means for measuring the judgment time, and
The determination means includes
Until the determination time becomes a predetermined time or more, the size of the motion vector of the connected block in one determination region is smaller than the second threshold, and the size of the motion vector of the connected block in the other determination region is the second The image processing apparatus according to any one of claims 1 to 4, wherein when it is larger than the threshold value, it is determined that the traffic is abnormal. Means for comparing the distribution density with a third threshold;
The determination means includes
The traffic abnormality is determined when the distribution density of the one determination region is larger than the third threshold until the determination time becomes a predetermined time or more. Image processing device. Judgment areas corresponding to multiple lanes in the same traveling direction are provided in the captured image,
Means for comparing the magnitude of the motion vector of the connected block with a second threshold;
Means for measuring the judgment time, and
The determination means includes
If the connected block is not specified in one determination area and the size of the motion vector of the connected block in the other determination area is larger than the second threshold until the determination time is equal to or longer than a predetermined time, traffic abnormality is determined. The image processing apparatus according to claim 1, wherein the determination is made. A restricted area corresponding to traffic restrictions is provided in the captured image,
The traffic flow regulation direction in the regulation area is stored,
Means for detecting the entry of the connecting block into the restricted area;
Means for comparing the direction of the motion vector of the connecting block and the restriction direction;
The determination means includes
9. A traffic violation is determined when the direction of a motion vector of a connected block that has entered the restriction area substantially matches the restriction direction. The image processing apparatus described. Means for identifying an overlapping region between the connected blocks based on a motion vector of the connected blocks;
Means for comparing the number of pixel blocks in the overlapping region with a fourth threshold value,
The determination means includes
The difference in luminance value of the pixels in the overlapping area between the first imaging time when the number of pixel blocks in the overlapping area is equal to or greater than the fourth threshold and the second imaging time before the first imaging time is The image processing apparatus according to claim 1, wherein a traffic accident is determined when the value is equal to or greater than the fifth threshold value. The determination means includes
A traffic accident is determined when the difference between the first imaging time point and the second imaging time point of the luminance value of each pixel in the region where the connected block exists at the second imaging time point is equal to or greater than a sixth threshold value. The image processing apparatus according to claim 10, wherein: In a traffic situation determination system for imaging a region including a road with an imaging device, and processing a captured image obtained by imaging with an image processing device to determine a traffic situation,
The image processing apparatus includes:
Means for identifying corresponding pixel blocks of captured images at different imaging points;
Means for calculating a motion vector of the pixel block based on a change in position of the pixel block;
Means for identifying a concatenated block obtained by concatenating adjacent pixel blocks in which the directions of the calculated motion vectors substantially match;
Vector calculation means for calculating a motion vector of the connected block based on a motion vector of each pixel block of the connected block;
Density calculating means for calculating a motion vector distribution density based on the number of pixel blocks of the connected block;
A traffic condition determination system comprising: determination means for determining a traffic condition based on the distribution density or a motion vector of a connected block. In a traffic situation determination system comprising an imaging apparatus that captures an area including a road, an image processing apparatus that processes a captured image obtained by imaging, and a determination apparatus that determines a traffic situation,
The image processing apparatus includes:
Means for identifying corresponding pixel blocks of captured images at different imaging points;
Means for calculating a motion vector of the pixel block based on a change in position of the pixel block;
Means for transmitting the calculated motion vector to the determination device;
The determination device includes:
Means for receiving a motion vector transmitted by the image processing device;
Means for identifying a concatenated block obtained by concatenating adjacent pixel blocks whose directions of the received motion vectors substantially coincide;
Vector calculation means for calculating a motion vector of the connected block based on a motion vector of each pixel block of the connected block;
Density calculating means for calculating a motion vector distribution density based on the number of pixel blocks of the connected block;
A traffic condition determination system comprising: determination means for determining a traffic condition based on the distribution density or the motion vector of the connected block. Three or more image pickup devices that pick up images of different areas including roads and image processing devices that process picked-up images picked up by the image pickup devices are provided along the roads.
The determination device includes:
Based on the distribution density or the motion vector of the connected block, the traffic congestion situation is determined,
The second traffic jam situation determined by the captured image captured by one imaging device is determined based on the captured images captured by the two most recent imaging devices installed with the one imaging device in between. 14. The traffic situation determination system according to claim 13, wherein when it is different from the traffic jam situation, the second traffic jam situation is judged. The determination device includes:
At one time point, the first traffic jam situation determined by the picked-up image picked up by the one image pickup device is a picked-up image picked up by each of the two latest image pickup devices installed with the one image pickup device in between. When the second traffic congestion situation is different from the second traffic congestion situation, the traffic congestion situation judged by the captured images captured by the one imaging device at a plurality of other time points different from the one time point is the second traffic condition. 15. The traffic situation determination system according to claim 14, wherein when it is a traffic jam situation, it is judged as a second traffic jam situation. The determination device includes:
When traffic congestion conditions determined by captured images captured by a plurality of continuous imaging devices are the same, a traffic congestion change point is determined based on the position of the imaging device at the downstream end of the traffic flow among the imaging devices. The traffic condition determination system according to claim 14 or 15, wherein the system is configured as described above. In a computer program for causing a computer to process an acquired captured image and determine a traffic situation,
Computer
Means for identifying a connected block obtained by connecting adjacent pixel blocks having substantially the same direction of motion vector;
Vector calculation means for calculating a motion vector of the connected block based on a motion vector of each pixel block of the connected block;
Density calculating means for calculating a motion vector distribution density based on the number of pixel blocks of the connected block;
A computer program that functions as a determination unit that determines a traffic situation based on the distribution density or a motion vector of a connected block. In a traffic situation determination method of imaging a region including a road with an imaging device and processing a captured image obtained by imaging to determine a traffic situation,
Identify connected blocks that connect adjacent pixel blocks whose motion vector directions substantially match,
Calculating a motion vector of the connected block based on a motion vector of each pixel block of the connected block;
Calculating a motion vector distribution density based on the number of pixel blocks of the connected block;
A traffic situation determination method, wherein a traffic situation is determined based on the distribution density or a motion vector of a connected block.

Images