JP5936527B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method, and in particular, periodically captures and processes an image including a moving object such as a vehicle traveling on a road to process traffic flow on an outdoor road or a tunnel. The present invention is applied to a road traffic monitoring system that monitors traffic by measuring traffic flow.

  In a road traffic monitoring system using image processing, speed and the like are measured from a change in the position of a vehicle traveling on the road, and the position information and speed information are used for determination of a road traffic event. For this reason, it is important to perform measurement processing as to which place in a three-dimensional space an arbitrary place in a two-dimensional image from a camera as an image sensor is present. In addition, for installation information such as camera declination and depression angle, it is necessary to set a reference position on the road and measure the distance from the position directly under the camera, and there is a problem that the cost increases as the number of cameras increases.

JP 2006-17676 A

  Therefore, the measurement method described in Patent Document 1 is characterized in that feature points such as a tail lamp and a headlight of a vehicle are measured in advance, and camera installation information is calculated by linearly moving in the measurement region. . However, in the above-mentioned Patent Document 1, it is necessary to pre-measure the characteristic points of the measurement vehicle, and it is necessary to move linearly in the road, and there are still restrictions on pre-preparation and photographing conditions.

  The present invention has been made to solve the above-described problems, and is intended to acquire installation information of a camera as an image sensor without restrictions such as prior surveying of a specific measurement point or road shape. .

In the image processing apparatus according to the present invention, an image periodically captured by an image sensor installed under a predetermined condition is processed, and a displacement state of the object to be imaged in a three-dimensional space is specified from a captured image that is a two-dimensional plane. In the apparatus, vanishing point information is acquired from an imaging unit that periodically captures an image with the imaging element, and a reduction rate of a mutual interval in a parallel linear linear extension direction extracted based on an image captured by the imaging unit. From vanishing point acquisition means, installation information acquisition means for acquiring installation information of the image sensor based on the vanishing point information acquired by the vanishing point acquisition means, and a two-dimensional movement state in a captured image of the imaged object Correction information acquisition means for adjusting the installation information of the image pickup device and acquiring correction information based on the obtained three-dimensional movement state of the object to be imaged; The obtaining means adjusts the installation information of the image sensor to obtain correction information so that the three-dimensional movement ratio obtained by converting the two-dimensional movement history in the photographed image of the imaged object to a three-dimensional movement history approaches a predetermined value. It is characterized by doing.

Further, in the image processing method according to the present invention, an image periodically captured by an image sensor installed under a predetermined condition is processed, and a displacement state in a three-dimensional space of an object to be imaged is determined from a captured image that is a two-dimensional plane. A method of specifying an imaging process in which images are periodically captured by the imaging device, and disappearance from a reduction rate of a mutual interval in a linear linear extension direction extracted based on a captured image in the imaging process In a vanishing point acquisition process of acquiring point information, an installation information acquisition process of acquiring installation information of the imaging element based on the vanishing point information acquired in the vanishing point acquisition process, and a captured image of the imaged object Correction information for adjusting the installation information of the imaging element acquired in the installation information acquisition process and acquiring correction information based on the three-dimensional movement state of the imaged object obtained from the two-dimensional movement state Including the acquisition process, and in the correction information acquisition process, the installation information acquisition process so that a three-dimensional movement ratio obtained by converting a two-dimensional movement history in the photographed image of the object to be captured into a three-dimensional movement history approaches a predetermined value. The correction information is acquired by adjusting the installation information of the image pickup device acquired in (1) .

According to the image processing apparatus of the present invention, the reduction ratio of the mutual interval in the linear linear extension direction extracted based on the image pickup means that periodically picks up an image with the image pickup device and the image picked up by the image pickup means Vanishing point acquisition means for acquiring vanishing point information from the camera, installation information acquisition means for acquiring installation information of the image sensor based on the vanishing point information acquired by the vanishing point acquisition means, and imaging of the object to be imaged Correction information acquisition means for adjusting the installation information of the image sensor and acquiring correction information based on the three-dimensional movement state of the object to be imaged obtained from the two-dimensional movement state in the image, and the correction information acquisition means includes the correction information acquisition means The installation information of the image sensor is adjusted to correct the correction information so that the three-dimensional movement ratio obtained by converting the two-dimensional movement history in the photographed image of the imaged object into the three-dimensional movement history approaches a predetermined value. By obtaining, we obtain the installation information of the image pickup device from the captured image, without requiring a complicated work such as driving the test vehicle, with can perform accurate image processing, to obtain a proper value of the image pickup device installation information Therefore, an image processing apparatus that can perform optimized image processing can be obtained.

Further, according to the image processing method of the present invention, an imaging process in which images are periodically captured by the imaging device, and a mutual interval in a parallel linear linear extending direction extracted based on the captured image in the imaging process A vanishing point acquiring process for acquiring vanishing point information from a reduction rate of the image, an installation information acquiring process for acquiring installation information of the image sensor based on the vanishing point information acquired in the vanishing point acquiring process, and the imaged object Correction information acquisition process for adjusting the installation information of the imaging element acquired in the installation information acquisition process and acquiring correction information based on the three-dimensional movement state of the object to be imaged obtained from the two-dimensional movement state in the captured image of the object In the correction information acquisition process, the three-dimensional movement ratio obtained by converting the two-dimensional movement history in the photographed image of the imaged object into a three-dimensional movement history is close to a predetermined value. By adjusting the installation information of the imaging element obtained by the location information acquisition step acquires the correction information, acquires the installation information of the image pickup device from the captured image, without requiring a complicated work such as driving the test vehicle Thus, it is possible to obtain an image processing method that can perform accurate image processing, obtain an appropriate value of image sensor installation information, and perform optimized image processing.

It is a block diagram which shows the structure of the image processing apparatus in Embodiment 1 by this invention. It is a flowchart which shows the process of the vanishing point calculation part in Embodiment 1 by this invention. It is a flowchart which shows the process of the camera information measurement part in Embodiment 1 by this invention. It is a flowchart which shows the process of the object detection part in Embodiment 1 by this invention. It is a block diagram which shows the structure of the image processing apparatus in Embodiment 2 by this invention. It is a block diagram which shows the structure of the image processing apparatus in Embodiment 3 by this invention. It is explanatory drawing regarding the movement locus | trajectory which shows the condition of the multilane vanishing point calculation part in Embodiment 3 by this invention. It is a block diagram which shows the structure of the image processing apparatus in Embodiment 4 by this invention. It is explanatory drawing regarding the linear conversion from the curve which shows the vanishing point calculation condition in the curve road in Embodiment 4 by this invention to a straight line. It is a block diagram which shows the structure of the image processing apparatus in Embodiment 5 by this invention. It is a diagram which shows the speed distribution of the two or more vehicles in Embodiment 5 by this invention. It is a flowchart which shows the process of the parameter correction process part in Embodiment 5 by this invention. It is a block diagram which shows the structure of the image processing apparatus in Embodiment 6 by this invention.

Embodiment 1 FIG.
Embodiment 1 of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing a configuration of a camera installation information measuring apparatus as an image processing apparatus in the first embodiment. FIG. 2 is a processing flow of the vanishing point calculation unit in the first embodiment. FIG. 3 is a flowchart showing a processing flow of the object detection unit in the first embodiment. FIG. 4 is a flowchart showing a processing flow of the camera installation information calculation processing unit in the first embodiment.

In FIG. 1 showing the configuration of the image processing apparatus according to the first embodiment of the present invention, an image capturing unit 10 captures an image with a predetermined cycle by an image sensor such as a monitoring ITV camera, and stores the image as digital data. .
A vanishing point calculating unit 11 that calculates a vanishing point from an image captured by the image capturing unit 10 is a three-dimensional vanishing point based on a parallel linear reduction rate in a vehicle traveling direction extracted from a white line that is a feature of road alignment. Is a processing unit for calculating.
The object detection unit 12 is a processing unit that detects a vehicle passing on a road from an image input from the image capturing unit 10 and measures a movement locus for each imaging cycle in the same vehicle.
The camera information measurement unit 13 is a processing unit that calculates optimal camera installation information, and is a camera information approximate measurement that constitutes an installation information acquisition unit that acquires approximate camera information from the vanishing point calculated by the vanishing point calculation unit 11. 13A and a camera information correction unit 13B constituting correction information acquisition means for calculating optimal camera installation information from the approximate information acquired by the camera information approximate measurement unit 13A and the movement locus calculated by the object detection unit 12. Is done.

  Here, each processing unit such as the vanishing point calculation unit 11, the object detection unit 12, and the camera information measurement unit 13 has a calculation central function using a CPU, a main memory, and the like, and a storage device such as an HDD or an input / output device is provided. Including a peripheral device including a general-purpose information processing device to which an image pickup device including a camera is connected, and a required processing operation such as an automatic calculation process by a predetermined program stored in a storage device Is done.

The processing operation of the image processing apparatus according to the first embodiment of the present invention is performed at the stages [1] and [2] shown below.
[1] Vanishing point information acquisition (1A) Parallel linear extraction: Approximate straight lines L1 and L2 are extracted from the left and right white lines of the captured image.
(1B) Vanishing point information derivation: Vanishing point coordinate information is derived from the approximate lines L1 and L2.
[2] Installation information acquisition (2A) Approximate installation information acquisition: An installation information approximate value is acquired from vanishing point coordinate information.
(2B) Acquisition of correction information: Correction information for correcting and fixing the approximate value of installation information is acquired.
(2B-1) Two-dimensional movement history extraction: A two-dimensional movement history of a vehicle is extracted from a captured image.
(2B-2) Movement history three-dimensional conversion: A two-dimensional movement history is converted into a three-dimensional movement history.
(2B-3) Movement ratio calculation / optimization: A three-dimensional movement ratio is calculated from a three-dimensional movement history and optimized to obtain correction information.

[1] Vanishing Point Information Acquisition FIG. 2 shows a processing flow of the vanishing point calculation unit 11. In processing step S11, an edge of a road white line in the image is extracted, and an edge sequence of left and right white lines is least squared in processing step S12. Straight line approximation is performed using the sum method, and the vanishing point in the image is calculated from the intersection of the two approximate lines L1 and L2 in processing step S13.
(1A) Parallel Linear Extraction Here, the road white line edge extraction processing in processing step S11 and the approximate straight line derivation processing from the left and right white line edge sequences E1 and E2 extracted in processing step S12 will be described.
In processing step S11, white line edge rows E1 and E2 provided in the vehicle traveling direction along the left and right edges of the road are extracted from the image captured by the image capturing unit 10, respectively.
In the processing step S12, straight line approximation is performed using the least square sum method for the extracted edge sequences E1 and E2 of the left and right white lines, and a parallel linear shape forming a pair of two straight lines L1 and L2 indicated by the approximate straight line is obtained.
When the coordinate distribution for the edge line E1 of the left white line is as follows:
(X, y) = (x1, y1) (x2, y2) ... (xn, yn)
Assuming that the linear approximation formula is y = ax + b, the slope a and the intercept b are obtained by the following formulas 1 and 2 from the least square sum, and the straight line L1 can be obtained.
An approximate straight line L2 can be similarly obtained for the edge row E2 of the right white line.

(1B) Vanishing point information derivation In processing step S13, the parallel linear extending direction formed by the pair of approximate straight lines L1 and L2 obtained in processing step S12, that is, the parallel forming a pair in the traveling direction of the vehicle along the road white line. The intersection of the parallel lines is obtained from the reduction rate of the mutual interval for the lines L1 and L2, and the vanishing point coordinates are calculated.
The camera installation information calculation method will be described. In the vanishing point calculation unit 11, any two points on the straight line L1 and the straight line L2 obtained in the processing step S12 are defined as P1 (x1, y1), P2 (x2, y2), P3 (x3, y3), P4 (x4, y4). ), And d is the vector dot product of each straight line, d is expressed by Expression 3, and the vanishing point coordinates (p, q) that are the intersections of the two straight lines are expressed by Expression 4 and Expression 5.
d = (y2-y1) * (x4-x3)-(y4-y3) * (x2-x1) Equation 3
p = ((x2-x1) * (x4 * y3-x3 * y4)
+ (X3-x4) * (x2 * y1-x1 * y2)) / d .............
q = ((y2-y1) * (y4 * x3-y3 * x4)
+ (Y3−y4) * (y2 * x1−y1 * x2)) / (− d) (5)

[2] Acquisition of Installation Information Next, a method for calculating camera installation information in the camera information measurement unit 13 will be described. As known information, the camera installation height H, lens focal length F, lens element size C, vanishing point information measured by the vanishing point calculation unit 11, movement history measured by the object detection unit 12, and vehicle passing speed information are found. doing. From these pieces of information, the camera deflection angle α and depression angle β information is calculated. The camera height H and the lens focal length F are approximate values because it is difficult to pre-calculate strictly accurate values because they include camera installation conditions and preset errors in the zoom turning type camera.

A processing flow in the camera information measurement unit 13 is shown in FIG. In processing step S21 by the camera information approximate measurement unit 13A, approximate declination angle α and depression angle β are calculated from vanishing point information, approximate values of camera height H and lens focal length F.
In the processing step S22 by the camera information correction unit 13B, the two-dimensional movement history is converted into the three-dimensional movement history from the approximate camera installation information calculated in the processing step S21, and the three-dimensional movement ratio is calculated. Similarly, in the processing step S23 by the camera information correction unit 13B, the moving ratio calculated in the processing step S32 becomes 1.0, and the camera height H is fixed in a state where the vanishing point position is fixed so that the vehicle speed matches the constant speed value. And the lens focal length F is changed to calculate the closest value.
The camera installation information composed of the declination angle α and the depression angle β calculated by the camera information approximate measurement unit 13A in the processing step S21 is a two-dimensional movement history and a three-dimensional movement related to a moving object such as a vehicle in a captured image by the image capturing unit 10. This indicates a correlation with a history, and by using this camera installation information, a two-dimensional movement history relating to a moving object such as a vehicle is converted into a three-dimensional movement history, a three-dimensional movement ratio is calculated, and a camera height H And the lens focal length F are optimized.

(2A) Acquisition of approximate installation information The camera installation information consisting of the declination angle α and the depression angle β calculated in the processing step S21 by the camera information approximate measurement unit 13A is related to a moving object such as a vehicle in a captured image by the image capturing unit 10. A correlation between a three-dimensional movement history and a three-dimensional movement history,
First, in processing step S21 by the camera information approximate measurement unit 13A, the approximate declination α and depression angle β are calculated from the vanishing point information calculated by the vanishing point calculation unit 11, the approximate value of the camera height H and the lens focal length F. To do.
The relationship between the vanishing point coordinate information calculated by the vanishing point calculation unit 11 in the processing step S13 and the declination angle α and the depression angle β of the camera obtained from the vanishing point coordinate information (p, q) is expressed by the following equation 6 and Equation 7 is obtained.
p = −F / C * tan (α) / cos (β) …………………………………
q = F / C * tan (β) …………………………………………………… Equation 7
From these equations, the following equations 8 and 9 are used to calculate the camera deflection angle α and depression angle β.
α = atan ((S / F) * q) .......................................
β = atan (− (S / F) * p * cos (β)) ……………………………… Equation 9
From the above, the camera deflection angle α and the depression angle β are values that depend on the focal length F of the camera and the lens element size C, and are values that do not depend on the camera height H.

(2B) Acquisition of Correction Information The camera information correction unit 13B obtains the two-dimensional movement history of the vehicle input from the object detection unit 12 based on the installation information calculated by the camera information approximate measurement unit 13A in the processing step S21. In step S22, a three-dimensional movement ratio is derived from the three-dimensional movement history. Then, optimization is performed so that the derived three-dimensional movement ratio is a predetermined value 1.0, and correction information is acquired.
(2B-1) Two-dimensional movement history extraction The object detection unit 12 extracts a two-dimensional movement history of the vehicle from the image captured by the image capturing unit 10. A method for measuring the same vehicle movement locus in the object detection unit 12 will be described.
FIG. 4 shows a processing flow of the object detection unit 12. In processing step S31, a moving object composed of vehicles existing in the image is detected, and in processing step S32, there is a vehicle at the lower end of the detected vehicle, that is, a vehicle on the image. Determine the road surface position. In processing step S33, the movement history of the vehicle bottom coordinate obtained from the results of processing steps S21 and S22 is managed with continuous input images.

  In the processing step S31 in the object detection unit 12, a moving object in the image is detected from the change area extracted by the background difference method or the inter-frame difference method, and the object having the characteristics of the vehicle is specified from among the moving objects. In addition, the extraction of the change region is not fixed as described above, and the change region may be extracted by paying attention to the fact that a vehicle having a sharp edge change passes on a road surface with poor edge change.

  In the processing step S32 in the object detection unit 12, the position which is the lower end on the road surface of the vehicle detected in the processing step S31 is calculated using the video feature. In a tunnel with daytime outdoor environments and lighting facilities, the road surface brightness value in the image is brighter than the shadow of the lower part of the vehicle generated by the vehicle. Therefore, it is possible to specify the lower end portion of the vehicle by searching for a place where the brightness is lower than the road luminance around the detected lower end of the vehicle.

  In the processing step S33 in the object detection unit 12, the movement history of the lower end position of the vehicle passing on the road is managed. When the vehicle travels at a constant speed, the two-dimensional movement ratio of the movement history is There is a dependency with the ratio in the vanishing point direction calculated by the vanishing point calculation unit 11. Further, the three-dimensional movement ratio when traveling at a constant speed is always constant 1.0.

(2B-2) Movement history three-dimensional conversion Further, the two-dimensional movement history measured by the object detection unit 12 is converted from the calculated camera installation information into a three-dimensional movement history, and the three-dimensional movement ratio becomes 1.0. Further, the vehicle speed is obtained from the moving distance per unit time, and the camera height H and the focal length F are changed so that the vehicle speed becomes equal to the constant speed value of the traveling vehicle, and the optimum solution is obtained.
(2B-3) Movement Ratio Calculation / Optimization The procedure for calculating the three-dimensional movement ratio from the three-dimensional movement history by the camera information correction unit 13B is as follows.
When the three-dimensional movement history for time n is defined by the following function:
Pn (x (n), y (n), z (n)) n = 1 ... n
To simplify the equation, assuming the z axis is the direction of travel of the moving object,
v = (Σz (n + 1) −z (n)) / (n−1) n = 1... n−1
By doing so, the average moving distance v in each time period is calculated.
d = (Σz (n + 1) −z (n)) / v n = 1... n−1
Thus, the ratio of the movement distance at each time to the average movement distance is cumulatively added, and this cumulative addition value becomes the movement ratio d, and this movement ratio is the camera closest to 1.0. Height H and focal length F are determined as optimum parameters.

In optimizing the movement ratio, the information processing apparatus automatically calculates the camera height H and the focal length F.
The values of the camera height H and the focal length F are changed in a round-robin format within a specific range at regular intervals, and a value in which the movement ratio after the three-dimensional movement history conversion in that case is closest to 1.0 is searched.
Regarding the focal length F, in the case of a fixed focal length lens, the calculation amount can be limited by giving a basic focal length (8 mm, 16 mm, 20 mm, 25 mm, 35 mm) in advance.

  The optimization based on the movement ratio is performed using the comparison processing step S24 by the comparison processing unit shown in FIG. Whether the three-dimensional movement ratio derived from the processing step S23 is 1.0 in the comparison processing step S24 and whether the speed calculation value derived from the processing step S23 meets the definition such as a predetermined speed. Determine if. If yes, the installation information such as the declination angle α, the depression angle β, and the focal length F is determined and derived as a corrected parameter output. If no, processing step S22 and processing step S23 are repeated. Thereby, the optimum value of the camera installation information can be calculated from the captured image of the vehicle traveling in the steady state.

  As described above, in the first embodiment, a three-dimensional vanishing point is calculated from a lane shape such as a white line on the road, and other known information is changed by recognizing a vehicle passing at a constant speed whose speed value is known. This makes it possible to calculate the optimum value of the camera installation information.

  The image processing apparatus according to the first embodiment processes an image periodically photographed by an imaging device such as an ITV camera in which installation conditions such as a declination α, a depression angle β, and a focal length F are installed under predetermined conditions. Traffic is determined by identifying the displacement state of the object to be imaged in a three-dimensional space from a photographed image that is a two-dimensional plane and measuring the speed etc. from the change in position of a moving object such as a vehicle traveling on an outdoor road surface or a road surface in a tunnel. In an apparatus that measures a flow and uses position information and speed information for determination of road traffic events, etc., an imaging unit that periodically captures an image by the imaging element, and a parallel that is extracted based on a captured image by the imaging unit Vanishing point acquisition means for acquiring vanishing point information from the reduction rate of the mutual interval in the linear linear extension direction, based on the vanishing point information acquired by the vanishing point acquisition means Since it has installation information acquisition means for acquiring installation information such as declination α and depression angle β related to the image sensor, it is necessary to perform complicated work such as acquiring the image sensor installation information from the captured image and running the test vehicle. Accurate image processing can be performed without this.

  The image processing apparatus according to the first embodiment converts the two-dimensional movement history in the photographed image of the object to be imaged into a three-dimensional movement history according to the installation information such as the declination angle α and the depression angle β. The installation information of the image sensor is adjusted so that the obtained three-dimensional movement ratio approaches the predetermined value 1.0, that is, 3 of the imaged object obtained from the two-dimensional movement state in the captured image of the imaged object. Since there is a correction information acquisition means for adjusting the installation information of the image pickup device and acquiring correction information based on the dimension movement state, complicated work such as acquiring the installation information of the image pickup device from the photographed image and running the test vehicle In addition to being able to perform accurate image processing without accompanying, it is possible to obtain an appropriate value of image sensor installation information consisting of camera installation information such as camera height H and focal length F. It is possible to perform image processing.

  Further, in the image processing method according to the first embodiment, an image picked up periodically by an image pickup device such as an ITV camera in which installation conditions such as declination α and depression angle β are installed under predetermined conditions is processed to be two-dimensional. The displacement state in the three-dimensional space of the object to be imaged is determined from the captured image, which is a flat surface, and the traffic flow is measured by measuring the speed etc. from the change of the position of the moving object such as the vehicle traveling on the road surface in the outdoor road or in the tunnel. A method of measuring and using the position information and speed information for determination of road traffic events, etc., which is extracted based on an imaging process in which images are periodically taken by the imaging device and images taken in the imaging process The vanishing point acquisition process for acquiring vanishing point information from the reduction rate of the mutual interval in the linear extension direction of the parallel linear, and the image sensor based on the vanishing point information acquired by the vanishing point acquisition process Because it includes an installation information acquisition process for acquiring installation information such as declination α and depression angle β, without acquiring complicated information such as acquiring the installation information of the image sensor from the captured image and running the test vehicle, Accurate image processing can be performed.

  In the image processing method according to the first embodiment, the two-dimensional movement history in the captured image of the object to be imaged is converted into the three-dimensional movement history according to the setting information such as the declination angle α and the depression angle β in the method described in the previous section. The installation information of the image sensor is adjusted so that the obtained three-dimensional movement ratio approaches the predetermined value 1.0, that is, 3 of the imaged object obtained from the two-dimensional movement state in the captured image of the imaged object. Since it has a correction information acquisition process that adjusts the installation information of the image sensor and acquires correction information based on the dimension movement state, it is complicated work such as acquiring the installation information of the image sensor from the photographed image and running the test vehicle Without being accompanied by accurate image processing, and it is possible to obtain an appropriate value of image sensor installation information including camera installation information such as camera height H and focal length F, Appropriate image processing can be performed.

Embodiment 2. FIG.
A second embodiment of the present invention will be described with reference to FIG. FIG. 5 is a block diagram showing the configuration of the camera installation information measuring apparatus according to the second embodiment. In the second embodiment, the blocks other than the specific configuration described here have the same configuration contents as the configuration in the first embodiment described above, and perform the same work. The same reference numerals in the drawings indicate the same or corresponding parts.

In FIG. 4 showing the configuration of the image processing apparatus according to the second embodiment, the point where the vanishing point calculation unit 11 uses the result of the vehicle detection unit 12 is different from the first embodiment.
In the first embodiment described above, the vanishing point coordinate information is calculated from the shape of the white line on the road, but here the vanishing point information is obtained from the reduction ratio of the lateral width when the vehicle moves, A parallel line is extracted from two-dimensional movement histories of different parts such as a left end and a right end of a vehicle in a moving object made of a vehicle, and vanishing point information is obtained.

The approximate straight line in the left end history and the approximate straight line in the right end history are obtained by the least square sum method from the two-dimensional coordinate history of the left end and the right end of the vehicle detected by the vehicle detection unit 12. The intersection of these two straight lines is the vanishing point coordinate similar to that in the first embodiment.
If the coordinate distribution for the left-end history of the vehicle is as follows:
(X, y) = (x1, y1) (x2, y2) ... (xn, yn)
Assuming that the linear approximation formula is y = ax + b, the slope a and the intercept b are obtained from the least square sum by the formulas 1 and 2 described in the first embodiment, and the straight line L1 can be obtained.
The approximate straight line L2 can be similarly obtained for the right end history of the vehicle.
The vanishing point information can be obtained from the approximate straight lines L1 and L2 extracted in this manner as in the first embodiment.

  As described above, the second embodiment enables the same camera installation information measurement as in the first embodiment even when it is unclear how the road alignment is defined by the white line, and the function can be realized at low cost. .

  The image processing apparatus according to the second embodiment processes an image periodically photographed by an imaging device such as an ITV camera in which the installation conditions such as the declination α and the depression angle β are installed under predetermined conditions, and performs a two-dimensional plane. The traffic flow is measured by identifying the displacement state of the object to be imaged in a three-dimensional space from a certain captured image and measuring the speed etc. from the change in the position of a moving object such as a vehicle traveling on an outdoor road surface or a road surface in a tunnel. In an apparatus that uses the position information and speed information for determination of road traffic events and the like, different parts of an imaging unit that periodically captures an image by the imaging element and a moving object that is extracted based on a captured image by the imaging unit Vanishing point acquisition means for extracting vanishing point information for extracting the parallel line from the two-dimensional movement history of the line, acquiring vanishing point information from the extracted parallel line, and the vanishing point acquiring unit When it is unclear how the road alignment is specified by a white line because it includes installation information acquisition means for acquiring installation information such as declination α and depression angle β related to the image sensor based on the vanishing point information acquired by a stage For example, it is possible to acquire image sensor installation information from a captured image of a moving object such as a general vehicle traveling on a road and perform accurate image processing without complicated operations such as traveling a test vehicle. .

  Further, in the image processing method according to the second embodiment, two-dimensional processing is performed by processing images periodically photographed by an imaging device such as an ITV camera in which the installation conditions such as the declination α and the depression angle β are installed under predetermined conditions. The displacement state in the three-dimensional space of the object to be imaged is determined from the captured image, which is a flat surface, and the traffic flow is measured by measuring the speed etc. from the change of the position of the moving object such as the vehicle traveling on the road surface in the outdoor road or in the tunnel. Measuring and using the position information and speed information for determination of road traffic events, and the like, an imaging process in which images are periodically taken by the imaging device, and movement extracted based on the images taken by the imaging means A vanishing point acquisition process of acquiring the three-dimensional vanishing point information for extracting the parallel line from the two-dimensional movement history of different parts in the object and acquiring vanishing point information from the extracted parallel line; Since there is an installation information acquisition process for acquiring installation information such as declination α and depression angle β related to the image pickup device based on the vanishing point information acquired by the vanishing point acquisition means, it is possible to specify road alignment by white lines. Even if it is unclear, accurate image processing can be performed without complicated operations such as acquiring imaging device installation information from a captured image of a moving object such as a general vehicle traveling on a road and driving a test vehicle. It can be carried out.

Embodiment 3 FIG.
A third embodiment of the present invention will be described with reference to FIGS. FIG. 6 is a block diagram showing the configuration of the image processing apparatus according to the third embodiment. FIG. 7 is an explanatory diagram relating to the movement trajectory showing the situation of the vehicle traveling on the multi-lane detected by the multi-lane vanishing point calculator 16 in the third embodiment. In this third embodiment, the blocks other than the specific configuration described here have the same configuration contents as the configurations in the first and second embodiments described above, and perform the same work. . The same reference numerals in the drawings indicate the same or corresponding parts.

  In FIG. 6 showing the configuration of the image processing apparatus in the third embodiment, a multilane vanishing point calculation unit 16 calculates vanishing point information from a vehicle traveling on the multilane.

  The vanishing point calculation method in the multilane vanishing point calculation unit 16 is shown in FIG. FIG. 7 shows the movement trajectory of the center portion at the lower end of a plurality of vehicles traveling in a plurality of lanes on a road having two or more lanes.

  In the third embodiment, the vanishing point primary approximate line L1 is obtained from the moving locus of the vehicle passing through the left lane, and the vanishing point primary approximate line L2 is obtained from the moving locus of the vehicle passing through the right lane. Thus, the vanishing point position can be calculated by using the same method as in the first embodiment. In this case, with respect to the movement trajectory of the vehicle traveling by changing the lane or the like, if the sum of squares of the residual with the approximate straight line with respect to the movement trajectory position of each vehicle exceeds a certain value Dmax, the vehicle is moved relative to the lane. A vanishing point calculation with higher accuracy is performed by considering the vehicle to change lanes and excluding it from the target vehicle for vanishing point calculation and repeating the vanishing point calculation again.

  As described above, in the third embodiment, the vanishing point can be calculated from the movement trajectory of a general group of vehicles traveling on the road, and correction processing can be performed even during the operation of the monitoring work by using this method.

  As described above, in the third embodiment, the vanishing point can be calculated from the movement trajectory of a general group of vehicles traveling on the road, and correction processing can be performed even during the operation of the monitoring work by using this method.

  The image processing apparatus according to the third embodiment processes an image periodically photographed by an imaging device such as an ITV camera in which the installation conditions such as the declination α and the depression angle β are installed under predetermined conditions, and performs a two-dimensional plane. The traffic flow is measured by identifying the displacement state of the object to be imaged in a three-dimensional space from a certain captured image and measuring the speed etc. from the change in the position of a moving object such as a vehicle traveling on an outdoor road surface or a road surface in a tunnel. In an apparatus that uses the position information and speed information for determination of road traffic events, etc., it travels in an imaging unit that periodically captures an image using the imaging element, and a multi-lane extracted based on the image captured by the imaging unit. Vanishing point acquisition means for deriving parallel alignment from the movement trajectory of the general vehicle group to acquire vanishing point information, and based on the vanishing point information acquired by the vanishing point acquisition means, Since it has installation information acquisition means for acquiring installation information, it does not involve complicated work such as acquiring the installation information of the image sensor from the captured image of a general vehicle group traveling in multiple lanes and running the test vehicle, Accurate image processing can be performed.

  Further, in the image processing method according to the third embodiment, two-dimensional processing is performed by processing images periodically photographed by an imaging device such as an ITV camera in which the installation conditions such as the declination α and the depression angle β are installed under predetermined conditions. The displacement state in the three-dimensional space of the object to be imaged is determined from the captured image, which is a flat surface, and the traffic flow is measured by measuring the speed etc. from the change of the position of the moving object such as the vehicle traveling on the road surface in the outdoor road or in the tunnel. In an apparatus for measuring and using the position information and speed information for determination of road traffic events, an image pickup means for periodically taking an image by the image pickup device, and a multilane extracted based on an image taken by the image pickup means Vanishing point acquisition means for deriving parallel linearity from the movement trajectory of the traveling general vehicle group to acquire vanishing point information, and the imaging based on the vanishing point information acquired by the vanishing point acquisition means. Since it has installation information acquisition means for acquiring the installation information of the image element, it involves complicated work such as acquiring the installation information of the image sensor from the captured image of a general vehicle group traveling in multiple lanes and running the test vehicle. Therefore, accurate image processing can be performed.

Embodiment 4 FIG.
A fourth embodiment of the present invention will be described with reference to FIGS. FIG. 8 is a block diagram showing the configuration of the image processing apparatus according to the fourth embodiment. FIG. 9 is an explanatory diagram relating to a linear conversion from a curve to a straight line showing a vanishing point calculation situation on a curved road in the fourth embodiment. In the fourth embodiment, the blocks other than the specific configuration described here are the same as those in the first to third embodiments described above, and perform the same work. is there. The same reference numerals in the drawings indicate the same or corresponding parts.

  In FIG. 8, the curve road vanishing point calculation unit 14 corresponds to the curve road and the vehicle traveling along the curve in the same manner as the vanishing point calculation unit 11 of the first embodiment.

  The vanishing point calculation method on the curved road is shown in FIG. In FIG. 9A, the edge lines E1 and E2 of the white line on the curved road are extracted, and in FIG. 9B, the X-axis position of the edge section center point C at the same Y-axis coordinate in order from the lower end of the screen is the center line CL. The situation is shown above. Thereby, vanishing point calculation can be performed by the same process as the vanishing point calculation process on the straight road shown in the first embodiment.

  Here, the optimal vanishing point on the curved road can be obtained by fixing the position of the Y coordinate at the vanishing point and changing the position of the X coordinate. However, since the change in the X direction of the vanishing point becomes a change in the X coordinate axis direction in the three-dimensional space, it does not affect the Z coordinate axis direction that is the measurement target of the vehicle speed. For this reason, the two-dimensional Y-coordinate position of the vanishing point is important in a system in which only the vehicle speed and the depth position from the camera can be determined.

  In the vanishing point calculation method based on the change in the vehicle lateral shape in the vehicle travel history according to the second embodiment described above, the vanishing point can be obtained by applying the fourth embodiment to a vehicle traveling in a curve. Become.

  Furthermore, also in the vanishing point calculation method based on the movement history of the group of vehicles traveling on the multi-lane according to the above-described third embodiment, the vanishing point can be obtained by applying the fourth embodiment to the vehicle traveling on the curve. It becomes.

  As described above, in the fourth embodiment, even when the road alignment is a curved road, it is possible to calculate the vanishing point and measure the camera installation information as before.

  The image processing apparatus according to the fourth embodiment processes an image periodically photographed by an imaging element such as an ITV camera in which the installation conditions such as the declination α and the depression angle β are installed under predetermined conditions, and performs a two-dimensional plane. The traffic flow is measured by identifying the displacement state of the object to be imaged in a three-dimensional space from a certain captured image and measuring the speed etc. from the change in the position of a moving object such as a vehicle traveling on an outdoor road surface or a road surface in a tunnel. In an apparatus that uses the position information and speed information for determination of road traffic events, etc., an imaging unit that periodically captures an image with the imaging element, and a curved road shape extracted based on a captured image by the imaging unit A parallel alignment is derived based on the vanishing point acquisition means for acquiring vanishing point information from the reduction rate of the mutual interval in the linear extension direction of the derived parallel alignment, and acquired by the vanishing point acquisition means Installation information acquisition means for acquiring the installation information of the image pickup device based on the vanishing point information that has been obtained, so that complicated work such as acquiring the installation information of the image pickup device from the captured image and running the test vehicle Accurate image processing can be performed without this.

  In addition, the image processing apparatus according to the fourth embodiment processes an image periodically photographed by an imaging element such as an ITV camera in which the installation state such as the declination α and the depression angle β is installed under a predetermined condition to be two-dimensional. The displacement state in the three-dimensional space of the object to be imaged is determined from the captured image, which is a flat surface, and the traffic flow is measured by measuring the speed etc. from the change of the position of the moving object such as the vehicle traveling on the road surface in the outdoor road or in the tunnel. In an apparatus for measuring and using the position information and speed information for determination of road traffic events, an imaging means for periodically taking an image by the imaging element, and a curve extracted based on the image taken by the imaging means are drawn. The parallel alignment is derived on the basis of the two-dimensional movement history of different parts of the moving object that moves and the vanishing point information is obtained from the reduction rate of the mutual interval in the linear extension direction of the derived parallel alignment. Vanishing point acquiring means and installation information acquiring means for acquiring installation information of the image sensor based on the vanishing point information acquired by the vanishing point acquiring means. Accurate image processing can be performed without complicated operations such as obtaining and running a test vehicle.

  Furthermore, the image processing apparatus according to the fourth embodiment processes an image periodically photographed by an imaging element such as an ITV camera in which the installation state such as the declination α and the depression angle β is installed under a predetermined condition, and performs two-dimensional processing. The displacement state in the three-dimensional space of the object to be imaged is determined from the captured image, which is a flat surface, and the traffic flow is measured by measuring the speed etc. from the change of the position of the moving object such as the vehicle traveling on the road surface in the outdoor road or in the tunnel. In an apparatus for measuring and using the position information and speed information for determination of road traffic events, an imaging means for periodically taking an image by the imaging element, and a curve extracted based on the image taken by the imaging means are drawn. A parallel alignment is derived based on the two-dimensional movement history of a plurality of moving objects that move in parallel, and vanishing point information is obtained from the reduction rate of the mutual interval in the linear extension direction of the derived parallel alignment. Since it comprises a vanishing point acquisition unit and an installation information acquisition unit that acquires the installation information of the imaging element based on the vanishing point information acquired by the vanishing point acquisition unit, the installation information of the imaging element is acquired from the photographed image. Accurate image processing can be performed without complicated operations such as running the test vehicle.

  In the image processing method according to the fourth embodiment, two-dimensional processing is performed by processing images periodically photographed by an imaging element such as an ITV camera in which the installation conditions such as the declination α and the depression angle β are installed under predetermined conditions. The displacement state in the three-dimensional space of the object to be imaged is determined from the captured image, which is a flat surface, and the traffic flow is measured by measuring the speed etc. from the change of the position of the moving object such as the vehicle traveling on the road surface in the outdoor road or in the tunnel. In an apparatus that measures and uses position information and speed information for determination of road traffic events, etc., an imaging unit that periodically captures an image by the imaging element, and a curved road extracted based on an image captured by the imaging unit Vanishing point acquisition means for deriving parallel alignment based on the shape and acquiring vanishing point information from the reduction rate of the mutual interval in the linear extension direction of the derived parallel alignment; and the vanishing point acquisition means Installation information acquisition means for acquiring the installation information of the image pickup device based on the vanishing point information acquired by the above, so that it is troublesome to acquire the installation information of the image pickup device from the photographed image and run the test vehicle. Accurate image processing can be performed without any work.

  Furthermore, in the image processing method according to the fourth embodiment, an image picked up periodically by an image pickup device such as an ITV camera having an installation condition such as the declination angle α and the depression angle β installed under a predetermined condition is processed. The traffic flow is determined by determining the displacement state of the object to be imaged in the three-dimensional space from the captured image, which is a three-dimensional plane, and measuring the speed etc. from the change in the position of the moving object such as a vehicle traveling on an outdoor road surface or a road surface in a tunnel. The position information and the speed information are used for determination of road traffic events, etc., and are extracted based on an imaging process in which images are periodically taken by the imaging device and images taken by the imaging means. A parallel alignment is derived based on the two-dimensional movement history of different parts of a moving object that draws a curve, and vanishing point information is obtained from the reduction rate of the mutual interval in the linear extension direction of the derived parallel alignment. And the installation information acquisition process of acquiring the installation information of the image sensor based on the vanishing point information acquired by the vanishing point acquisition means. Accurate image processing can be performed without complicated operations such as acquiring information and running a test vehicle.

  In addition, in the image processing method according to the fourth embodiment, an image periodically shot by an imaging element such as an ITV camera in which installation conditions such as declination α and depression angle β are installed under predetermined conditions is processed. The traffic flow is determined by determining the displacement state of the object to be imaged in the three-dimensional space from the captured image, which is a three-dimensional plane, and measuring the speed etc. from the change in the position of the moving object such as a vehicle traveling on an outdoor road surface or a road surface in a tunnel. The position information and the speed information are used for determination of road traffic events, etc., and are extracted based on an imaging process in which images are periodically taken by the imaging device and images taken by the imaging means. A parallel alignment is derived based on the two-dimensional movement history of multiple moving objects that move in a curved line, and vanishing point information is obtained from the reduction rate of the mutual interval in the linear extension direction of the derived parallel alignment. A vanishing point acquisition process, and an installation information acquisition process for acquiring installation information of the image sensor based on the vanishing point information acquired by the vanishing point acquisition means. Accurate image processing can be performed without complicated operations such as obtaining and running a test vehicle.

Embodiment 5 FIG.
A fifth embodiment of the present invention will be described with reference to FIGS. FIG. 10 is a block diagram showing the configuration of the image processing apparatus according to the fifth embodiment. FIG. 11 is a diagram showing a speed distribution of a plurality of vehicles in the fifth embodiment. FIG. 12 is a flowchart showing the processing steps of the parameter correction processing unit in the fifth embodiment. In the fifth embodiment, blocks other than the specific configuration described here have the same configuration contents as the configurations in the first to fourth embodiments described above, and perform the same work. . The same reference numerals in the drawings indicate the same or corresponding parts.

In FIG. 10 showing the configuration of the image processing apparatus in the fifth embodiment, a parameter correction processing unit 15 determines whether or not parameters are corrected from the speed history of a vehicle traveling within the imaging range and performs correction processing.
In the fifth embodiment, the object detection unit 12 extracts a two-dimensional movement history of a moving object such as a vehicle from a captured image from the image capturing unit 10, and the parameter correction processing unit 15 inputs the vehicle input from the object detection unit 12. The two-dimensional movement history is converted into a three-dimensional movement history based on the camera installation information such as the declination α and the depression angle β described in the first to fourth embodiments, and speed and acceleration information is derived. Correction information is acquired based on the speed and acceleration information.
Here, the configuration from the first embodiment to the fourth embodiment is provided, and the camera position return error due to the preset operation of the zoom turning type camera is generated from the installation information calculated once, or the camera is replaced and cleaned externally. Even when an error occurs in the installation information due to a factor, the installation information is corrected from the situation of a plurality of moving vehicles.

In the fifth embodiment, the speed of a general vehicle that is running normally is measured, and the camera installation information is detected and corrected from the speed distribution.
In FIG. 11, the speed distribution of a plurality of passing vehicles is represented, the horizontal axis represents the distance from the camera, and the vertical axis represents the vehicle speed at that distance. When traffic events such as traffic congestion, low speed, stop, and avoidance have not occurred, the speed distribution map changes at a relatively high speed value, and the vehicle travels at a substantially constant speed within a short section monitored by one camera.

The correction processing method in the parameter correction processing unit 15 is shown in FIG. In processing step S41, accelerations Acc (n) of a plurality of vehicles n that measure the speed V1 or higher are calculated. In process step S42, constants AccLow1 and AccHigh1 are defined, and the allowable range of acceleration is set as follows.
AccLow1 <Acc (n) <AccHigh1
This excludes vehicles that are accelerating / decelerating and noise components. In the processing step S43, the vehicle speed distribution existing in the allowable acceleration range in S42 is normalized, an approximate straight line is obtained by the least square sum method, and the average acceleration Acc_ave for the entire passing vehicle is calculated. Further, in processing step S45 for adjusting which installation information of the declination depression angle is executed when it is determined as no in processing step S44 for determining whether the average acceleration is in the allowable acceleration range near the predetermined value 0.0. Acc_ave allowable range constants AccLow2 and AccHigh2 are defined, and the speed change allowable range is set as follows.
AccLow2 <Acc_ave <AccHigh2
Accordingly, it is determined that correction is not necessary when the average acceleration Acc_ave is within the allowable range, and when the average acceleration Acc_ave is out of the allowable range, the lens focal length F and the camera installation height H such that Acc_ave approaches the predetermined value 0.0 are processed. Change and search with.
An allowable area range having a predetermined allowable width is set in the vicinity of the acceleration target value 0.0, and correction information is acquired so that the average acceleration reaches the allowable range and approaches the predetermined value 0.0. .

  As described above, in the fifth embodiment, it is possible to calculate an average acceleration from a vehicle that does not require a plurality of true speed values traveling at a constant speed, and to correct the set parameters.

  The image processing apparatus according to the fifth embodiment processes images periodically photographed by an imaging element such as an ITV camera in which installation conditions such as the declination α, the depression angle β, and the focal length F are installed under predetermined conditions. Traffic is measured by identifying the displacement state of the object to be imaged in a three-dimensional space from a two-dimensional image and measuring the speed from the position of a moving object such as a vehicle traveling on an outdoor road surface or a road surface in a tunnel. In an apparatus that measures a flow and uses position information and speed information for determination of road traffic events, etc., an imaging unit that periodically captures an image by the imaging element, and a parallel that is extracted based on a captured image by the imaging unit Vanishing point acquisition means for acquiring vanishing point information from the reduction rate of the mutual interval in the linear linear extending direction, and the imaging based on the vanishing point information acquired by the vanishing point acquisition means. Installation information acquisition means for acquiring installation information such as declination α and depression angle β related to the image element, and an average acceleration of the vehicle as the object to be imaged in the captured image of the object to be imaged approaches a predetermined value 0.0. Adjusting the installation information of the imaging element acquired by the installation information acquisition means, that is, based on the three-dimensional movement state of the imaged object obtained from the two-dimensional movement state in the captured image of the imaged object Since it has correction information acquisition means for adjusting the installation information of the image sensor and acquiring the correction information, it is possible to acquire the installation information of the image sensor from the photographed image and accurately perform without complicated work such as running the test vehicle. Appropriate image sensor installation information consisting of camera installation information such as camera height H and focal length F from the average acceleration of a vehicle traveling in a steady state. A value can be obtained and optimized image processing can be performed.

  Further, in the image processing method according to the fifth embodiment, an image picked up periodically by an imaging element such as an ITV camera in which installation conditions such as the declination α and the depression angle β are installed under predetermined conditions is processed to be two-dimensional. The displacement state in the three-dimensional space of the object to be imaged is determined from the captured image, which is a flat surface, and the traffic flow is measured by measuring the speed etc. from the change of the position of the moving object such as the vehicle traveling on the road surface in the outdoor road or in the tunnel. A method of measuring and using the position information and speed information for determination of road traffic events, etc., which is extracted based on an imaging process in which images are periodically taken by the imaging device and images taken in the imaging process The vanishing point acquisition process for acquiring vanishing point information from the reduction rate of the mutual interval in the linear extension direction of the parallel linear, and the image sensor based on the vanishing point information acquired by the vanishing point acquisition process. The installation information acquisition process for acquiring installation information such as the declination angle α and the depression angle β, and the installation so that the average acceleration of the vehicle as the imaged object in the captured image of the imaged object approaches a predetermined value 0.0 The installation information of the imaging element acquired by the information acquisition unit is adjusted, that is, based on the three-dimensional movement state of the imaging object obtained from the two-dimensional movement state in the captured image of the imaging object. It includes a correction information acquisition process that adjusts the installation information and acquires correction information, so that accurate image processing can be performed without complicated operations such as acquiring the installation information of the image sensor from the captured image and running the test vehicle. And obtaining an appropriate value of image sensor installation information including camera installation information such as camera height H and focal length F from the average acceleration of a vehicle traveling in a steady state. And can perform an appropriate image processing.

Embodiment 6 FIG.
A sixth embodiment of the present invention will be described with reference to FIG. FIG. 13 is a block diagram showing the configuration of the image processing apparatus according to the sixth embodiment. In the sixth embodiment, blocks other than the specific configuration described here have the same configuration contents as the configurations in the first to fifth embodiments described above, and perform the same operations. The same reference numerals in the drawings indicate the same or corresponding parts.

  In FIG. 13 illustrating the configuration of the image processing apparatus according to the sixth embodiment, a shake correction determination unit 17 determines whether a temporary shake has occurred in the camera or the installation conditions have been changed. The data determination unit 18 determines whether to use the correction image output from the shake correction determination unit 17 or the correction parameter output from the camera information measurement unit 13.

A processing procedure of the shake correction determination unit 17 will be described. The shake correction determination unit 17 registers three or more points with reference to a specific region with many edge components from the captured image when setting the camera installation information. A rectangular image including these reference points is extracted from the captured image and registered as a template.
The registered template image and the input image from the image capturing unit 10 are subjected to matching processing in units of image capturing periods with reference points as references, and the shift of the images at each execution time is monitored to determine the amount of movement at the reference points. It is detected as a time error.
Reference point coordinates are monitored in time series by matching processing, and the following (a), (b), and (c) are discriminated. When the following (b) is discriminated, installation information of a camera as an image sensor is determined. A correction process is performed for.
(A) When the movement amount as the time error is continued at 0, the angle of view is not changed.
(B) If the amount of movement in a certain direction as a time error is detected as a fixed value, the angle of view is changed.
(C) When the movement amount indicating the time-dependent error constantly changes reciprocally periodically, camera shake is occurring.
If it is determined as (c) above, it is determined that the camera shake has temporarily occurred, the captured image is translated so that the amount of movement from the original reference point becomes zero, and the original image and the angle of view are obtained. By correcting so that the conditions match, the measurement error at the time of camera shake is reduced.
When the shake correction determination unit 17 determines (b) above, the shake correction determination unit 17 performs a correction process on the installation information of the camera as the image sensor, and outputs an image output having the same parameters as the parameter output of the camera information measurement unit 13 as data. The data is output to the determination unit 18, and the parameter output of the camera information measurement unit 13 is output from the data determination unit 18 as utilization data. The contents of the implementation in the third and fifth embodiments described above can be applied to the correction processing related to the installation information here.
When it is determined as (c) above, the image corrected as described above is output to the data determination unit 18, and the data determination unit 18 uses the corrected image data from the shake correction determination unit 17 as the camera information measurement. It is determined that the parameter output from the unit 13 is different, and the corrected image data from the correction determination unit 17 is output as utilization data to perform correction processing.

  In this method, a means for distinguishing and correcting camera shake caused by wind and vibration and field angle deviation caused by condition change is shown. With regard to camera shake, it is possible to absorb the influence of fluctuation only by parallel movement of the captured image, so that a flexible system can be constructed by combining this method.

  As described above, in the sixth embodiment, it is possible to effectively select the correction of the installation condition and the correction of the captured image by determining whether the camera installation condition has been changed or the camera shake has occurred. This makes it possible to make the system more flexible, leading to improved measurement accuracy under all conditions.

  The image processing apparatus according to the sixth embodiment is the image processing apparatus according to any one of the first to fifth embodiments, in which the reference image information for extracting and storing the template image as the reference image information from the captured image of the imaged object is stored. Since it comprises an extraction means and a temporal error detection means for detecting a temporal error by comparing a captured image of the object to be imaged with a template image as reference image information, and adjusting and correcting the installation information of the imaging device. It is possible to perform accurate image processing without acquiring complicated information such as acquiring installation information of a camera as an image pickup device from an image and running a test vehicle, and the camera height H, focal length F, and the like. It is possible to obtain appropriate values of image sensor installation information consisting of camera installation information, and to perform optimized image processing. It is possible to really fix.

  Further, in the image processing method according to the sixth embodiment, the image processing apparatus according to the first to fifth embodiments extracts a template image as reference image information from the captured image of the imaged object, and stores the reference image information. Since it includes an image information extraction process and a time-lapse error detection process that detects a time-lapse error by comparing a captured image of the imaged object and a template image as reference image information, and adjusts and corrects the installation information of the image sensor. It is possible to perform accurate image processing without acquiring complicated information such as acquiring installation information of a camera as an image pickup element from a photographed image and running a test vehicle, as well as camera height H and focal length F, etc. It is possible to obtain an appropriate value of the image sensor installation information composed of the camera installation information of the above, and to perform an appropriate image processing, and it is possible to obtain an error over time. It is possible to reliably fix the.

  Furthermore, in the image processing method according to the sixth embodiment, the installation information correction process is performed when the amount of movement in one direction as the time error is detected as a fixed value in the time error detection process in the method of the preceding paragraph. If the amount of movement that indicates a time-dependent error is constantly reciprocating periodically, it is determined that the camera shakes as an image sensor, and the captured image is translated in order to reduce the effects of camera shake. Since the correction is performed so that the angle of view condition matches the original image in the template image, accurate information can be obtained without complicated operations such as obtaining the installation information of the image sensor from the captured image and running the test vehicle. Image processing can be performed, and appropriate values of image sensor installation information including camera installation information such as camera height H and focal length F can be obtained and optimized. Be one capable of performing image processing, moreover, those which can be implemented accurately corrected over time errors.

  Note that the present invention can be partly or wholly combined with each other within the scope of the invention, or can be appropriately modified or omitted.

  DESCRIPTION OF SYMBOLS 10 Image imaging part (imaging means), 11 Vanishing point calculation part (vanishing point acquisition means), 12 Object detection part, 13 Camera information measurement part, 13A Camera information rough measurement part (installation information acquisition means), 13B Camera information correction part (Correction information acquisition means), 14 curve road disappearance calculation section, 15 parameter correction processing section (correction information acquisition means), 16 multilane disappearance calculation section (vanishing point acquisition means), 17 shake correction determination section, 18 data determination section.

Claims (19)

In an apparatus for processing an image periodically photographed by an imaging device installed under a predetermined condition to identify a displacement state in a three-dimensional space of an object to be imaged from a photographed image that is a two-dimensional plane, the image is picked up by the imaging device. An imaging unit that periodically shoots, a vanishing point acquisition unit that acquires vanishing point information from a reduction rate of a mutual interval in a linearly extending direction of a parallel line extracted based on a captured image by the imaging unit, and the vanishing point Installation information acquisition means for acquiring installation information of the imaging element based on the vanishing point information acquired by the acquisition means, and three-dimensional of the imaged object obtained from a two-dimensional movement state in a captured image of the imaged object Correction information acquisition means for adjusting the installation information of the image sensor based on the moving state and acquiring correction information, and the correction information acquisition means is a captured image of the object to be imaged. The image processing apparatus characterized by three-dimensional movement ratio obtained by converting the two-dimensional movement history to the three-dimensional movement history to obtain the adjusted correction information the installation information of the imaging element to approach a predetermined value in the. In an apparatus for processing an image periodically photographed by an imaging device installed under a predetermined condition to identify a displacement state in a three-dimensional space of an object to be imaged from a photographed image that is a two-dimensional plane, the image is picked up by the imaging device. An imaging unit that periodically shoots, a vanishing point acquisition unit that acquires vanishing point information from a reduction rate of a mutual interval in a linearly extending direction of a parallel line extracted based on a captured image by the imaging unit, and the vanishing point Installation information acquisition means for acquiring installation information of the imaging element based on the vanishing point information acquired by the acquisition means, and three-dimensional of the imaged object obtained from a two-dimensional movement state in a captured image of the imaged object Correction information acquisition means for adjusting the installation information of the image sensor based on the moving state and acquiring correction information, and the correction information acquisition means is a captured image of the object to be imaged. The image processing apparatus characterized by average acceleration of the vehicle as the object to be imaged to obtain the adjusted correction information the installation information of the imaging element to approach a predetermined value in the.   The vanishing point acquisition unit extracts the parallel line shape based on a road shape in a captured image by the image sensor, and acquires vanishing point information from the extracted parallel line shape. 2. The image processing apparatus according to 2.   The vanishing point acquisition means extracts the parallel alignment based on the two-dimensional movement history of different parts in the moving object extracted based on the image captured by the image sensor, and the vanishing point information is extracted from the extracted parallel alignment. The image processing apparatus according to claim 1, wherein the image processing apparatus acquires the image processing apparatus.   3. The image processing apparatus according to claim 1, wherein the vanishing point acquisition unit acquires the vanishing point information by extracting the parallel alignment based on a two-dimensional movement history related to a plurality of moving objects. .   The image processing apparatus according to claim 3, wherein the vanishing point acquisition unit extracts the parallel alignment based on a curved road shape in an image captured by the image sensor.   The image processing apparatus according to claim 4, wherein the vanishing point acquisition unit extracts the parallel alignment based on a two-dimensional movement history of different parts of a moving object that moves along a curve.   The image processing apparatus according to claim 5, wherein the vanishing point acquisition unit extracts the parallel alignment based on a two-dimensional movement history related to a plurality of moving objects that move along a curve. Reference image information extracting means for extracting and storing reference image information from a photographed image of the imaged object, error detection means for comparing the photographed image information of the imaged object and reference image information, and detecting an error over time, and The image according to any one of claims 1 to 8, further comprising an installation information correction unit that adjusts and corrects the installation information of the image pickup device in accordance with a temporal error detected by the error detection unit. Processing equipment. A method of processing an image periodically photographed by an image sensor installed under a predetermined condition to identify a displacement state of a subject to be imaged in a three-dimensional space from a photographed image that is a two-dimensional plane. An imaging process for periodically capturing images, and a vanishing point acquisition process for acquiring vanishing point information from the reduction rate of the mutual interval in the linear extension direction of the parallel linear lines extracted based on the captured image in the imaging process, An installation information acquisition process for acquiring installation information of the imaging element based on the vanishing point information acquired in the vanishing point acquisition process, and the imaged object obtained from a two-dimensional movement state in a captured image of the imaged object A correction information acquisition process that adjusts the installation information of the image sensor acquired in the installation information acquisition process based on the three-dimensional movement state of the image sensor and acquires correction information. Indicates the installation information of the imaging element acquired in the installation information acquisition process so that the three-dimensional movement ratio obtained by converting the two-dimensional movement history in the captured image of the imaged object into a three-dimensional movement history approaches a predetermined value. An image processing method comprising adjusting and acquiring correction information . A method of processing an image periodically photographed by an image sensor installed under a predetermined condition to identify a displacement state of a subject to be imaged in a three-dimensional space from a photographed image that is a two-dimensional plane. An imaging process for periodically capturing images, and a vanishing point acquisition process for acquiring vanishing point information from the reduction rate of the mutual interval in the linear extension direction of the parallel linear lines extracted based on the captured image in the imaging process, An installation information acquisition process for acquiring installation information of the imaging element based on the vanishing point information acquired in the vanishing point acquisition process, and the imaged object obtained from a two-dimensional movement state in a captured image of the imaged object A correction information acquisition process that adjusts the installation information of the image sensor acquired in the installation information acquisition process based on the three-dimensional movement state of the image sensor and acquires correction information. Characterized by obtaining the adjusted correction information the installation information of the image sensor average acceleration of the vehicle as the object to be imaged is acquired at the installation information acquisition step to a predetermined value in the captured image of the object to be imaged is An image processing method. And in the vanishing point obtaining process, according to claim 10 or claim, wherein the parallel linear and extraction, to obtain the vanishing point information from the extracted the parallel linear based on the road shape in the image captured by the imaging device The image processing method according to 11 . In the vanishing point acquisition process, the parallel alignment is extracted based on the two-dimensional movement history of different parts in the moving object extracted based on the image captured by the image sensor, and the vanishing point information is extracted from the extracted parallel alignment. the image processing method according to claim 10 or claim 11, characterized in that to obtain. And in the vanishing point obtaining step, image processing method according to claim 10 or claim 11, characterized in that to obtain the vanishing point information extracting said parallel linear on the basis of a two-dimensional movement history according to the plurality of moving objects . The image processing method according to claim 12 , wherein in the vanishing point acquisition process, the parallel alignment is extracted based on a curved road shape in a captured image by the image sensor. The image processing method according to claim 13 , wherein in the vanishing point acquisition process, the parallel alignment is extracted based on a two-dimensional movement history of different parts of a moving object that moves along a curve. The image processing method according to claim 14 , wherein in the vanishing point acquisition process, the parallel alignment is extracted based on a two-dimensional movement history related to a plurality of moving objects that move along a curve.   Reference image information extraction process for extracting reference image information from a captured image of the imaged object and storing it, and comparing the captured image of the imaged object with the reference image information to detect a time-dependent error and install information of the image sensor The image processing method according to claim 10, further comprising a time-dependent error detection process for adjusting and correcting the error.   The step of detecting error with time includes detecting whether the error with time is a fixed value or a periodic change value, and performing an installation information correction process when the error with time is a fixed value. The image processing method according to claim 18.

Images