JP2021018068A - Position measurement data generating apparatus, position measuring apparatus, position measurement data generating method, and position measurement data generating program - Google Patents

Abstract

To prevent a reduction in measuring accuracy of a position on a ground surface being not flat.SOLUTION: A position measurement data generating apparatus comprises: an acquiring section 64 that acquires a captured image in which a measurement vehicle is imaged by a camera installed at a position where a road travelled by a measurement vehicle comprising a measuring section measuring a three-dimensional position can be imaged, and a three-dimensional position of the measurement vehicle which is measured by the measuring section when the measurement vehicle travels on the road; a detecting section 66 that detects an image position of the measurement vehicle from the captured image; an association information generating section 68 that generates association information in which the image position is associated with the three-dimensional position; and a vertex information generating section 70 that generates vertex information in which the image position detected by the detection section 66 is associated with the three-dimensional position representing a vertex constituting a triangle in a coordinate system of the three-dimensional position.SELECTED DRAWING: Figure 4

Description

The present disclosure relates to a position measurement data generation device, a position measurement device, a position measurement data generation method, and a position measurement data generation program.

Patent Document 1 describes a world coordinate system (ground coordinate system) based on a camera installation position in a method of capturing an image in which the camera line of sight is inclined with respect to a measurement area on the ground and measuring the position of an object recognized by image processing. Assuming an intermediate coordinate plane that two-dimensionally connects between the image coordinate system and the image coordinate system corresponding to the image pickup surface of the camera, and a geometry that defines the positional relationship between the world coordinate system and the image coordinate system with respect to the intermediate coordinate surface. A position measurement method by image processing is disclosed, which comprises creating a scholarly model and using this model to obtain a position on the ground from a position on an image of the object.

Japanese Unexamined Patent Publication No. 10-332334

However, in the technique described in Patent Document 1, since it is assumed that the ground surface, which is the shooting range of the camera, is horizontal, the position of the ground surface that is not flat such as undulating terrain or slopes is measured. There was a problem that the accuracy might decrease.

The present disclosure provides a position measurement data generation device, a position measurement device, a position measurement data generation method, and a position measurement data generation program that can suppress a decrease in position measurement accuracy on an uneven ground surface. The purpose is to provide.

In the position measurement data generation device according to the first aspect of the present disclosure, the measurement vehicle is photographed by a camera installed at a position capable of photographing the road on which the measurement vehicle is equipped with a measurement unit for measuring a three-dimensional position. The acquisition unit (64) that acquires the photographed image and the three-dimensional position of the measurement vehicle measured by the measurement unit when the measurement vehicle travels on the road, and the measurement vehicle from the photographed image. A detection unit (66) that detects an image position, a mapping information generation unit (68) that generates mapping information that associates the image position with the three-dimensional position, and the image detected by the detection unit. A vertex information generation unit (70) that generates vertex information in which a position is associated with the three-dimensional position representing a vertex forming a triangle in the coordinate system of the three-dimensional position is provided.

The position measuring device according to the second aspect of the present disclosure is on the captured image based on the association information and the vertex information generated by the position measurement data generating device according to any one of the first to fourth aspects. A conversion unit (62) for converting an arbitrary image position in the above into the three-dimensional position is provided.

In the position measurement data generation method according to the third aspect of the present disclosure, a computer (30A) uses a camera installed at a position where a measurement vehicle equipped with a measurement unit for measuring a three-dimensional position can photograph a road. The photographed image taken by the measuring vehicle and the three-dimensional position of the measuring vehicle measured by the measuring unit when the measuring vehicle travels on the road are acquired, and the measured image of the measuring vehicle is obtained from the photographed image. The image position is detected, association information for associating the image position with the three-dimensional position is generated, and the detected image position represents a vertex forming a triangle in the coordinate system of the three-dimensional position. Generate vertex information associated with a three-dimensional position.

The position measurement data generation program (40) according to the fourth aspect of the present disclosure uses a camera installed in a computer at a position where a measurement vehicle equipped with a measurement unit for measuring a three-dimensional position can photograph a road. The photographed image taken by the measuring vehicle and the three-dimensional position of the measuring vehicle measured by the measuring unit when the measuring vehicle travels on the road are acquired, and the measured image of the measuring vehicle is obtained from the photographed image. The image position is detected, association information for associating the image position with the three-dimensional position is generated, and the detected image position represents a vertex forming a triangle in the coordinate system of the three-dimensional position. It is a position measurement data generation program for executing a process including generating vertex information associated with a three-dimensional position.

According to the present disclosure, there is an effect that it is possible to suppress a decrease in measurement accuracy of a position on an uneven ground surface.

It is a block diagram of a position measurement system. It is a figure for demonstrating the road and the world coordinate system photographed by a camera. It is a block diagram which shows the hardware composition of a position measuring apparatus. It is a functional block diagram of the CPU of the position measuring apparatus. It is a flowchart of the data generation process for position measurement. It is a figure for demonstrating the image position of the measurement vehicle detected from the photographed image. It is a figure for demonstrating the correspondence between the image position of a measuring vehicle, and the three-dimensional position of a measuring vehicle. It is a figure for demonstrating the generation of vertex information. It is a figure for demonstrating the selection of the vertices of a triangle. It is a figure for demonstrating the case of converting an arbitrary point of an image coordinate system into an arbitrary point of a world coordinate system. It is a figure for demonstrating the error which occurs in the coordinate transformation by linear approximation. It is a figure for demonstrating the error which occurs in the coordinate transformation by linear approximation. It is a figure for demonstrating the case where the depth direction seen from the camera is along the lane width direction. It is a figure for demonstrating the coordinate transformation. It is a figure for demonstrating the coordinate transformation. It is a flowchart of the data generation process for position measurement. It is a figure for demonstrating the correction of an image position. It is a figure for demonstrating addition of a vertex. It is a figure for demonstrating the selection of the vertices of a triangle. It is a figure for demonstrating the case which enlarges a triangle. It is a figure for demonstrating the accuracy of the coordinate transformation near a vertex. It is a figure for demonstrating the accuracy of the coordinate transformation near a vertex.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.

FIG. 1 is a configuration diagram of a position measurement system 1 according to the present embodiment. The position measurement system 1 has a configuration in which a position measurement device 10, a photographing device 14, and an in-vehicle device 12 are connected via a network 16.

The in-vehicle device 12 is mounted on the measurement vehicle S as shown in FIG. 2, for example. As shown in FIG. 1, the in-vehicle device 12 includes a measuring unit 18 for measuring a three-dimensional position. The measuring unit 18 measures, for example, the three-dimensional position of the vehicle-mounted device 12 in the three-dimensional space, that is, the three-dimensional position of the measuring vehicle S by using GNSS (Global Navigation Satellite System). As shown in FIG. 2, the three-dimensional position in the three-dimensional space, which is the real space, is represented as the coordinates in the world coordinate system in which the x-axis, the y-axis, and the z-axis are orthogonal to each other.

The photographing device 14 includes a camera 20 and an angle of view adjusting device 22. The camera 20 is a monocular camera provided with an image sensor such as a CCD (Charge-coupled devices) sensor or a CMOS (Complementary metal-xide-processor) sensor. As shown in FIG. 2, for example, the camera 20 is installed at a position where the road RD on which the measuring vehicle S travels can be photographed. In the present embodiment, as an example, a case where the road RD is a road having two lanes on each side and a total of four lanes L1 to L4 will be described. Further, in the present embodiment, the road RD refers to a road included in the shooting range of the camera 20.

The camera 20 photographs the road RD on which the measuring vehicle S and other vehicles travel at predetermined imaging intervals. The captured image data of the captured image captured by the camera 20 is transmitted to the position measuring device 10 via the network 16 and stored in the position measuring device 10. Here, the photographed image data is, for example, data in which the photographed image and the photographed time are associated with each other. The shooting interval is set to an interval at which the measuring vehicle S is photographed a plurality of times while traveling on the road RD. Further, the camera 20 may capture the road RD as a moving image. In this case, the shooting interval is defined by the frame rate.

The measuring vehicle S travels on the road RD photographed by the camera 20 in advance, and measures the three-dimensional position in the world coordinate system at a predetermined measurement interval. The conversion accuracy is higher when the measurement interval is shorter than the shooting interval. Therefore, it is preferable that the measurement interval is shorter than the imaging interval.

The three-dimensional position measurement data obtained by measuring the three-dimensional position of the measuring vehicle S is transmitted to the position measuring device 10 via the network 16 and stored in the position measuring device 10. Here, the three-dimensional position measurement data is, for example, the measured three-dimensional position, the measurement time at which the three-dimensional position is measured, the movement vector of the measurement vehicle S at the measurement time, and the movement of the measurement vehicle S at the measurement time. It is the data associated with the speed.

The angle of view adjusting device 22 has a function of adjusting the angle of view of the camera 20 to an arbitrary angle of view based on an instruction from the position measuring device 10. In the present embodiment, the case where the photographing device 14 includes the angle of view adjusting device 22 will be described, but the angle of view of the camera 20 may be fixed and the angle of view adjusting device 22 may be omitted.

Although details will be described later, the position measuring device 10 detects the image position of the measurement vehicle S on the captured image by performing image processing on the captured image captured by the camera 20. Therefore, it is preferable to provide a marker for facilitating the detection of the measurement vehicle S, for example, on the roof portion of the measurement vehicle S. Examples of the marker include, but are not limited to, a specific predetermined pattern such as a chess board pattern.

FIG. 3 is a configuration diagram of the position measuring device 10. The position measuring device 10 is composed of a device including a general computer.

As shown in FIG. 3, the position measuring device 10 includes a controller 30. The controller 30 includes a CPU (Central Processing Unit) 30A, a ROM (Read Only Memory) 30B, a RAM (Random Access Memory) 30C, a non-volatile memory 30D, and an input / output interface (I / O) 30E as an example of a computer. .. The CPU 30A, ROM 30B, RAM 30C, non-volatile memory 30D, and I / O 30E are connected to each other via the bus 30F.

Further, the operation unit 32, the display unit 34, the communication unit 36, and the storage unit 38 are connected to the I / O 30E.

The operation unit 32 includes, for example, a mouse and a keyboard.

The display unit 34 is composed of, for example, a liquid crystal display or the like.

The communication unit 36 is an interface for performing data communication with external devices such as the in-vehicle device 12 and the photographing device 14.

The storage unit 38 is composed of a non-volatile storage device such as a hard disk. The storage unit 38 stores the position measurement data generation program 40, the position measurement program 42, the three-dimensional position measurement data 44, the captured image data 46, the association information 48, the vertex information 50, and the like, which will be described later. The CPU 30A reads and executes the position measurement data generation program 40 and the position measurement program 42 stored in the storage unit 38.

Next, the functional configuration of the CPU 30 of the position measuring device 10 will be described.

As shown in FIG. 4, the CPU 30A functionally includes a position measurement data generation device 60 and a conversion unit 62. The position measurement data generation device 60 includes an acquisition unit 64, a detection unit 66, a correspondence information generation unit 68, and a vertex information generation unit 70.

The acquisition unit 64 includes a photographed image taken by the measurement vehicle S by a camera 20 installed at a position where the road RD on which the measurement vehicle S traveling with the measurement unit 18 for measuring a three-dimensional position can be photographed, and a measurement vehicle. The three-dimensional position of the measurement vehicle S measured by the measurement unit 18 when S travels on the road RD is acquired.

In this embodiment, as an example, the case where the three-dimensional position of the measurement vehicle S is the three-dimensional position measured when traveling in each of the oncoming lanes of the road RD will be described.

The detection unit 66 detects the image position of the measuring vehicle S from the captured image taken by the camera 20.

The association information generation unit 68 generates the association information 48 in which the image position of the measurement vehicle S in the captured image and the three-dimensional position are associated with each other. The generated association information 48 is stored in the storage unit 38.

The vertex information generation unit 70 generates vertex information 50 in which the image position detected by the detection unit 66 is associated with the coordinate system of the three-dimensional position, that is, the three-dimensional position representing the vertices forming the triangle in the world coordinate system. .. The generated vertex information 50 is stored in the storage unit 38.

The conversion unit 62 sets an arbitrary image position on the captured image in three dimensions in the world coordinate system based on the position measurement data including the correspondence information 48 and the vertex information 50 generated by the position measurement data generation device 60. Convert to position. Specifically, the conversion unit 62 calculates the conversion parameter based on the vertex information of the triangle including the image position of the conversion target in the area, and uses the calculated conversion parameter to change the image position of the conversion target to the three-dimensional position. Convert.

Next, the operation of the position measuring device 10 according to the present embodiment will be described. First, the data generation process for position measurement will be described. As shown in FIG. 3, the position measurement data generation program 40 is stored in the storage unit 38. When the CPU 30A reads out and executes the position measurement data generation program 40, the position measurement data generation process shown in FIG. 5 is executed. The position measurement data generation process shown in FIG. 5 is executed when the operator operates the operation unit 32 to instruct the execution of the position measurement data generation process. The process shown in FIG. 5 may be repeatedly and automatically executed every time a predetermined time elapses through the operation of the operator.

The photographed image data 46 taken by the camera 20 when the measuring vehicle S travels on the road RD, and the three-dimensional position measured by the measuring unit 18 of the in-vehicle device 12 when the measuring vehicle S travels on the road RD. It is assumed that the three-dimensional position measurement data 44 is already stored in the storage unit 38. Further, in the present embodiment, for the sake of simplicity, the case where the number of measuring vehicles S is one will be described.

Further, each of the camera 20 and the measuring unit 18 has a time synchronization function, and measures the shooting time of the captured image captured by the camera 20 and the three-dimensional position of the measuring vehicle S measured by the measuring unit 18. It is assumed that the time is synchronized. As the time synchronization method, various known methods can be used, and for example, a method such as NTP (Network Time Protocol) can be used, but the method is not limited thereto.

In step S100, the CPU 30A acquires the road RD by reading the captured image data 46 captured by the camera 20 when the measuring vehicle S travels from the storage unit 38.

In step S102, the CPU 30A detects the image position of the measurement vehicle S from the captured image based on the captured image data. Specifically, first, the vehicle is detected from the captured image by using a method such as pattern matching. Then, for example, when a marker for facilitating the detection of the measurement vehicle S is provided on the roof portion of the measurement vehicle S or the like, the vehicle provided with the marker is extracted as the measurement vehicle from the detected vehicles. .. When the marker is not provided on the measurement vehicle S, the shape image of the front side shape and the rear side shape of the measurement vehicle S is stored in advance in the storage unit 38, and the shape image is extracted from the captured image. The measurement vehicle S may be specified by doing so.

Then, as shown in FIG. 6, the coordinates of the point at the lower left corner of the bounding box BB of the measuring vehicle S detected from the captured image G are set as the image position of the measuring vehicle S. In FIG. 6, the image position of the measurement vehicle S traveling in the lane L1 detected from the captured image G taken at a certain time is represented as p ₁ (u ₁ , v ₁ ). As shown in FIG. 6, the image coordinate system in the captured image G is a two-dimensional coordinate system composed of u-axis and v-axis orthogonal to each other. Further, the origin Om of the image coordinate system is a point in the upper left corner of the captured image G.

Further, since the measurement vehicle S traveling on the road RD is photographed a plurality of times by the camera 20, the measurement vehicle S is detected for each of the plurality of captured images continuously photographed by the measurement vehicle S traveling on the road RD. To do. In the captured image G of FIG. 6, the image position when the measuring vehicle S traveling in the lane L1 detected in the captured image G is detected from the captured image taken a while ago is p ₃ (u ₃ , v _3). ). Further, the image position when the measurement vehicle S is detected from the photographed image taken when the measurement vehicle S travels in the opposite lane L4 is represented as p ₂ (u ₂ , v ₂ ).

By detecting the image positions of the measuring vehicle S from the plurality of captured images in this way, a plurality of image positions of the measuring vehicle S traveling in the lanes L1 and L4 are detected for each of the lanes L1 and L4.

In step S104, the CPU 30A acquires the three-dimensional position measurement data corresponding to each of the plurality of image positions of the measurement vehicle S detected in step S102 by reading from the storage unit 38. Here, the three-dimensional position measurement data corresponding to the image position of the measuring vehicle S is the measurement data of the three-dimensional position measured at the time closest to the shooting time of the captured image in which the image position of the measuring vehicle S is captured. is there.

As a result, for example, as shown in FIG. 7, the three-dimensional position P ₁ (x ₁ , y ₁ , z ₁ ) corresponding to the image position p ₁ (u ₁ , v ₁ ) and the image position p ₂ (u ₂ , v ₁ ) ₂₎ corresponding to three-dimensional position _{P 2 (x 2, y 2} , z 2), image position _{p 3 (u 3, v 3} ) corresponding to three-dimensional position _{P 3 (x 3, y 3} , z 3) Is obtained.

By the way, for example, when the shooting time of the captured image in which the image position p ₁ (u ₁ , v ₁ ) is detected and the measurement time in which the three-dimensional position P ₁ (x ₁ , y ₁ , z ₁ ) is measured coincide with each other. Is not always. Therefore, it is necessary to correct the three-dimensional position of the measurement vehicle S.

Therefore, in step S106, the CPU 30A corrects the three-dimensional position with the shooting time as the reference time. That is, the three-dimensional position of the measurement vehicle S at the reference time is calculated. Specifically, for example, the three-dimensional position at the reference time is calculated based on the movement vector and the moving speed at the measurement time of the measurement vehicle S included in the three-dimensional position measurement data and the difference between the shooting time and the measurement time. To do.

In step S108, the CPU 30A generates association information in which the image position of the measurement vehicle S detected in step S102 and the three-dimensional position corrected in step S106 are associated with each other and stores them in the storage unit 38. For example, in the example of FIG. 7, the image position p ₁ (u ₁ , v ₁ ) is associated with the corrected three-dimensional position P ₁ (x ₁ , y ₁ , z ₁ ). Similarly, associated with the image position _{p 2 (u 2, v 2} ), and three-dimensional position of the corrected _{P 2 (x 2, y 2} , z 2), the. Further, the image position p ₃ (u ₃ , v ₃ ) is associated with the corrected three-dimensional position P ₃ (x ₃ , y ₃ , z ₃ ).

In step S110, the CPU 30A generates vertex information in which the image positions detected in step S102 are associated with the three-dimensional positions representing the vertices forming the triangle in the world coordinate system of the three-dimensional positions, and stores them in the storage unit 38. To do. For example, in the example of FIG. 8, the image positions p ₁ , p ₂ , and p ₃ representing the vertices of the triangle TG in the captured image G are set to the three-dimensional positions P ₁ , P ₂ , P ₂ representing the vertices constituting the triangle TW in the world coordinate system. generating a vertex information associated with P _3.

In reality, a large number of image positions may be detected from the captured image, and how to select the vertices becomes a problem. The method of selecting vertices will be described below. Further, in the following, when the three-dimensional position is not distinguished, it is simply referred to as the three-dimensional position P.

For example, as shown in FIG. 9, a plurality of three-dimensional positions P are measured when the measuring vehicle S travels in the lane L1, and a plurality of tertiary positions P are measured when the measuring vehicle S travels in the oncoming lane L4 of the lane L1. It is assumed that the original position P is detected.

In this case, two adjacent three-dimensional positions Pa and Pb are selected from the plurality of three-dimensional positions P measured along the lane L1. Next, assuming that the perpendicular bisector N of the line segment connecting the two selected three-dimensional positions Pa and Pb is drawn toward the lane L4, the third order of the shortest distance from the vertical bisector N is assumed. Select the original position Pc. Then, the vertex information in which the three-dimensional positions Pa and Pb measured on the lane L1 and the three-dimensional positions Pc measured on the lane L4 are associated with each other as the vertices of a triangle is generated. This vertex information generation process is executed for all of the adjacent three-dimensional positions P measured on the lane L1. Further, the same process is executed for the adjacent three-dimensional positions P measured on the oncoming lane L4.

Next, a process of converting an arbitrary image position on the captured image into a three-dimensional position based on the association information and the vertex information generated by executing the above process will be described.

First, when converting a two-dimensional image position in the image coordinate system to a three-dimensional position in the world coordinate system, the approximation principle of conversion using linear approximation will be described.

As shown in FIG. 10, the three-dimensional position of the three-dimensional world coordinate system C3 corresponding to an arbitrary image position of the two-dimensional image coordinate system C2 is a triangular TG including an arbitrary image position of the image coordinate system C2. It is obtained by coordinate transformation based on the image positions p1 to p3 of the apex and the three-dimensional positions P1 to P3 of the apex of the triangle TW in the world coordinate system C3 associated with the apex p1 to p3 of the triangle TG.

For the sake of simplicity, as shown in FIG. 11, a spatial coordinate system is set so that the XZ plane consisting of the X axis and the Z axis in the world coordinate system is parallel to the optical axis Zcam of the camera 20. In this case, as shown in FIG. 11, when the XZ plane is viewed from the Y-axis direction, the length in the X direction in the world coordinate system per unit pixel in the v direction in the captured image, that is, the depth as seen from the camera 20. The length in the direction depends on the pixel position in the v direction. For example, when the distance from the image position p1 to the image position p3 is divided into four equal parts in the v direction in the image coordinate system C2, the respective distances d1 to d4 are all the same distance. On the other hand, the distances D1 to D4 in the world coordinate system C3 corresponding to the distances d1 to d4 in the image coordinate system C2 are not the same distance. That is, the distance D1 on the short distance side from the camera 20 is the shortest, and the distance D4 on the long distance side from the camera 20 is the longest. As described above, the length in the depth direction when viewed from the camera 20 in the image coordinate system C2 and the length in the depth direction when viewed from the camera 20 in the world coordinate system C3 have a non-linear relationship. That is, the triangle in the image coordinate system C2 and the triangle in the world coordinate system C3 when the XZ plane is viewed from the Y-axis direction in the world coordinate system C3 are not similar to each other. Therefore, when the coordinates of an arbitrary point in the image coordinate system C2 are converted into the coordinates in the world coordinate system C3, if the coordinates are converted by linear approximation, an error in the depth direction occurs when viewed from the camera 20.

On the other hand, as shown in FIG. 12, when the XY plane is viewed from the Z-axis direction, the length in the Y-axis direction in the world coordinate system C3 per unit pixel in the u direction in the image coordinate system C2 is in the u direction. It does not differ depending on the pixel position and is the same. For example, when the distance from the image position p1 to the image position p3 is divided into two equal parts in the u direction in the image coordinate system C2, the respective distances d1 and d2 are the same distance. Further, the distances D1 and D2 in the world coordinate system C3 corresponding to the distances d1 and d2 in the image coordinate system C2 are also the same distance. That is, the triangle in the image coordinate system C2 and the triangle in the world coordinate system C3 when the XY plane is viewed from the Z-axis direction are similar. Therefore, no error occurs when the coordinates of an arbitrary point in the image coordinate system C2 are converted into the coordinates of the world coordinate system by linear approximation.

Therefore, if an error in the length in the depth direction from the camera 20 caused by the conversion by linear approximation is acceptable, the coordinates of an arbitrary point in the image coordinate system C2 can be converted into the coordinates of the world coordinate system. For example, as shown in FIG. 6, when the road RD is along the depth direction when viewed from the camera 20, for example, when the optical axis direction of the camera 20 is along the lane width direction A of the road RD as shown in FIG. In many cases, an error in the length in the depth direction when viewed from the camera 20 is acceptable. This is because when the road RD is along the depth direction when viewed from the camera 20, many three-dimensional positions of the measuring vehicle S are measured along the road RD, so that the three-dimensional positions adjacent to each other along the road RD are measured. This is because the distance becomes shorter and the influence of the error becomes smaller. The error becomes smaller as the measurement interval of the three-dimensional position and the shooting interval of the camera 20 are shortened, but whether or not the error can be tolerated depends on the accuracy required by the application that uses the position measurement result of the position measurement device 10. Even if the road RD is along the depth direction when viewed from the camera 20, if the measurement interval of the three-dimensional position or the shooting interval of the camera 20 does not satisfy the accuracy required by the application, the error cannot be tolerated. .. Therefore, it is preferable to adjust the measurement interval and the shooting interval according to the accuracy required by the application.

Hereinafter, as shown in FIG. 6, a case where the road RD is along the depth direction when viewed from the camera 20 will be described first. On the other hand, as shown in FIG. 13, when the optical axis direction of the camera 20 is along the lane width direction A of the road RD, for example, as shown in FIG. 6, the road RD is along the depth direction when viewed from the camera 20. Compared with the case, an error in the length in the depth direction when viewed from the camera 20 is often unacceptable. This is because when the optical axis direction of the camera 20 is along the lane width direction A of the road RD, the three-dimensional position of the measurement vehicle S measured along the lane width direction A of the road RD is small, so that it is in the lane width direction. This is because the distance between adjacent three-dimensional positions becomes longer and the influence of error becomes larger. Therefore, it is necessary to generate vertex information with the position where the lane of the road RD and the triangle intersect as a new vertex, and the details of this process will be described later.

Hereinafter, a specific conversion method will be described.

For example, as shown in FIG. 14, any image position within the triangle TG in the image coordinate system _{C2 p i (u i, v} i) a three-dimensional position in the global coordinate system C3 shown in FIG. 15 _P i _(x i, A case of converting to y _i , z _i ) will be described.

When approximating the above-described principle holds, any image position within the triangle TG in the image coordinate system _{C2 p i (u i, v} i) are two-dimensional in the world coordinate system C3 corresponding to the triangle TG in the image coordinate system C2 It is considered that it can be converted into a point in the triangle TW of.

Therefore, the image position p is based on the image positions p ₁ (u ₁ , v ₁ ), p ₂ (u ₂ , v ₂ ), and p ₃ (u ₃ , v ₃ ), which are the vertices of the triangle TG in the image coordinate system C2. _{i (u i, v i)} first a calculation expression for calculating the a vertex of the two-dimensional triangular TW in the world coordinate system C3 corresponding to the triangle TG in the image coordinate system C2 three-dimensional position _P 1 _{(x 1} , Y ₁ , z ₁ ), P ₂ (x ₂ , y ₂ , z ₂ ), P ₃ (x ₃ , y ₃ , z ₃ ) based on the three-dimensional position _Pi (x _i , y _i , z _i) ) Is the same as the second calculation formula. That is, both calculation formulas have the same coefficient and differ only in the dimension of the variable.

First, the first calculation formula in the image coordinate system C2 is expressed by the following formula.

p _i = p ₁ + α _i (p _2- p ₁ ) + β _i (p _3- p ₁ )
= {1- (α _i + β _i )} p ₁ + α _i × p ₂ + β _i × p ₃ ... (1)

However, α _i + β _i ≤ 1, 0 ≤ α _i , 0 ≤ β _i ... (2)

In the above equation (1), p _i , p ₁ , p ₂ , and p ₃ represent vectors.

The conversion parameters α _i and β _i are calculated from the above equations (1) and (2).

Then, the second calculation formula in the world coordinate system C3 is expressed by the following formula in the same manner as the first calculation formula.

P _i = {1- (α _i + β _i )} P ₁ + α _i × P ₂ + β _i × P ₃ ... (3)

In the above equation (3), _Pi , P ₁ , P ₂ , and P ₃ represent vectors.

The three-dimensional position _Pi is calculated by substituting the conversion parameters α _i and β _i calculated from the above equations (1) and (2) into the above equation (3).

Next, a position measurement process for converting an arbitrary image position on a captured image into a three-dimensional position using the above conversion method will be described with reference to the flowchart shown in FIG.

The position measurement process shown in FIG. 16 is executed when the operator instructs the execution of the position measurement process.

First, in step S200, the CPU 30A causes the display unit 34 to display the captured image captured by the camera 20.

In step S202, CPU 30A receives an designation of image position _{p i} of the converted by the operator. The operator, the image position to be converted by operating the operating unit 32 p _i, to specify a vehicle or the like which is traveling, for example, road.

In step S204, CPU 30A converts the designated image position _{p i} on the three-dimensional position _{P i} in the world coordinate system. Specifically, with reference to the vertex information stored in the storage unit 38, extracts the vertex information for the vertex of the triangle that contains the image position p _i specified in step S202. The image positions of the vertices represented by the extracted vertex information are p ₁ , p ₂ , and p ₃ .

Next, referring to the association information stored in the storage unit 38, the three-dimensional positions P ₁ , P ₂ corresponding to the image positions p ₁ , p ₂ , and p ₃ of the three vertices represented by the detected vertex information. , P ₃ are extracted respectively.

Then, the image positions p ₁ , p ₂ , and p ₃ are substituted into the above equation (1), and the conversion parameters α _i and β _i are calculated by the above equations (1) and (2). Next, by substituting the calculated conversion parameters α _i and β _i into the extracted three-dimensional positions P ₁ , P ₂ and P ₃ into the above equation (3), the image position p _i specified in step S202 and calculates the three-dimensional position P _i that corresponds to.

In step S206, CPU 30A causes display of the three-dimensional position _{P i} calculated in step S204 on the display unit 34. This makes it possible to know, for example, the three-dimensional position of the vehicle specified by the operator. In the process of FIG. 16, the case where the captured image is displayed and the image position specified by the operator is converted into the three-dimensional position in the world coordinate system has been described, but the present invention is not limited to this. For example, a vehicle is detected from a captured image using a known method such as pattern matching, and a process of converting the detected vehicle image position into a three-dimensional position in the world coordinate system is automatically executed at predetermined time intervals. May be good.

As described above, in the present embodiment, the road photographed by the camera 20 is represented by a set of triangles having the image position of the measurement vehicle S detected from the captured image as the apex, and the image position and the world of the apex in the image coordinate system C2. The three-dimensional position of the measurement vehicle in the coordinate system C3 is associated with it. Then, an arbitrary image position in the triangle in the image coordinate system C2 is converted into a three-dimensional position in the world coordinate system C3. As a result, the accuracy of position detection is reduced even when the ground surface of the position detection target is not flat, as compared with the case where the ground surface, which is the shooting range of the camera, is assumed to be horizontal. Can be suppressed. In addition, since the camera installation position information is not required, the camera installation work becomes easy.

By the way, as described above, when the optical axis direction of the camera 20 is along the lane width direction A of the road RD as shown in FIG. 13, an error in the length in the depth direction when viewed from the camera 20 is unacceptable. In this case, the vertex information is generated with the position where the lane of the road RD and the triangle intersect as a new vertex. Specifically, as shown in FIG. 17, first, the image positions p1 to p3 are corrected so as to be pixels of the surface on which the measuring vehicle S is installed with the road. Further, the three-dimensional positions P1 to P3 are corrected so that the three-dimensional positions P1 to P3 corresponding to the image positions p1 to p3 coincide with the corrected image positions p1 to p3. For example, the installation position information regarding the installation position of the measurement unit 18 and the shape information regarding the shape of the measurement vehicle S are stored in advance in the storage unit 38, and correspond to the image positions p1 to p3 based on the installation position information and the shape information. The three-dimensional positions P1 to P3 are corrected so that the three-dimensional positions P1 to P3 coincide with the corrected image positions p1 to p3. As the shape information, it is preferable to prepare, for example, a three-dimensional model or the like according to the type (sedan, wagon, truck, bus) of the measurement vehicle S.

It also adds vertices on the boundaries of adjacent lanes. Specifically, for example, as shown in FIG. 18, the boundary lines K1 to K3 of the adjacent lanes of the lanes L1 to L4 are detected by using a known method. Here, the coordinate information of the boundary lines K1 to K3 in the world coordinate system C3 is stored in the storage unit 38 in advance. Then, the intersection of each side of the triangle TG whose vertices are the image positions p ₁ , p ₂ , and p ₃ and the boundary lines K1 to K3 is added as a new vertex. Further, based on the coordinate information of the boundary lines K1 to K3 in the world coordinate system C3, the three-dimensional position corresponding to the image position representing the newly added vertex is calculated, and both are associated with each other. When calculating the three-dimensional position corresponding to the image position representing the newly added vertex, it is assumed that the lane width of each lane is known and the boundary line and the traveling path of the measurement vehicle S are parallel. And calculate.

In the example of FIG. 18, the three-dimensional position as a vertex _P 4, _{P 5} is added to the top border K1 between lanes L1 and L2, the three-dimensional position as the vertex on the boundary line K2 of the lane L2 and L3 _P 6, P ₇ is added, three-dimensional position as the vertex on the boundary K3 between lane L3 and L4 _P 8, _{P 9} are added.

Then, the newly added vertices are used as the vertices of a new triangle to generate vertex information. For example, if two 3D positions added on the same boundary are selected as the vertices of the base of the triangle, and there is no 3D position added on the boundary on both sides of this base or a boundary is not present next to it, the measurement vehicle S Select the third vertex from the three-dimensional position on the travel path. At this time, the vertex closer to the perpendicular bisector of the line connecting the two three-dimensional positions added on the same boundary line is selected as the third vertex.

For example, as shown in FIG. 19, for explaining the three-dimensional position P _4, P ₅ the newly added as an example. In this case, if it is assumed that draws a three-dimensional position P _4, P ₅ the connecting perpendicular bisector N of lines lane width direction, next to the three-dimensional position P _4, the boundary line K1 which P ₅ is present Of the newly added three-dimensional positions P ₆ and P ₇ on the matching boundary line K2, the three-dimensional position P ₇ closer to the perpendicular bisector N is selected as the vertex. As for the opposite side of the boundary line K2 relative to the boundary line K1, since the adjacent border does not exist, out of the running vertices P ₁ on the path line _SK1, P _3, closer to the perpendicular bisector N The three-dimensional position P _{1 of} is selected as the vertex. Thus, a triangle T1 having vertices a three-dimensional position _{P 4, P 5, P 7} , the three-dimensional position _{P 4,} the P 5, _{P 1} and triangle T2 having vertices, is newly defined. A new triangle is defined by performing the same processing for the other three-dimensional positions P ₁ , P ₃ , P ₆ , P ₇ , P ₈ , and P ₉ . As a result, the vertices increase along the lane width direction A, so that it is possible to suppress a decrease in conversion accuracy when converting an arbitrary image position in the image coordinate system C2 to a three-dimensional position in the world coordinate system C3. it can.

In the present embodiment, the case where the measuring vehicle S travels in the lane L1 and the lane L4 to measure the three-dimensional position and performs coordinate conversion using the three-dimensional position measurement data has been described. The defined triangle does not cover the shoulder of the road RD. Therefore, as shown in FIG. 20, the vertex information of the enlarged triangle TW2, which is an enlargement of the triangle TW1 to the sidewalk or the building beyond the roadway, may be generated. Specifically, a known method such as edge detection may be used to detect the boundary between the roadway and the sidewalk and the boundary between the roadway and the building, and the vertex information of the enlarged triangle obtained by expanding the triangle to the detected boundary may be generated.

In addition, the accuracy of coordinate transformation improves as the position is closer to the apex of the triangle. For example, as shown in FIG. 21, the neighborhood region R1 of the three-dimensional position P1, the neighborhood region R2 of the three-dimensional position P2, and the neighborhood region R3 of the three-dimensional position P3 have better coordinate conversion accuracy than other regions. Become. Therefore, as shown in FIG. 22, it is preferable to run the measuring vehicle S in all lanes L1 to L4 to measure the three-dimensional position P so that the ratio of coordinate conversion near the apex increases. As a result, the number of regions R near the apex where the accuracy of the coordinate conversion is high increases, and the ratio of the coordinate conversion near the apex increases.

Further, in the present embodiment, the case where the angle of view of the camera 20 is fixed has been described, but the photographed image data obtained by photographing the road RD at a plurality of different angles of view is acquired, and the correspondence information corresponding to various angles of view and the correspondence information The vertex information may be generated.

Further, it is preferable that the measurement vehicle S is driven to acquire the three-dimensional position measurement data even after the correspondence information and the vertex information are once generated, and the correspondence information and the vertex information are updated. As a result, even if the angle of view of the camera 20 is deviated due to the influence of disturbance or the like, it is possible to suppress a decrease in the accuracy of coordinate conversion.

Further, in the present embodiment, the case where the position measurement device 10 includes the position measurement data generation device 60 and the conversion unit 62 has been described, but the position measurement device including the position measurement data generation device 60 and the conversion unit 62 has been described. 10 and may be a separate configuration.

Further, in the present embodiment, the case where the road RD on which the measuring vehicle S travels is a road having a total of four lanes with two lanes on each side has been described, but the road may be a road with a total of two lanes with one lane on each side, while It may be a one-lane road. That is, the number of lanes is not limited.

Although the embodiments have been described above, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various changes or improvements can be made to the above embodiments without departing from the gist of the invention, and the modified or improved forms are also included in the technical scope of the present invention.

Further, the above-described embodiment does not limit the invention according to the claim, and it is said that all combinations of features described in the embodiment are indispensable for the means for solving the invention. Not exclusively. The above-described embodiments include inventions at various stages, and various inventions are extracted by combining a plurality of disclosed constituent requirements. Even if some constituents are deleted from all the constituents shown in the embodiment, a configuration in which some of the constituents are deleted can be extracted as an invention as long as the effect is obtained.

Further, in the above embodiment, the case where the position measurement data generation program 40 and the position measurement program 42 are pre-installed in the storage unit 38 has been described, but the present invention is not limited thereto. For example, the position measurement data generation program 40 and the position measurement program 42 may be stored in a storage medium such as a CD-ROM (Compact Disc Read Only Memory) and provided, or may be provided via a network. Good.

Further, in the above-described embodiment, the case where the position measurement data generation process and the position measurement process are realized by executing a program by a software configuration using a computer has been described, but the present invention is limited thereto. It's not something. For example, the position measurement data generation process and the position measurement process may be realized by a hardware configuration or a combination of a hardware configuration and a software configuration.

In addition, the configuration of the position measuring device 10 (see FIG. 3) described in the above embodiment is an example, and unnecessary parts are deleted or new parts are added within a range that does not deviate from the gist of the present invention. It goes without saying that you can do it.

Further, the processing flow (see FIGS. 5 and 16) of the position measurement data generation program 40 and the position measurement program 42 described in the above embodiment is also an example, and unnecessary steps are taken within a range that does not deviate from the gist of the present invention. Needless to say, you can delete, add new steps, or change the processing order.

62 Conversion unit, 64 Acquisition unit, 66 Detection unit, 68 Correspondence information generation unit, 70 Vertex information generation unit

Claims (11)

A photographed image of the measuring vehicle taken by a camera installed at a position capable of photographing the road on which the measuring vehicle has a measuring unit for measuring a three-dimensional position, and when the measuring vehicle travels on the road. The acquisition unit (64) for acquiring the three-dimensional position of the measurement vehicle measured by the measurement unit, and
A detection unit (66) that detects the image position of the measurement vehicle from the captured image, and
A mapping information generation unit (66) that generates mapping information in which the image position and the three-dimensional position are associated with each other.
A vertex information generation unit (70) that generates vertex information in which the image position detected by the detection unit is associated with the three-dimensional position representing the vertices forming a triangle in the coordinate system of the three-dimensional position.
A position measurement data generator (60). The position measurement data generation device according to claim 1, wherein the three-dimensional position of the measurement vehicle is a three-dimensional position measured when traveling in each of the oncoming lanes of the road. The position measurement data generation device according to claim 1 or 2, wherein the vertex information generation unit generates vertex information of an enlarged triangle obtained by enlarging the triangle. When the optical axis direction of the camera is along the lane width direction of the road, the vertex information generation unit generates vertex information with the three-dimensional position where the lane of the road and the triangle intersect as a new vertex. The position measurement data generation device according to any one of claims 1 to 3. Based on the association information and the vertex information generated by the position measurement data generation device according to any one of claims 1 to 4, an arbitrary image position on the captured image is converted into the three-dimensional position. Conversion unit (62)
A position measuring device (10). The position according to claim 5, wherein the conversion unit calculates a conversion parameter based on the vertex information of a triangle including the image position of the conversion target, and converts the image position of the conversion target into a three-dimensional position using the calculated conversion parameter. Measuring device. The computer (30A)
A photographed image of the measuring vehicle taken by a camera installed at a position capable of photographing the road on which the measuring vehicle has a measuring unit for measuring a three-dimensional position, and when the measuring vehicle travels on the road. Obtaining the three-dimensional position of the measuring vehicle measured by the measuring unit,
The image position of the measurement vehicle is detected from the captured image,
The correspondence information in which the image position and the three-dimensional position are associated with each other is generated.
A position measurement data generation method for generating vertex information in which the detected image position is associated with the three-dimensional position representing the vertices forming a triangle in the coordinate system of the three-dimensional position. The position measurement data generation method according to claim 7, wherein the three-dimensional position of the measurement vehicle is a three-dimensional position measured when traveling in each of the oncoming lanes of the road. The position measurement data generation method according to claim 7 or 8, wherein the vertex information of the enlarged triangle obtained by enlarging the triangle is generated. When the optical axis direction of the camera is along the lane width direction of the road, any one of claims 7 to 9 for generating vertex information with the three-dimensional position where the lane of the road and the triangle intersect as a new vertex. The data generation method for position measurement according to item 1. On the computer
A photographed image of the measuring vehicle taken by a camera installed at a position capable of photographing the road on which the measuring vehicle has a measuring unit for measuring a three-dimensional position, and when the measuring vehicle travels on the road. Obtaining the three-dimensional position of the measuring vehicle measured by the measuring unit,
The image position of the measurement vehicle is detected from the captured image,
The correspondence information in which the image position and the three-dimensional position are associated with each other is generated.
Position measurement data generation for executing a process including generating vertex information in which the detected image position is associated with the three-dimensional position representing the vertices constituting the triangle in the coordinate system of the three-dimensional position. Program (40).