JP2012122911A - Position specification apparatus, position specification method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the accuracy of position specification of an object by stereo vision performed by using a pickup image on a position where a base line length from a reference position cannot be sufficiently secured.SOLUTION: An imaging position specification part 1 specifies the position information of an imaging position based on the position information of a reference position of which the position information is known, a reference position image obtained by imaging an object on the reference position and an imaging position image obtained by imaging the object on an arbitrary imaging position. An object position specification part 2 specifies the position information of the object based on the position information of a known position, a known position image obtained by imaging the object on the known position, the position information of the imaging position, and an imaging position image obtained by imaging the object on the imaging position. The known position is located on a point on which a distance from an imaging position whose position information is specified by the imaging position specification part 1 is longer than the reference position and is a position whose position information is known.

Description

  The embodiments discussed herein relate to object location techniques.

2. Description of the Related Art In recent years, various technologies for providing a driver with a general-purpose image that is difficult to see from a driver or a blind spot have been proposed as technologies for supporting safe driving of a car.
In addition, as a safe driving support technology, by measuring the distance of each object around the host vehicle from the host vehicle, for example, by generating and displaying a three-dimensional image around the host vehicle, There is also a system that provides the surrounding information to the driver without a sense of incongruity. Realization of such a system requires stable distance measurement technology that does not depend on the speed of the host vehicle and low cost.

  As one of distance measurement methods, a method of calculating a distance by the principle of triangulation based on a parallax image obtained by performing stereo viewing is known. For example, as a first technique, with a single imaging device installed in a vehicle, two images with time-series parallax are captured at two points, movement data between the two points is obtained, and both images are obtained. A technique is known in which the distance to a three-dimensional object shown in is determined by the principle of triangulation.

  In addition, while switching between monocular stereo and compound eye stereo, or adjusting both weights according to the surrounding situation (distance to the three-dimensional object or vehicle speed), both are used to compensate for the disadvantages of both. There is also known a method for accurately obtaining the distance to a three-dimensional object. For example, as a second technique, when the vehicle is stationary, only the compound eye stereo is used to determine the distance to the three-dimensional object, while when the vehicle is moving, the results of distance measurement by each of the monocular stereo and the compound eye stereo are integrated. A technique for obtaining a distance to a three-dimensional object is known. Also, for example, as a third technique, in a region where compound eye stereo can be used, the distance to the three-dimensional object is obtained using compound eye stereo, while in a region where compound eye stereo cannot be used, the distance to the three-dimensional object is determined using monocular stereo. The technology of seeking is also known.

JP 2007-263657 A

However, in the first technique described above, since the movement data between two points depends on the speed of the host vehicle, it is difficult to stably measure the distance.
In the second technique described above, since both monocular stereo and compound eye stereo are used, the system cost becomes high.

  In the compound-eye stereo used in the second and third techniques described above, the estimation accuracy of the position of the three-dimensional object depends on the distance (baseline) between the two cameras, and the longer the baseline, the longer the distance estimation. Accuracy is improved. However, considering that two cameras are mounted on the vehicle, it is difficult to lengthen the baseline, so the range of three-dimensional objects that can perform accurate distance measurement is limited to a short distance around the host vehicle. .

Even with a monocular stereo, a sufficient baseline length cannot be ensured immediately after the host vehicle starts moving, so accurate distance measurement cannot be performed.
In view of the above-described problems, the position specifying device described later in this specification improves the accuracy of specifying the position of an object by stereo vision using a captured image at a position where the baseline length from the reference position cannot be sufficiently secured. Let

  One of the position specifying devices described later in this specification includes an imaging position specifying unit and an object position specifying unit. Here, the imaging position specifying unit images the position information of the reference position whose position information is known, the reference position image obtained by imaging the object at the reference position, and the object at an arbitrary imaging position. Based on the imaging position image obtained in this way, position information of the imaging position is specified. Then, the object position specifying unit, the position information of the known position, the known position image obtained by imaging the object at the known position, the position information of the imaging position, and the object at the imaging position. Based on the imaging position image obtained by imaging, the position information of the object is specified. Here, the known position is a position where the distance from the imaging position whose position information is specified by the imaging position specifying unit is larger than the reference position and the position information is known. That is.

  Further, one of the position specifying methods described later in this specification is that the imaging position specifying unit specifies the position information of the imaging position when the imaging position image is obtained by imaging the object at an arbitrary imaging position. Do. The position information of the imaging position is specified by the position information of the reference position where the position information is known, the reference position image obtained by imaging the object at the reference position, and the above-described imaging position image. Based on. Next, position information of a known position, a known position image obtained by imaging the object at the known position, position information of the imaging position, and imaging obtained by imaging the object at the imaging position The target object position specifying unit specifies the position information of the target object based on the position image. Here, the known position is a position where the distance from the imaging position where the imaging position specifying unit specified the position information is larger than the reference position, and the position information is known. That is.

  One of the programs described later in this specification causes a computer to perform the following processing. This process is obtained by first imaging the position information of the reference position whose position information is known, the reference position image obtained by imaging the object at the reference position, and imaging the object at an arbitrary imaging position. Based on the captured image, the position information of the captured position is specified. Then, the position information of the known position, the known position image obtained by imaging the above-described object at the known position, the position information of the imaging position, and the object at the imaging position are imaged. Based on the obtained imaging position image, the position information of the object is specified. Here, the known position is located at a point where the distance from the imaging position where the position information is specified by specifying the position information of the imaging position is larger than the reference position, and the position information is known. It is a certain position.

  According to the position specifying device to be described later in this specification, the effect of improving the accuracy of specifying the position of an object by stereo viewing performed using a captured image at a position where the baseline length from the reference position cannot be sufficiently secured. Play.

It is a functional block diagram of one example of a position specifying device. It is a functional block diagram of another example of a position specifying device. It is an example (the 1) of the structure where the camera is being fixed to the door mirror which can be opened and closed by an actuator. It is an example (the 2) of the structure where the camera is being fixed to the door mirror which can be opened and closed by an actuator. It is explanatory drawing of a 1st picked-up image. It is explanatory drawing of a 2nd picked-up image. It is a figure explaining the operation | movement of the position specification of the target object by the position specification part in the position specification apparatus of FIG. It is a functional block diagram of another another Example of a position specifying device. It is the flowchart which illustrated the processing content of the position specific process. It is explanatory drawing of the flowchart of FIG. It is the flowchart which illustrated the processing content of the image acquisition process. It is the flowchart which illustrated the processing content of the imaging position specific process. It is the flowchart which illustrated the processing content of the selection process. It is the flowchart which illustrated the processing content of position specification and distance calculation processing. It is a hardware structural example of a computer.

First, FIG. 1 will be described. FIG. 1 is a functional block diagram of an embodiment of the position specifying device. The position specifying device includes an imaging position specifying unit 1 and an object position specifying unit 2.
The imaging position specifying unit 1 captures the position information of the reference position where the position information is known, a reference position image obtained by imaging the object at the reference position, and the target object at an arbitrary imaging position. Based on the obtained imaging position image, the position information of the imaging position is specified.

  The target object position specifying unit 2 captures the target object with the position information of the known position, the position information of the known position, the position information of the image capturing position whose position information is specified by the image capturing position specifying unit 1, and the image capturing position. Based on the obtained imaging position image, the position information of the object is specified. The known position here is located at a point where the distance from the imaging position whose position information is specified by the imaging position specifying unit 1 is larger (longer) than the reference position, and the position information is known. Position. The known position image is an image obtained by imaging the above-described object at the known position.

  In this position specifying device, a known position located at a point where the distance from the image pickup position whose position information is specified by the image pickup position specifying unit 1 is larger (longer) than the reference position, and a known position image at the known position, The position information of the object is specified using. In general, the accuracy of specifying the position of an object by stereo vision increases as the distance (baseline) between the imaging positions of two images used for specifying the position becomes longer. Therefore, the accuracy of position specification by the position specifying device of FIG. 1 is improved as compared with the case where the reference position and the reference position image are used.

In addition, as illustrated using a broken line in FIG. 1, the position specifying device may be configured to include the selection unit 3.
The selection unit 3 represents aptitude when using the plurality of known positions as a basis for specifying the position of the object by the object position specifying unit 2 from the plurality of known positions. The one that matches the predetermined condition is selected as the aforementioned selected known position.

  In this case, the object position specifying unit 2 includes the position information of the selected known position selected by the selection unit 3, the selected known position image, the position information of the imaging position described above, and the above-described information. Based on the imaging position image, the position information of the object is specified. The selected known position image is an image obtained by imaging the above-described object at the selected known position selected by the selection unit 3.

  In the selection of the selected known position by the selection unit 3, for example, among the plurality of known positions described above, a predetermined distance order from the imaging position at which the imaging position specifying unit 1 specifies the position information (long order) is predetermined. Try to select a few.

  As described above, the accuracy of specifying the position of the object in stereo view increases as the baseline between the imaging positions of the two images used for specifying the position becomes longer. Therefore, as described above, the selection unit 3 selects a long distance from the above-described imaging position from among a plurality of known positions, whereby the target position specifying unit 2 specifies the target position. Further improvement in accuracy can be expected.

  Further, as illustrated using a broken line in FIG. 1, the position specifying device may be configured to include the recording unit 4. In the recording unit 4, a history of identification results of position information of the object performed in the past is recorded. In this configuration, the selection unit 3 selects a selected known position from a plurality of known positions as follows, for example.

  That is, among the plurality of known positions, identification of past position information recorded in the recording unit 4 from a perpendicular bisector connecting the imaging positions where the imaging position specifying unit 1 specifies the position information is specified. You may make it select a predetermined number in order with which the distance to the position of the target object which concerns on a result becomes short.

  Generally, the accuracy of specifying the position of an object by stereo vision improves as the difference in distance from two imaging positions at which two images used for specifying the position are taken is smaller. That is, the closer the position of the object is to the perpendicular bisector of the line connecting the two imaging positions, the more accurate the position specification of the object. Therefore, when the selection unit 3 selects the selection known position as described above, further improvement in the accuracy of the position identification of the object by the object position identification unit 2 can be expected.

When the selection unit 3 selects a plurality of selected known positions, the object position specifying unit 2 specifies position information about the object as follows, for example.
That is, the object position specifying unit 2 first specifies the position information about the object for each of the plurality of selected known positions selected by the selection unit 3. Then, the center of gravity of a plurality of positions indicated by the plurality of position information obtained by this specification is obtained, and the position information of the center of gravity is used as the result of specifying the position information of the target object. In this way, by using the center of gravity of the plurality of position information specified for each of the plurality of selected known positions as the result of specifying the position information of the object, further improvement in position specifying accuracy can be expected. .

When the selection unit 3 selects a plurality of selected known positions, the object position specifying unit 2 may specify the position information about the object as follows, for example.
That is, the object position specifying unit 2 first specifies position information about the object for each of the plurality of selected known positions selected by the selection unit 3. Next, for each of a plurality of positions indicated by the plurality of position information obtained by this specification, weighting is performed according to the selection order when the selection unit 3 selects the selected known position used for specifying the position information. give. Then, the weighted centroids of a plurality of positions each given this weighting are obtained, and the position information of the weighted centroids is set as the identification result of the position information of the object. In this way, the weighted centroid when the weight according to the selection order of the selected known positions is given to the plurality of position information specified for each of the plurality of selected known positions is the position information of the target object. By making the specific result, further improvement in the accuracy of position specification can be expected.

  Moreover, you may comprise so that the determination part 5 may be provided in this position specific apparatus as illustrated using the broken line in FIG. The determination unit 5 determines the length of a baseline distance that is a distance between the reference position and the imaging position specified by the imaging position specifying unit 1 and a predetermined distance threshold. In this configuration, the object position specifying unit 2 specifies the position information of the object based on, for example, the following.

  First, when the determination unit 5 determines that the baseline distance is shorter than the distance threshold, the position information of the target is specified by the position information of the selected known position selected by the selection unit 3, the selected known position image, and the imaging position. This is performed based on the position information of the imaging position specified by the specifying unit 1 and the imaging position image. On the other hand, when the determination unit 5 determines that the baseline distance is longer than the distance threshold, the position information of the target is specified as the position information of the reference position and the reference position image, and the imaging position specified by the imaging position specifying unit 1. Based on the position information and the captured position image.

  In this configuration, when the above-described baseline distance (that is, the base line) is longer than a distance threshold value that provides a desired position specifying accuracy, the object position specifying unit 2 specifies the position information of the object as a reference. This is performed using the position information of the position and the reference position image.

  In addition, as illustrated using a broken line in FIG. 1, the position specifying device may be configured to further include a single imaging device 6. This imaging device 6 is installed in a moving body such as an automobile, for example, and captures all of the above-described reference position image, imaging position image, and known position image.

  In addition, as illustrated using a broken line in FIG. 1, the position specifying device may be further provided with a distance calculating unit 7. The distance calculation unit 7 calculates the distance from the imaging device 6 to the above-described object, the position information of the object specified by the object position specifying unit 2, and the position information of the image pickup position specified by the image pickup position specifying unit 1. Calculate based on

  Next, with respect to an example in which the position of an object around the automobile is specified using the position specifying device and the distance of the object from the vehicle is determined, an example in which the automobile is stationary, and This will be described separately from the example in the case where the automobile, which has been stationary, has just started moving.

  First, FIG. 2 will be described. FIG. 2 is a functional block diagram of another embodiment of the position specifying device. The position specifying device in FIG. 2 specifies the position of an object around the automobile and calculates the distance of the object from the vehicle when the automobile is stationary.

  2 includes a camera 11, an actuator 12, an actuator control unit 13, an image acquisition unit 14, an image processing unit 15, a feature point tracking result storage memory 16, a position specification unit 17, and a distance calculation unit 18. Yes.

  The camera 11 is an imaging device that captures an image of an object, and the actuator 12 moves the position of the camera 11. The actuator control unit 13 controls the movement operation of the camera 11 by the actuator 12.

Here, the relationship among the camera 11, the actuator 12, and the actuator control unit 13 will be described with reference to FIGS. 3A and 3B.
As illustrated in FIG. 3A, the camera 11 is fixed to a front end portion of the door mirror 19 of the automobile, and images an object on the side of the automobile. The actuator 12 is a motor for driving the automatic opening / closing operation of the door mirror 19, and the actuator controller 13 controls the actuator 12 to open / close the door mirror 19, thereby moving the camera 11. Therefore, the movement path (trajectory) of the camera 11 when the door mirror 19 is opened and closed is known.

  The actuator control unit 13 can detect that the door mirror 19 is completely closed and fully opened, but cannot detect how much the door mirror 19 is open during the opening / closing operation. Further, the position of the camera 11 (relative position and orientation with respect to the vehicle) and the trajectory of the camera 11 when the door mirror 19 is completely closed and fully opened are known, but the position of the camera 11 during the opening / closing operation is known. You cannot know (somewhere on the trajectory).

The camera 11 sequentially repeats imaging while moving by the opening / closing operation of the door mirror 19.
The image acquisition unit 14 acquires images of a plurality of objects captured while the camera 11 is moving.

  The image processing unit 15 performs various types of image processing on the plurality of images acquired by the image acquisition unit 14. More specifically, the image processing unit 15 includes a feature point extraction unit 15a and a feature point tracking unit 15b, and performs so-called block matching processing.

  The feature point extraction unit 15a extracts a partial image region (feature point) from the captured image acquired by the image acquisition unit 14. The feature point extraction unit 15a extracts feature points from an image area in which an object captured by the camera 11 is shown. As a method for extracting the feature points, various well-known methods such as a method using a shape feature and a method using a color feature can be used.

  The feature point tracking unit 15b extracts, from the two captured images acquired by the image acquisition unit 14, the maximum position where the similarity between the feature points extracted by the feature point extraction unit 15a is equal to or greater than a predetermined threshold, and Track the movement of feature points between images.

  The feature point tracking result storage memory 16 stores and stores the captured image acquired by the image acquisition unit 14 and the tracking result of the feature point between the captured images, which is the processing result by the image processing unit 15. It is a storage device.

  The position specifying unit 17 uses the information stored in the feature point tracking result storage memory 16 to specify the position of the target (relative position with respect to the car) captured by the camera 11. The position specifying unit 17 includes a first position specifying unit 17a and a second position specifying unit 17b.

  The 1st position specific | specification part 17a specifies the position of the 1st target object which is one of the target objects which the camera 11 imaged. The position of the first object is obtained by the camera 11 taking an image from each of the positions (two known points) when the door mirror 19 is fully closed and fully opened. This is performed based on two captured images (first captured images) of the first object.

  The second position specifying unit 17b specifies the position of a second object that is one of the objects captured by the camera 11 and is different from the first object. This position is specified by one piece of the second object including the image of the first object obtained from the position of the first object and an arbitrary position on the trajectory of the camera 11. This is performed based on the captured image (second captured image) and the first captured image including the image of the second object.

Here, the first captured image and the second captured image will be described with reference to FIGS. 4A and 4B.
4A and 4B illustrate changes in the field of view of the camera 11 that is moved by opening and closing the door mirror 19.

  In FIG. 4A, [1] is a field of view when the door mirror 19 is completely closed, [2] is a field of view when the door mirror 19 is fully opened, and [3] is on the locus by the opening / closing operation of the door mirror 19. The field of view at a certain position. [4] indicates that the field of view of [1] and the field of view of [2] overlap, and the person H standing on the left side of the vehicle is located at the common part of both. expressing. Therefore, the image of the person H is included in both the image captured by the camera 11 with the field of view [1] and the image captured by the camera 11 with the field of view [2]. In this case, the person H is the first object, and the two captured images obtained by the camera 11 capturing each of [1] and [2] are the first captured images, and the first position specifying unit 17a. Specifies the position of the person H from the two captured images.

  On the other hand, in FIG. 4B, the field of view of the camera 11 in which only the person H is included is changed by the opening operation of the door mirror 19 which has been completely closed until then, and a part of the vehicle V is included in the field of view. It shows the state. In this case, the person H who has specified the position by the first position specifying unit 17a is the first object, the car V is the second object, and the camera 11 views both the person H and the car V. The captured image captured at the position included in the second captured image. The second position specifying unit 17b includes the position of the person H specified by the first position specifying unit 17a, the second captured image, and the image of the car V among the first captured images (see FIG. 4A [2] image), the position of the vehicle V is specified.

  In addition, in the said description, although the case where a 1st target object and a 2nd target object are separate objects is taken as an example, a present Example is not restricted to this. Both the first object and the second object may be feature points or feature areas that can be identified on the image captured by the camera 11. In that sense, the first object and the second object may be different parts of the same object. For example, when a long object such as a track or a wall exists in the imaging range in any of the states [1], [2], and [3] in FIG. 4A on the side of the vehicle V on which the camera 11 is mounted. Alternatively, a part of the object may be the first object, and a part other than the part may be the second object. Further, if there is road surface paint that exists in the imaging range in any of the states [1], [2], and [3] in FIG. 4A on the side of the vehicle V on which the camera 11 is mounted, A part may be used as the first object, and a part other than the part may be used as the second object.

Returning to the description of FIG.
The distance calculation unit 18 detects the first object, the second object, the third object,... Specified by the position specifying unit 17 from the predetermined position of the automobile where the camera 11 is installed. Calculate the distance.

  The position specifying unit 17 in FIG. 2 further includes a third position specifying unit 17c. The third position specifying unit 17c specifies the position of a third object that is one of the objects captured by the camera 11 and is different from the first object and the second object. This specification includes the position of the first object, one captured image (third captured image) of the third object including the image of the first object, and the third of the first captured images. This is based on what also contains the image of the object.

Next, the operation of specifying the position of the object by the position specifying unit 17 will be further described with reference to FIG.
FIG. 5 shows the object A (represented by a circle in FIG. 5), the object B (represented by a square in FIG. 5), and the object C (represented in FIG. 5) while moving the camera 11 by opening and closing the door mirror 19. 5 schematically represents a state of imaging (represented by triangular marks).

  In FIG. 5, P1, P2, and Qi represent the positions of the camera 11. Of these, P1 and P2 are known positions, and are positions of the camera 11 when the door mirror 19 is completely closed and fully opened. Qi is an arbitrary position on the locus between P1 and P2 drawn by the movement of the camera 11 by the opening / closing operation of the door mirror 19.

  Further, the P1 image, the Qi image, and the P2 image are images obtained by the camera 11 capturing images at positions P1, Qi, and P2, respectively. Here, since the object A is located at a place included in both the field of view of the camera 11 at the position P1 and the field of view of the camera 11 at the position P2, any of the P1 image, the Qi image, and the P2 image is included. Also includes an image of the object A. On the other hand, since the object B is out of the field of view of the camera 11 at the position P2, the image of the object B is included in the P1 image and the Qi image, but is not included in the P2 image. Further, since the object C is out of the field of view of the camera 11 at the position P1, the image of the object C is included in the Qi image and the P2 image, but is not included in the P1 image.

In the example of FIG. 5, the first position specifying unit 17 a and the second position specifying unit 17 b included in the position specifying unit 17 operate as follows.
First, the first position specifying unit 17a specifies the position of the object A (first object). The first position specifying unit 17a is two picked-up images (first picked-up images) obtained by the camera 11 picking up images from two known points P1 and P2 whose positions are known. This is performed based on the P1 image and the P2 image. That is, the first position specifying unit 17a uses the first captured image as a parallax image, and the position of the object A (target corresponding to each feature point extracted by the feature point extracting unit 15a by the above-described triangulation principle). By calculating the position of each part of the object A), the position is specified.

  The second position specifying unit 17b specifies the position of an object B (second object) that is different from the object A. The second position specifying unit 17b specifies the position of the Qi image (second picked-up image) that is a picked-up image of the target B including the position of the target A already specified and the image of the target A. ) And the P1 image including the image of the object B in the first captured image. More specifically, the position of the object B by the second position specifying unit 17b is performed as follows.

First, the rotation component of the movement amount of the camera 11 due to the movement from the position P1 to the position Qi is represented as a rotation matrix Ri, and the translation component is represented as a translation column Ti. Further, the position of the object A specified by the first position specifying unit 17a (the coordinates of the feature point m on the object A extracted from the P1 image by the feature point extracting unit 15a) is represented as Xrm. At this time, the position of the object A in the Qi image (the coordinates of the feature point m on the object A in the Qi image, which is the result of tracking by the feature point tracking unit 15b) Xm is expressed by the following equation [1]. The
Xm = Ri (Xrm-Ti) ............ Formula [1]

  For a plurality of feature points on the object A, the equation [1] is established, and the least square method is applied to the obtained plurality of equations to obtain Ri and Ti. Thereby, since the movement amount of the camera 11 by the movement from the known position P1 to the position Qi is obtained, the position Qi can be specified based on the position P1. Thereafter, the second position specifying unit 17b uses the P1 image captured at the known position and the Qi image captured at the specified position as a parallax image, and the position of the object B according to the above-described triangulation principle. To determine the position.

  By the way, the 3rd position specific | specification part 17c specifies the position of the target object C which is different from the target object A and the target object B. FIG. In the example of FIG. 5, the third position specifying unit 17 c specifies the position as Qi, which is a captured image of the target C that includes the position of the target A that has already been specified and an image of the target A. This is performed based on the image (third captured image) and the P2 image including the image of the object C in the first captured image.

  The position specifying unit 17 can successively specify the positions of a plurality of objects by repeating the same process in this way.

In addition, you may make it obtain | require the imaging position of the image with the camera 11 as follows.
First, let Xj and Xk be arbitrary positions on the trajectory between P1 and P2 drawn when the camera 11 is moved by the opening / closing operation of the door mirror 19.
First, the rotation component of the movement amount of the camera 11 due to the movement from the reference position X0 to the position Xj is represented as a rotation matrix Rj, and the translation component is represented as a translation column Tj. Further, the rotation component of the movement amount of the camera 11 due to the movement from the reference position X0 to the position Xk is represented as a rotation matrix Rk, and the translation component is represented as a translation column Tk.

When the position P1 is represented as Xr, the position Xj and the position Xk can be represented by the following expressions.
At this time,
Xj = Rj (X0−Tj) ………… .Formula [2]
Xk = Rk (X0−Tk) ............ Formula [3]
When X0 is eliminated from the above equations [2] and [3], the following equation [4] is obtained.
^{Xk = RkRj T {Xj-Rj} (Tk-Tj)} .................. formula [4]
“Rj ^T ” is a transposed matrix of Rj.

Here, every the Rjk = RkRj ^T, putting the Tjk = Tk-Tj, the following formula [5] can be obtained.
Xk = Rjk (Xj-Tjk) ............ Formula [5]

  By using this equation [5] and obtaining Rjk and Tjk by the least square method as described above, the position Xk can be obtained from the previously obtained position Xj. In this way, the image pickup position of the camera 11 may be obtained sequentially.

  Next, FIG. 6 will be described. FIG. 6 is a functional block diagram of still another embodiment of the position specifying device. The position specifying device in FIG. 2 specifies the position of an object around the automobile and the calculation of the distance of the object from the vehicle immediately after the automobile that has been stationary has just started moving. Do.

  6 includes an imaging device 21, a feature point extraction unit 22, a feature point tracking unit 23, a position specification unit 24, a recording device 25, an imaging device position specification unit 26, an imaging device position selection unit 27, and a distance calculation unit. 28, and a distance recording device 29.

  Among the functional blocks illustrated in FIG. 6, the imaging device 21, the feature point extraction unit 22, the feature point tracking unit 23, and the position specifying unit 24 have the same functions as the functional blocks of the position specifying device illustrated in FIG. 2. I will provide a.

  More specifically, the imaging device 21 includes the camera 11, the actuator 12, the actuator control unit 13, and the image acquisition unit 14 in the position specifying device in FIG. 2, and the imaging device 6 in the position specifying device in FIG. 1. It is equivalent to. In addition, the camera 11 is being fixed to the door mirror 19 of the motor vehicle which can be opened and closed by the actuator 12, as illustrated in FIG. 3A and FIG. 3B. In the present embodiment, the camera 11 continues to capture images around the car at predetermined time intervals from the state in which the door mirror 19 is closed.

  Further, the feature point extraction unit 22, the feature point tracking unit 23, and the position specifying unit 24 have the same functions as the feature point extracting unit 15a, the feature point tracking unit 15b, and the position specifying unit 24 in FIG. ing.

  The recording device 25 stores the same storage data as that stored in the feature point tracking result storage memory 16 of FIG. In addition, the recording device 25 stores the position information of the shooting position Qi in FIG. 5 that is specified during the process of specifying the position of the object performed by the position specifying unit 24 when the automobile is stationary. Also do. As described above, the photographing position Qi is a position on the locus between P1 and P2 drawn by the camera 11 moving by the opening / closing operation of the door mirror 19. Further, the recording device 25 also records the history of the result of specifying the position information of the object performed in the past by the position specifying device of FIG. That is, the recording device 25 provides a function corresponding to the recording unit 4 in the position specifying device of FIG.

  As described above, the position specifying device in FIG. 6 specifies the position of the object in advance when the automobile is stationary in the same manner as the position specifying device in FIG. As a result, the position information of the plurality of positions Qi on the section from the position P1 to the position P2 in FIG. 5 becomes known, and the position information of these positions P1, P2, and Qi and the camera at each of these positions. The captured image of the object captured by 11 is stored in the recording device 25.

  The imaging device position specifying unit 26 specifies an imaging position when the camera 11 captures images around the automobile at predetermined time intervals after the automobile starts moving. The imaging device position specifying unit 26 provides a function corresponding to the imaging position specifying unit 1 in the position specifying device of FIG.

  The imaging device position selection unit 27 is suitable for use for specifying the position of the object around the vehicle based on the position information of the imaging position specified by the imaging device position specifying unit 26 and the captured image at the imaging position. The selection of the shooting position from the known shooting positions is performed from the viewpoint of the accuracy of specifying the position of the object. The imaging device position selection unit 27 provides functions corresponding to the selection unit 3 and the determination unit 5 in the position specifying device of FIG.

  The distance calculation unit 28 specifies the position of the object and calculates the distance from the automobile to the object. This position is specified by the position information of the image pickup position specified by the image pickup apparatus position specifying unit 26, the image taken at the image pickup position, the position information of the shooting position selected by the image pickup apparatus position selecting unit 27, and the selected shooting. This is performed using a captured image at a position. The distance calculation unit 28 provides a function corresponding to each of the object position specifying unit 2 and the distance calculating unit 7 in the position specifying device of FIG.

  The distance recording device 29 is a recording device that records the position of the object and the distance from the vehicle at the imaging position specified by the imaging device position specifying unit 26 calculated by the distance calculating unit 28.

  Next, with reference to FIGS. 7 and 8, a processing procedure for specifying the position of an object around the automobile after the automobile in a stationary state starts moving by the position specifying device in FIG. 6 will be described. To do. FIG. 7 is a flowchart illustrating the processing contents of the position specifying process performed by the position specifying device of FIG. 6, and FIG. 8 is an explanatory diagram of the flowchart of FIG.

  At the start of the process of FIG. 7, the position of the object around the car and the position of the object are identified when the car is stationary by the same method as the position specifying device of FIG. And have already been completed. Therefore, in the recording device 25, position information of a plurality of positions in the still section in FIG. 8, images captured by the camera 11 at the plurality of positions, and a result of specifying the position information of the object by the position specifying unit 24 It is assumed that the history of is already recorded.

  Note that the stationary section shown in FIG. 8 is a section of a trajectory drawn when the camera 11 (not shown in FIG. 8) is moved by the opening / closing operation of the door mirror 19 of the stationary vehicle V. It is. Further, the travel start section shown in FIG. 8 is a section of a trajectory drawn by the camera 11 that moves when the automobile V starts traveling from a stationary state with the door mirror 19 fully opened.

Note that the processing in FIG. 7 is started when the automobile V starts running.
In FIG. 7, first, in step 1, the imaging device position specifying unit 26 performs processing for setting a reference position. In this process, a process is performed for setting the position of the camera 11 as the reference position when the vehicle V is stationary and the door mirror 19 is fully open, that is, the boundary point between the stationary section and the travel start section. . In FIG. 5 described above, the position of the camera 11 when the door mirror 19 is fully opened is displayed as P2. In FIG. 8, this position is displayed again as the reference position P0. is doing.

  Next, in step 2, the image acquisition process is performed by the imaging device 21, the feature point extraction unit 22, and the feature point tracking unit 23. In this process, an image around the automobile V is picked up by the camera 11, feature points in the image of the object A are extracted from the obtained picked-up image, and picked up at the picked-up image and the reference position. This is a process of tracking the movement of the feature point between the captured image (reference position image). The captured image obtained by this image acquisition process and the tracking result of the feature points are stored in the recording device 25.

  Next, in step 3, the imaging device position specifying unit 26 performs an imaging position specifying process for specifying the position information of the position (imaging position) of the camera 11 when the imaging is performed by the processing in step 2. In this process, the position information of the imaging position is specified based on the position information of the reference position, the reference position image, and an image (imaging position image) obtained by imaging of the camera 11 at the imaging position. It is. In FIG. 8, the imaging position whose position information is specified by this processing is displayed as P1.

  Next, in step 4, the imaging device position selection unit 27 performs a selection process of selecting an imaging position that matches a predetermined condition from a plurality of imaging positions within a still section including the reference position P0. In this process, an imaging position having aptitude to be used as a basis for position identification when the position of the object A is identified based on the above-described imaging position P1 and the imaging position image at the imaging position P1. This is a process of selecting from a plurality of imaging positions within the still section.

  Next, in step 5, the distance calculation unit 28 performs the process of specifying the position of the object A and calculating the distance from the imaging position specified in the process of step 3 to the position of the object A. This process is performed by specifying the position of the object A based on the imaging position selected by the process of step 4 and the selected imaging position, and the imaging position P1 and the imaging position image at the imaging position P1. This is a process for calculating the distance. When the position specifying and distance calculating process in step 5 is finished, the position specifying process in FIG. 7 is ended.

Next, details of each process from step 2 to step 5 in the position specifying process of FIG. 7 will be described.
First, FIG. 9 will be described. FIG. 9 is a flowchart illustrating the processing content of the image acquisition processing, which is the processing of step 2 in FIG.

  In FIG. 9, first, in S201, the imaging device 21 performs an imaging process. This process is a process of causing the camera 11 located at the imaging position P1 in FIG. 8 to capture an image around the automobile V including the object A, and storing the obtained imaging position image in the recording device 25.

  In step S202, the feature point extraction unit 22 performs feature point extraction processing. This process is a process of extracting feature points from the image of the object A in the image acquired by the process of S201. As a method for extracting the feature points, various well-known methods such as a method using a shape feature and a method using a color feature are used as in the feature point extraction unit 15a in FIG. Can do.

  In step S203, the feature point tracking unit 23 performs feature point tracking processing. In this process, first, a process of reading a reference position image captured at the reference position P0 from the recording device 25 is performed. Next, a process of extracting the maximum position where the similarity of the feature points extracted in the process of S202 is equal to or greater than a predetermined threshold from the reference position image and the captured position image captured at the imaging position P1 by the process of S201. Is done. Then, a process of acquiring the extracted position as a tracking result from the reference position image to the captured position image for the feature point is performed.

  Next, in S204, the process of storing the feature point tracking result from the reference position image to the imaging position image obtained by the processing in S203 in association with the imaging position image and storing it in the recording device 25 is characterized. This is performed by the point tracking unit 23. After that, the image acquisition process in FIG. 9 ends.

Next, FIG. 10 will be described. FIG. 10 is a flowchart illustrating the processing content of the imaging position specifying process, which is the process of step 3 in FIG.
In FIG. 10, first, in S301, the imaging device position specifying unit 26 performs a process of reading the tracking result of the feature point from the reference position image stored in the recording device 25 to the imaging position image by the image acquisition process in Step 2. .

  Next, in S302, a process of calculating a rotation matrix R representing a rotation component of the movement amount of the camera 11 from the reference position P0 to the imaging position P1 is performed based on the tracking result obtained in the process of S301. The identification unit 26 performs this. In the subsequent S303, a process of calculating a translation sequence T representing a translational component of the movement amount of the camera 11 from the reference position P0 to the imaging position P1 is performed based on the tracking result obtained in the process of S301. The identification unit 26 performs this.

  Next, in S304, the position information of the imaging position P1 is specified based on the position information of the reference position P0, the rotation matrix R representing the movement amount of the camera 11 calculated by the processing in S302 and S303, and the parallel progression T. The imaging device position specifying unit 26 performs the processing to be performed. Thereafter, the imaging position specifying process in FIG. 10 ends.

  The processing contents from S302 to S304 will be further described. Here, a method for specifying the position information of the imaging position P1 using epipolar geometry will be described.

First, the position of the feature point on the reference position image is
The corresponding position on the imaging position image for this feature point,
It expresses.

At this time, with respect to the fundamental matrix F due to epipolar constraints
Therefore, the fundamental matrix F is
And
Holds.

  Therefore, eight or more sets of feature points corresponding to the two are extracted from the reference position image and the captured position image, and the basic matrix F is derived by applying the least square method to the position data of the feature points of each set. .

  Next, assuming that the internal parameter K of the camera 11 is known, the basic matrix F is normalized according to the following equation to obtain an essential matrix E.

  Here, assuming that the translation vector of the geometric relationship between the images from the reference position image to the captured position image is t, for the basic matrix E,

Holds.

next,
Eigenvalue decomposition is performed to derive an eigenvector e corresponding to the smallest eigenvalue.

Then
From the relationship
Is obtained.

Here, for the rotation matrix R described above,
Therefore, the rotation matrix R is obtained from this relationship.

Next, a method for calculating the parallel progression T will be described.
First, among the feature points on the reference position image, those whose distance from the reference position P0 is known by processing performed when the automobile is stationary.
It expresses. In addition, the feature point on the imaging position image corresponding to the feature point on the reference position image is
It expresses.

In p _i and q _j , a set of correspondences can be obtained respectively.
as well as
It expresses. Furthermore, the position on the three-dimensional space of the real for p _k,
And the position on the reference position image
It expresses. And the position of q _{k in} the real 3D space is
And the position on the imaging position image
It expresses.

  At this time, the following three expressions hold.

Here, using the unknown l as the parallel progression T
In addition,
[Equation 20] can be organized as follows.

  The value of l in [Equation 23] is determined using the least square method, and the obtained l value is substituted into [Equation 21], whereby the parallel progression T is obtained.

The position information of the imaging position P1 is specified by performing the calculation for moving the reference position P0 by the movement amount corresponding to the rotation matrix R and the parallel progression T calculated as described above.
The specification of the position information of the image pickup position P1 by the image pickup apparatus position specifying unit 26 is performed as described above, for example. This identification result is recorded in the distance recording device 29.

Next, FIG. 11 will be described. FIG. 11 is a flowchart illustrating the processing contents of the selection process, which is the process of step 4 in FIG.
In FIG. 11, first, in step S <b> 401, a process of acquiring the movement amount of the camera 11 (movement amount from the reference position P <b> 0 to the imaging position P <b> 1) calculated by the imaging position specifying process in step 3 from the imaging device position specifying unit 26. The imaging position selection unit 27 performs this.

  Next, in S402, the imaging position selection unit 27 performs a process of determining whether or not the movement amount of the camera 11 acquired by the process of S401 is greater than or equal to a predetermined distance threshold. By this process, a process of determining the length of the base line distance, which is the distance between the reference position P0 and the imaging position P1, and the distance threshold is performed.

  In the determination process of S402, when it is determined that the movement amount (baseline distance) of the camera 11 is equal to or greater than a predetermined distance threshold (when the determination result is Yes), the process proceeds to S403. In step S403, the imaging position selection unit 27 performs a process of selecting the reference position P0. Thereafter, the selection process of FIG. 11 ends. In other words, when the imaging position selection unit 27 determines that the baseline distance is longer than the distance threshold that provides the desired position specifying accuracy, the position calculation unit 28 specifies the position information of the object A as the reference position. This is performed using the position information of P0 and the reference position image. Therefore, in the process of step 5 of FIG. 7 performed after this process, the distance calculation unit 28 specifies the position information of the object A, the position information of the reference position P0, the reference position image, and the imaging position P1. This is performed based on the position information and the captured position image.

  On the other hand, in the determination process of S402, when it is determined that the movement amount (baseline distance) of the camera 11 is less than the predetermined distance threshold (when the determination result is No), the process proceeds to S404. In S404, the imaging position selection unit 27 performs a process of selecting a position that matches a predetermined condition described later from a plurality of positions in the still section of FIG. After that, the selection process in FIG. 11 ends. That is, when the imaging position selection unit 27 determines that the baseline distance is shorter than the distance threshold, the position calculation unit 28 specifies the position information of the object A by the process of S404. The position information and the image captured at the position are used. Therefore, in the process of Step 5 of FIG. 7 performed after this process, the distance calculation unit 28 specifies the position information of the object A, the position information, the captured image at this position, and the imaging position P1. This is performed based on the position information and the captured position image.

  Note that, as is clear from FIG. 8, any of the plurality of positions in the still section where the position information is recorded in the recording device 25 has a distance from the imaging position P1 that is greater than the reference position P0. Located at (long) point. Therefore, the plurality of positions in these stationary sections are examples of known positions used in the description of the position specifying device in FIG. In the following description, the positions in these still sections are referred to as “known positions”, and an image obtained by imaging the object A at the known positions is referred to as a “known position image”. Furthermore, the known position selected by the process of S404 is referred to as “selected known position”, and an image obtained by imaging the object A at the selected known position is referred to as “selected known position image”.

  Next, the conditions used as the selection criterion in the process of S404 will be described. This condition represents suitability when used as a basis for specifying the position of the object A by the distance calculation unit 28. For example, the following two conditions can be given.

First, the first condition is that the distance from the imaging position P1 is the longest among a plurality of known positions.
If such a known position is selected, the baseline between the imaging positions of two images used for specifying the position of the object A (that is, between the imaging position P1 and the selected known position) becomes the longest. Therefore, it is possible to specify the position of the object by stereo vision performed by the distance calculation unit 28 with high accuracy.

The second condition is that the distance from the perpendicular bisector connecting the imaging position P1 to the position of the object A is the shortest among a plurality of known positions.
As described above, the accuracy of specifying the position of the object by stereo vision improves as the difference in distance from the two imaging positions where the two images used for specifying the position are taken is smaller. Here, if the known position is selected according to the second condition described above, the object A is selected from the two imaging positions (that is, the imaging position P1 and the selected known position) obtained by capturing two images used for specifying the position of the object A. The difference in distance to the position is the shortest. Therefore, it is possible to specify the position of the object by stereo vision performed by the distance calculation unit 28 with high accuracy.

  In addition, when employ | adopting 2nd conditions for the process of S404, it carries out using the specific result of the past positional information about the target object A currently recorded on the recording device 25 as a position of the target object A. .

In the process of S404, either the first condition or the second condition described above may be employed.
Further, in the process of S404, instead of selecting only one known position that matches the above-described condition, a plurality of positions may be selected in an order corresponding to the degree of meeting the condition. That is, regarding the first condition, a plurality of known positions may be selected in order of increasing distance from the imaging position P1 (long order). As for the second condition, a plurality of known positions may be selected in the order of a short distance from the vertical bisector connecting the imaging position P1 to the position of the object A. . A method for specifying the position of the object A by the distance calculation unit 28 when a plurality of known positions are selected in the process of S404 will be described later.
Note that the result of the selection performed by the imaging position selection unit 27 is stored in the distance recording device 29.

  Next, FIG. 12 will be described. FIG. 12 is a flowchart illustrating the processing contents of the position specifying and distance calculating process which is the process of step 5 in FIG.

  In FIG. 12, first, in step S501, the position of the feature point corresponds to the captured position image and the captured image (that is, the reference position image or the selected known position image) at the position selected by the processing in step 4 in FIG. The distance calculation unit 28 performs processing for acquiring information on the relationship. Note that these pieces of information have already been obtained by the feature point extraction unit 22 and the feature point tracking unit 23, and the distance calculation unit 28 acquires these pieces of information from the recording device 25.

  Next, in S502, a process of specifying the actual position of the feature point on the object A in the three-dimensional space using the principle of triangulation and storing the specification result in the distance recording device 29 is performed. 28 does. This position specifying process includes the information acquired by the process of S501, the position information of the imaging position P1, and the position information of the position selected by the process of step 4 in FIG. 7 (ie, the reference position P0 or the selected known position). Based on.

  Next, in S503, the distance calculation unit calculates the distance between the position of the feature point for the object A identified by the process of S502 and the imaging position P1, and stores the calculation result in the distance recording device 29. After that, the process of FIG. 12 ends.

Here, the processing content of the position specifying process in S502 will be further described.
The following relational expression listed in [Equation 20]
Represents the relationship between the feature points p _k and q _k in an actual three-dimensional space. Here, in order to express this relational expression using the positions of the feature points p _k and q _{k on} the image, by introducing variables k ₁ and k ₂ ,
It is expressed.

Expanding this [Equation 25],
It becomes. This formula
It expresses. Note that the positions of the feature points p _k and q _{k on} the image are expressed by three elements such as [Equation 17] and [Equation 19], and therefore, the correspondence between one set of feature points is 3 Two relational expressions are obtained.

Here, by obtaining n pairs of feature point correspondences between the captured position image and the captured image at the position selected by the processing of step 4 in FIG. 7, the relational expression of [Equation 27] is obtained for each set. By generating
Is obtained. This formula is
It has the shape of Therefore,
Is obtained by applying the least square method to obtain the values of k (k ₁ and k ₂ ). Thus the value of k ₁ and k ₂ obtained in the position from the position on the image feature point p _k and q _k, the actual feature point p _k and q _k on the three-dimensional space (i.e. , Position information of the position of the object A) is obtained.

  When the imaging position selection unit 27 selects a plurality of known positions in the process of S404 in FIG. 11, the distance calculation unit 28 performs the process of specifying the position of the object A as follows in S502. .

  First, the distance calculation unit 28 performs position information specifying processing for the object A as described above based on each of a plurality of known positions selected by the imaging position selection unit 27. Next, the distance calculation unit 28 performs a process of obtaining the positions of the centroids of the plurality of positions indicated by the plurality of position information obtained by the specifying process. The position information of the center of gravity obtained in this way is used as the position information specifying result of the object A by the position specifying process in S502.

  In addition, when obtaining the position of the center of gravity as described above, a weight according to the selection order of known positions by the imaging position selection unit 27 may be considered. That is, the distance calculation unit 28 first performs position information specifying processing for the object A as described above based on each of a plurality of known positions selected by the imaging position selection unit 27. Next, in the distance calculation unit 28, for each of a plurality of positions for which position information has been obtained by this specifying process, the imaging position selecting unit 27 selects the selected known position used for specifying the position information in the process of S404. A process of giving a weight according to the selection order when selected. Next, the distance calculation unit 28 performs processing for obtaining the centroids (weighted centroids) of the plurality of positions each weighted in this way. Then, the position information of the weighted center of gravity obtained in this way is set as the identification result of the position information of the object A by the position identification process of S502. The distance calculation unit 28 may specify the position information of the object A in this way.

The position specifying device in FIG. 6 specifies the position of the object A after the automobile V that has been stationary starts moving by performing the above-described processing by each component.
Note that the imaging apparatus of FIG. 6 can similarly perform the specification of the imaging position P2 of the new image when the camera 11 newly captures an image after acquiring the imaging position image. That is, in this case, in step 1 of FIG. 7, the imaging device position specifying unit 26 performs a process of setting the imaging position P1 specified as described above to a new reference position P0. Through the processing, the imaging position P2 of the new image can be specified.

The identification of the imaging position P2 of the new image based on the imaging position P1 can also be performed as follows.
First, the coordinates representing the reference position P0, the imaging position P1, and the imaging position P2 of the new image are X0, X1, and X2, respectively.

  Here, the rotation component of the movement amount of the camera 11 in the movement from the reference position P0 to the imaging position P1 is represented as a rotation matrix R1, and the translation component thereof is represented as a translation sequence T1. In addition, the rotation component of the movement amount of the camera 11 in the movement from the reference position P0 to the imaging position P2 of the new image is represented as a rotation matrix R2, and the translation component is represented as a translation sequence T2.

At this time, the following equation holds.
X1 = R1 (X0-T1) .............. [6]
X2 = R2 (X0-T2) ............ Formula [7]
When the above equation [6] is transformed,
X0 = R1 ^T X1 + T1 Expression (8)
Substituting this equation [8] into equation [7],
X2 = R2 (R1 ^T X1 + T1-T2)
= R2R1 ^T (X1−R1 (T2−T1))... Formula [9]
Here, when R12 = R2R1 ^T and T12 = R1 (T2-T1) are set, Equation [9] is
X2 = R12 (X1-T12) ............ Formula [10]
It is expressed.

  Therefore, by obtaining R12 and T12 using R1, R2, T1, and T2, the amount of movement of the camera 11 in the movement from the imaging position P1 to the imaging position P2 of the new image is obtained, and accordingly, the imaging position of the new image P2 can be specified based on the imaging position P1.

As a result of the operation as described above, the position specifying device of FIG. 6 can perform position specifying and distance measurement for the object A existing around the vehicle V with high accuracy even immediately after the automobile V starts moving. . Therefore, immediately after the start of the vehicle V, object information in the vicinity can be provided to the driver of the vehicle V, and the driver can be safe and secure. 6 is a monocular camera system system, and can be realized at low cost.
In addition, each functional block of the position specifying device according to each embodiment described above can be configured using a computer.

Here, FIG. 13 will be described. FIG. 13 illustrates a hardware configuration example of a computer that can be operated as each functional block of the position specifying device.
The computer 30 includes an MPU 31, a ROM 32, a RAM 33, an input device 34, a display device 35, an interface device 36, and a recording medium drive device 37. These components are connected via a bus line 38, and various data can be exchanged under the management of the MPU 31.

An MPU (Micro Processing Unit) 31 is an arithmetic processing unit that controls the operation of the entire computer 30.
A ROM (Read Only Memory) 32 is a read-only semiconductor memory in which a predetermined control program and various constant values are recorded in advance. The MPU 31 can control the operation of each component of the computer 30 by reading and executing the control program when the computer 30 is started.

  A RAM (Random Access Memory) 33 is a semiconductor memory that can be written and read at any time and used as a working storage area as necessary when the MPU 31 executes various control programs. The RAM 33 functions as the recording unit 4 in FIG. 1, the feature point tracking result storage memory 16 in FIG. 2, or the recording device 25 and the distance recording device 29 in FIG.

  The input device 34 is, for example, a keyboard device. When operated by a user of the computer 30, the input device 34 acquires input of various information from the user associated with the operation content, and inputs the acquired input information to the MPU 31. Send.

The display device 35 is a liquid crystal display, for example, and displays various texts and images according to display data sent from the MPU 31.
The interface device 36 manages the exchange of various data with various devices connected to the computer 30. More specifically, the interface device 36 performs analog-digital conversion of a captured image signal sent from the camera 11, outputs a drive signal for driving the actuator 12, and the like.

  The computer 30 having such a configuration can be caused to function as each functional block of the position specifying device according to each embodiment described above. For example, a control program for causing the MPU 31 to perform each processing procedure of the position specifying process described with reference to FIGS. 7 and 9 to 12 is created. The created control program is stored in the ROM 32 in advance. Then, a predetermined execution start instruction is given to the MPU 31, and this control program is read and executed. By doing so, the MPU 31 functions as the feature point extraction unit 22, the feature point tracking unit 23, the position specification unit 24, the imaging device position specification unit 26, the imaging position selection unit 27, and the distance calculation unit 28 in FIG. 6. become. Therefore, by connecting the imaging device 21 to the interface device 36 of the computer 30, the position specifying device of FIG. 6 can be configured.

  The recording medium driving device 37 is a device that reads various control programs and data recorded on the portable recording medium 39. For example, the flash memory is used as the ROM 32, and the computer 30 is configured so that the MPU 31 reads the above-described control program recorded on the portable recording medium 39 through the recording medium driving device 38 and stores it in the ROM 32. May be. In this case, when the MPU 31 receives a predetermined execution start instruction, the MPU 31 reads and executes the control program stored in the ROM 32.

  As the portable recording medium 39, for example, a non-transitory recording medium such as a CD-ROM (Compact Disc Read Only Memory) and a DVD-ROM (Digital Versatile Disc Read Only Memory) can be used. It is. As the portable recording medium 39, for example, a semiconductor memory provided with a USB (Universal Serial Bus) standard connector can also be used.

  In addition, the position specifying device according to each of the above-described embodiments may be configured by a computer system including a terminal device and a server device that can exchange various data between the terminal device. For example, when the position specifying device of FIG. 6 is configured with this computer system, the terminal device installed in the automobile V is connected to the imaging device 21, the communication device for communication with the server device, and the object A. A display device that provides the driver with the results of position specification and distance calculation. On the other hand, the server device includes the configuration of the computer 30 in FIG. 13, for example. However, the interface device 36 includes a communication device that manages data communication with the communication device of the terminal device. And a terminal device sends the picked-up image around the motor vehicle V sequentially imaged with the imaging device 21 to a server apparatus. On the other hand, the server device performs position specification and distance calculation for the object A as described above based on the captured image received from the terminal device, and sends the result to the terminal device to be displayed on the display device. In this manner, the position specifying device of FIG. 6 may be configured by the above-described computer system.

DESCRIPTION OF SYMBOLS 1 Image pick-up position specific | specification part 2 Object position specific | specification part 3 Selection part 4 Recording part 5 Judgment part 6 Imaging device 7, 18, 28 Distance calculation part 11 Camera 12 Actuator 13 Actuator control part 14 Image acquisition part 15 Image processing part 15a, 22 Feature point extraction unit 15b, 23 Feature point tracking unit 16 Feature point tracking result storage memory 17, 24 Position specifying unit 17a First position specifying unit 17b Second position specifying unit 17c Third position specifying unit 19 Door mirror 21 Imaging device 25 Recording device 26 imaging device position specifying unit 27 imaging device position selecting unit 29 distance recording device 30 computer 31 MPU
32 ROM
33 RAM
34 Input Device 35 Display Device 36 Interface Device 37 Recording Medium Drive Device 38 Bus Line 39 Portable Recording Medium

Claims (11)

The position information of the reference position where the position information is known, the reference position image obtained by imaging the object at the reference position, and the imaging position image obtained by imaging the object at an arbitrary imaging position. An image pickup position specifying unit that specifies position information of the image pickup position based on the image pickup position, and a position where the distance from the image pickup position whose position information is specified by the image pickup position specifying unit is larger than the reference position Position information of a known position where information is known, a known position image obtained by imaging the object at the known position, position information of the imaging position, and imaging the object at the imaging position An object position specifying unit for specifying the position information of the object based on the captured position image;
A position specifying device comprising: When there are a plurality of known positions, one of the plurality of known positions that meets a predetermined condition indicating suitability for use as a basis for position identification of the object by the object position identifying unit Is further selected as a selection known position,
The object position specifying unit includes position information of the selected known position selected by the selection unit, a selected known position image obtained by imaging the object at the selected known position, and position information of the imaging position. Identifying the position information of the object based on the imaging position image,
The position specifying device according to claim 1.   The selection unit selects, as the selected known positions, a predetermined number of the plurality of known positions in descending order from the imaging position where the imaging position specifying unit specifies the position information. The position specifying device according to claim 2. A recording unit in which a history of the result of specifying the position information of the object performed in the past is recorded;
The selection unit includes a plurality of known positions, the past recorded in the recording unit from a perpendicular bisector connecting the imaging positions where the imaging position specifying unit specified the position information. Selecting a predetermined number of the selected known positions in the order in which the distance to the position of the object according to the identification result of the position information becomes shorter,
The position specifying device according to claim 2.   When the selection unit selects a plurality of the selected known positions, the object position specifying unit specifies the position information about the object, and selects each of the plurality of selected known positions selected by the selection unit. The center of gravity of a plurality of positions indicated by the plurality of position information obtained by the identification is obtained, and the position information of the center of gravity is used as the identification result of the position information of the object. The position specifying device according to any one of 4.   When the selection unit selects a plurality of the selected known positions, the object position specifying unit specifies the position information about the object, and selects each of the plurality of selected known positions selected by the selection unit. First, and for each of a plurality of positions indicated by the plurality of position information obtained by the identification, according to the selection order when the selection unit selects the selected known position used for identifying the position information. 5. The weighted centroid of each of the plurality of positions to which the weight is given is obtained, and the position information of the weighted centroid is used as the identification result of the position information of the object. The positioning device described. A determination unit that determines a length of a baseline distance that is a distance between the reference position and the imaging position specified by the imaging position specifying unit and a predetermined distance threshold;
The object position specifying unit includes:
When the determination unit determines that the baseline distance is shorter than the distance threshold, the position information of the selected known position selected by the selection unit and the selected known position obtained by imaging the object at the selected known position Based on the image, the position information of the imaging position specified by the imaging position specifying unit, and the imaging position image obtained by imaging the object at the imaging position, the position information of the object is specified,
When the determination unit determines that the baseline distance is longer than the distance threshold, the position information of the reference position, the reference position image, the position information of the imaging position specified by the imaging position specifying unit, and the imaging Based on the imaging position image obtained by imaging the object at a position, the position information of the object is specified.
The position specifying device according to claim 2, wherein the position specifying device is a device.   8. The apparatus according to claim 1, further comprising a single imaging device that is installed in a moving body and that performs imaging of all of the reference position image, the imaging position image, and the known position image. The position specifying device according to any one of the above.   Distance calculation for calculating the distance from the imaging device to the object based on the position information of the object specified by the object position specifying unit and the position information of the image pickup position specified by the imaging position specifying unit The position specifying device according to claim 8, further comprising a unit. The position information of the reference position where the position information is known, the reference position image obtained by imaging the object at the reference position, and the imaging position image obtained by imaging the object at an arbitrary imaging position. Based on this, the imaging position specifying unit specifies the position information of the imaging position,
The position information of the known position where the distance from the imaging position where the imaging position specifying unit specified the position information is farther than the reference position and the position information is known, and the object is imaged at the known position The position information of the object is specified based on the known position image obtained in this way, the position information of the imaging position, and the imaging position image obtained by imaging the object at the imaging position. The specific department performs,
A position specifying method characterized by the above. The position information of the reference position where the position information is known, the reference position image obtained by imaging the object at the reference position, and the imaging position image obtained by imaging the object at an arbitrary imaging position. Based on the position information of the imaging position based on,
The position information of the known position where the distance from the imaging position where the position information is specified by specifying the position information of the imaging position is farther than the reference position and the position information is known; and Identification of position information of the object based on a known position image obtained by imaging the object, position information of the imaging position, and an imaging position image obtained by imaging the object at the imaging position I do,
A program that causes a computer to execute processing.