JP2008020225A - Self position estimation program, self position estimation method and self position estimation apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the accuracy for estimating a self position. <P>SOLUTION: The maximum included angle for indicating the maximum of an included angle between landmarks and the minimum included angle for indicating the minimum of the included angle between the landmarks are calculated from the maximum distance and the minimum distance between the landmarks among a plurality of the landmarks included in an input image. The maximum included angle constrained circle defined by the maximum included angle and the minimum included angle constrained circle defined by the minimum included angle are calculated based on the calculated maximum and minimum included angles. A likelihood model for defining a likelihood as an included angle between the maximum and minimum included angles is applied to a region encompassed by the calculated maximum included angle constrained circle and minimum included angle constrained circle among a plurality of the landmarks. A score for indicating a possibility of the self position is calculated at each position within the region. The scores calculated among a plurality of the landmarks are added at coordinate positions. The self position is estimated as the coordinate position with the maximum score based on the calculated and added scores. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a self-position estimation program, a self-position estimation method, and a self-position estimation apparatus.

  Conventionally, a self-position estimation device that estimates the self-position of a moving body is known. In such a self-position estimation device, the position information of a plurality of landmarks is searched from images taken by a camera mounted on a moving body, and each landmark centered on the camera mounted on the moving body is searched. The included angle is calculated, and based on this, a calculation process using the trigonometry that is the position locating principle is performed to estimate the self-position. That is, in such a self-position estimation device, a constraint circle in which a self-position may exist is calculated from the position information and the included angle of two landmarks, and such a constraint circle is converted into two different combinations. A landmark is also calculated, and the self-position is estimated at the intersection of two or more constraint circles (see, for example, Patent Document 1 and Patent Document 2).

  Here, when calculating the included angle between landmarks, the self-position estimation apparatus selects representative pixels of landmarks on the image and uses the parameter information of the camera element and optical lens to determine the distance between the pixels. Thus, the angle between the landmarks is calculated by converting the angle. In the self-position estimation apparatus, since it is very difficult to have the computer automatically determine the selection of the representative pixel even if an advanced recognition algorithm is used, the center value of the detected landmark image area is used as the representative pixel. It is common to use it simply as In the self-position estimation apparatus, the intersection of two or more constraint circles calculated based on the included angle calculated from the representative pixel is calculated by a method such as a least square method.

JP-A-8-178654 JP 2000-337887 A

  However, the above-described conventional technique has a problem in that the accuracy of self-position estimation is poor. That is, in the above-described prior art, the center value is necessarily selected as a representative pixel from a wide landmark image, the included angle is calculated, and the constraint circle is calculated. There was a problem that the accuracy of self-position estimation deteriorated.

  An analytical method such as a least square method is used to calculate the intersection of two or more constrained circles, but if the calculated included angle is not accurate, any analytical method can be used. However, the accuracy of self-position estimation did not improve, and the accuracy of self-position estimation still did not improve.

  In addition, a plurality of constraining circles calculated from redundant pairs of landmarks are used for self-position estimation, but if the probability of constraining constraining circles determined from inaccurate included angles increases, Increasing the number of landmark combinations did not improve the accuracy of self-position estimation, and still did not improve the accuracy of self-position estimation.

  Accordingly, the present invention has been made to solve the above-described problems of the prior art, and provides a self-position estimation program, a self-position estimation method, and a self-position estimation apparatus that can improve self-position estimation accuracy. The purpose is to do.

  In order to solve the above-described problem and achieve the object, the invention according to claim 1 is characterized in that each landmark centered on a self-position that is an imaging position of the input image is provided between a plurality of landmarks included in the input image. A self-position estimation program that causes a computer to execute a self-position estimation method that calculates a narrow angle between each and calculates a restraint circle constrained as the self-position from each of the calculated narrow angles and estimates the self-position In addition, for each of the plurality of landmarks, from the farthest distance and the closest distance between the landmarks, the maximum included angle indicating the maximum value of the included angle between the landmarks and the minimum indicating the minimum value of the included angle Based on the included angle calculation procedure for calculating the included angle and the maximum included angle and the minimum included angle calculated by the included angle calculating procedure for each of the plurality of landmarks, the maximum A constraint circle calculation procedure for calculating a maximum narrow-angle constraint circle determined from an angle and a minimum narrow-angle constraint circle determined from the minimum narrow angle, and a maximum included angle calculated by the constraint circle calculation procedure for each of the plurality of landmarks Applying a likelihood model that defines the likelihood as the narrow angle at each angle between the maximum included angle and the minimum included angle to an area surrounded by the restricted circle and the minimum included angle restricted circle, A score calculation procedure for calculating a score indicating the possibility as the self-position for each coordinate position and adding a score calculated for each of the plurality of landmarks for each coordinate position; and the score calculation procedure And causing the computer to execute a self-position estimation procedure for estimating the coordinate position where the score is maximized as the self-position based on the score calculated and added by And butterflies.

  In the invention according to claim 2, in the above invention, the score calculation procedure uses a likelihood model in which the likelihood is defined by a single-release curve between the maximum included angle and the minimum included angle. Then, the score is calculated.

  In the invention according to claim 3, in the above invention, the score calculation procedure applies a likelihood model in which the likelihood is defined by a uniform straight line between the maximum included angle and the minimum included angle. The score is calculated.

  According to a fourth aspect of the present invention, in the above invention, the score calculation step is performed between the maximum and minimum included angles when the imaging state of the landmark in the input image is good. The score is calculated by applying a likelihood model in which the likelihood is defined by a single-release curve, and when the landmark imaging state in the input image is not good, the maximum sandwich angle and the minimum sandwich angle are calculated. The score is calculated by applying a likelihood model in which the likelihood is defined by a straight line between corners.

  In the invention according to claim 5, in the above invention, the score calculation procedure applies the likelihood model according to the imaging state of the landmark in the input image with a weight corresponding to the imaging state. And calculating the score.

  According to the first aspect of the present invention, for each of the plurality of landmarks, the maximum included angle indicating the maximum value of the included angle between the landmarks from the farthest distance and the nearest distance between the landmarks, and the included angle The maximum narrow angle constraint circle determined from the maximum narrow angle based on the maximum sandwich angle and the minimum sandwich angle calculated by the sandwich angle calculation procedure for each of a plurality of landmarks. And the minimum narrow angle constraint circle determined from the minimum narrow angle is calculated, and the area surrounded by the maximum included angle constraint circle and the minimum included angle constraint circle calculated by the constraint circle calculation procedure between a plurality of landmarks, Apply a likelihood model that defines the likelihood as a narrow angle at each angle between the maximum and minimum included angles, and calculate a score indicating the possibility of the self position for each coordinate position in the region As well as multiple The score calculated for each mark is added for each coordinate position, and based on the score calculated by the score calculation procedure and added, the coordinate position where the score is maximum is estimated as the self-position. The influence of the calculation error can be reduced, and the self-position estimation accuracy can be improved.

  According to the invention of claim 2, the score is calculated by applying a likelihood model in which the likelihood is defined by a single-release curve between the maximum included angle and the minimum included angle. The circumference of the maximum included angle constrained circle or the minimum included angle constrained circle defined by the edges of the minimum included angle is not self-positioned, and the center included angle that is the center value of the maximum included angle and the minimum included angle is A score corresponding to a single-release curve with maximum likelihood can be calculated. For example, when the imaging state of two landmarks is good, self-position estimation accuracy can be particularly improved. . Specifically, when the imaging state of two landmarks is good, the center included angle consisting of the center of each landmark gives the maximum likelihood indicating its own position, so it is specified by a single-release curve. By applying the likelihood model, it is possible to improve the self-position estimation accuracy.

  According to the invention of claim 3, the score is calculated by applying a likelihood model in which the likelihood is defined by a uniform straight line between the maximum included angle and the minimum included angle. The circumferential portion of the maximum included angle constrained circle or the minimum included angle constrained circle defined by is not self-positioned, and is a uniform straight line with respect to the angle between the maximum included angle and the minimum included angle For example, when the imaging state of two landmarks is not good, it is possible to improve the self-position estimation accuracy. Specifically, when the imaging state of the two landmarks is not good, it is impossible to determine which angle between the calculated maximum and minimum included angles gives the maximum likelihood. By applying the likelihood model defined in (2), it is possible to improve the self-position estimation accuracy.

  According to a fourth aspect of the present invention, when the imaging state of the landmark in the input image is good, the likelihood is defined by a single-release curve between the maximum included angle and the minimum included angle. A likelihood model that calculates a score by applying a likelihood model, and that defines the likelihood as a uniform straight line between the maximum and minimum included angles if the landmark imaging state in the input image is not good Is applied to calculate the score, the likelihood model can be automatically selected according to the imaging state, and the self-position estimation accuracy can be improved.

  According to the invention of claim 5, the score is calculated by applying the likelihood model according to the weighting according to the imaging state according to the imaging state of the landmark in the input image. In particular, weighting can be applied to the score calculated from the likelihood model, and the self-position estimation accuracy can be improved.

  Exemplary embodiments of a self-position estimation program, a self-position estimation method, and a self-position estimation apparatus according to the present invention will be described below in detail with reference to the accompanying drawings. In the following description, the self-position estimation apparatus configured to include the self-position estimation program according to the present invention will be described as an embodiment mounted on a vehicle. In the following, the configuration and processing procedure of the self-position estimation apparatus in the first embodiment and the effects of the first embodiment will be described in order, and then the self-position estimation apparatus according to the second embodiment will be described. The self-position estimation apparatus according to Example 3 and the self-position estimation apparatus according to Example 4 will be described in order.

[Outline and Features of Self-Position Estimating Device in Embodiment 1]
First, the main features of the self-position estimation apparatus according to the first embodiment will be specifically described with reference to FIG. FIG. 1 is a diagram for explaining the outline and features of the self-position estimation apparatus according to the first embodiment.

  The self-position estimation apparatus according to the first embodiment calculates a narrow angle between landmarks centered on a self-position that is an imaging position of the input image for each of a plurality of landmarks included in the input image, and calculates the calculation. The outline is to calculate the restraint circle restrained as the self-position from each narrow angle and estimate the self-position. Here, the “landmark” is used as a landmark for estimating the self position, and examples thereof include a building and a road sign. The “self position” is information on a location where the image pickup apparatus (or a vehicle on which the image pickup apparatus is mounted) exists. For example, latitude and longitude where the vehicle is located correspond to this. In addition, the “squeezed angle” is an angle between two sides connecting the vehicle and the landmark among the triangle composed of the vehicle and the two landmarks. The “constrained circle” is a circle defined by the position information of the two landmarks and “the included angle”, and the “self position” that is information on the location where the imaging device exists on the circumference of the circle. Exists.

  Here, the present invention is mainly characterized in improving self-position estimation accuracy. This main feature will be briefly described. The self-position estimation apparatus according to the first embodiment determines the maximum value of the included angle between the landmarks from the farthest distance and the nearest distance between the landmarks for each of the plurality of landmarks. And the minimum included angle indicating the minimum value of the included angle is calculated. Specifically, as shown in FIG. 1A, the included angle between the landmarks is calculated as an angle having a width defined from the maximum included angle and the minimum included angle based on the width of the landmark. . FIG. 1 shows an example in which three landmarks of landmark 1, landmark 2, and landmark 3 are detected in an input image input from the imaging device mounted on the vehicle to the self-position estimation device. In FIG. 1A, the maximum sandwich angle and the minimum sandwich angle for the landmark 1 and the landmark 2 are obtained. Similarly, the landmark 1 and the landmark 3, For the landmark 2 and the landmark 3, the maximum included angle and the minimum included angle are calculated.

  Subsequently, the self-position estimation apparatus according to the first embodiment includes a maximum narrow-angle constraining circle determined from the maximum narrow angle based on each of the maximum included angle and the minimum included angle calculated between the plurality of landmarks, and A minimum narrow angle constrained circle determined from the minimum narrow angle is calculated. Specifically, as shown in FIG. 1 (B), the maximum included angle constrained circle and the minimum included angle constrained circle based on the absolute position information of landmarks described later together with the calculated maximum included angle and minimum included angle. Is calculated. A region surrounded by the maximum included angle constrained circle and the minimum included angle constrained circle is a region where the self-position that is the imaging position may exist. Here, the “absolute position information” of the landmark corresponds to, for example, the latitude and longitude at which the landmark is located.

  Subsequently, the self-position estimation apparatus according to the first embodiment includes the maximum included angle and the minimum included in the region surrounded by the maximum included angle restricted circle and the minimum included angle restricted circle calculated above for each of the plurality of landmarks. A likelihood model that defines the likelihood as a narrow angle at each angle with respect to the angle is applied, and a score indicating the possibility as the self-position is calculated for each coordinate position in the region. Specifically, a center included angle constraint circle is calculated from the center included angle that is the center value of the maximum included angle and the minimum included angle, and a score is calculated using the center included angle restricted circle. That is, as shown in FIG. 1C, the score calculated from the likelihood model is calculated for the entire region surrounded by the maximum included angle constraint circle and the minimum included angle constraint circle. In FIG. 1C, a line segment composed of an intersection point A, which is an intersection point between a straight line passing through the center of the center included angle constraint circle and the maximum included angle constraint circle, and an intersection point B between the minimum included angle constraint circle. The score calculated along AB is shown in a graph. Moreover, what is shown here is a score calculated from a likelihood model consisting of a single-release curve. Further, the xy plane is divided into grids in advance, and scoring is performed on the grid corresponding to the coordinate position through which the line segment AB passes.

  Subsequently, the self-position estimation apparatus according to the first embodiment adds the score calculated for each of the plurality of landmarks for each coordinate position. Specifically, as shown in FIG. 1D, a score calculated for each landmark and associated with the grid is added again for each grid. In FIG. 1D, the score calculated for the area surrounded by the maximum included angle constrained circle and the minimum included angle constrained circle calculated from the landmark 1 and the landmark 2, the landmark 2, The case where the score calculated with respect to the area | region enclosed by the maximum included angle constraint circle calculated from the landmark 3 and the minimum included angle constraint circle is shown is shown. Further, in FIG. 1D, an intersection that is an intersection of the maximum included angle constrained circle defined by the landmark 1 and the landmark 2 and the maximum included angle constrained circle defined by the landmark 2 and the landmark 3 is shown. A line segment CD composed of an intersection D, which is an intersection of a minimum included angle constraint circle defined by C, the landmark 1 and the landmark 2, and a minimum included angle constraint circle defined by the landmark 2 and the landmark 3. The score calculated along the line and the pattern of the added score are shown in a graph.

  Subsequently, the self-position estimation apparatus according to the first embodiment estimates the coordinate position where the score becomes the maximum as the self-position based on the score added above. In other words, the self-position estimation apparatus according to the first embodiment outputs, as a self-estimated position, a place indicating the maximum value from the scores calculated and added between a plurality of landmarks at all coordinate positions (grids). Specifically, in FIG. 1E, based on the result of adding the score calculated from the landmark 1 and the landmark 2 and the score calculated from the landmark 2 and the landmark 3, the maximum The coordinate position (grid) that is the value is output as the self-estimated position.

  For this reason, the self-position estimation apparatus according to the first embodiment can reduce the influence of the calculation error of the included angle, and can improve the self-position estimation accuracy as described above.

[Configuration of Self-position Estimation Device in Embodiment 1]
Next, the self-position estimation apparatus according to the first embodiment will be described with reference to FIGS. FIG. 2 is a block diagram illustrating a configuration of the self-position estimation apparatus according to the first embodiment, FIG. 3 is a diagram illustrating a landmark detection result stored in a landmark detection result storage unit, and FIG. FIG. 5 is a diagram showing landmark information stored in the mark information storage unit, FIG. 5 is a diagram showing landmark search results stored in the landmark search result storage unit, and FIG. 6 is a narrow angle calculation processing unit. FIG. 7 is a diagram illustrating a narrow angle calculation result stored in the narrow angle calculation result storage unit, and FIG. 8 is a constraint circle calculation by the constraint circle calculation processing unit. FIG. 9 is a diagram for explaining the processing, FIG. 9 is a diagram illustrating a constraint circle calculation result stored in the constraint circle calculation result storage unit, and FIG. 10 is a likelihood stored in the likelihood model storage unit. FIG. 11 is a diagram showing a model, and FIG. FIG. 12 is a diagram showing the voting process result stored in the voting process result storage unit, and FIG. 13 is a diagram for explaining the self-position estimation by the self-position estimating unit. FIG.

  As illustrated in FIG. 2, the self-position estimation apparatus 10 according to the first embodiment includes a camera image input unit 11, a self-position estimation output unit 12, an input / output unit I / F unit 13, a storage unit 20, and a processing unit. 30 and further connected to a car navigation output device 40 and a collision avoidance vehicle control device 50.

  The camera image input unit 11 inputs camera image data used for various processes by the processing unit 20. More specifically, image data in which landmarks are imaged and used for self-position estimation, such as image data stored in a format such as JPEG (Joint Photographic Experts Group), is directly input from the camera. The self-position estimation apparatus 10 includes a keyboard and a touch panel as an input unit (not shown), thereby receiving a self-position estimation request.

  The self-estimated position output unit 12 outputs a self-estimated position obtained as a processing result by the self-position estimating unit 36 described later to the car navigation output device 40 and the collision avoidance vehicle control device 50. Specifically, coordinates on the earth such as longitude and longitude are output as the self-position estimation position.

  The input / output control I / F unit 13 controls data transfer between the camera image input unit 11 and the self-estimated position output unit 12, the storage unit 20, and the processing unit 30.

  The storage unit 20 stores data used for various types of processing by the processing unit 30 and various types of processing results by the processing unit 30, and as particularly closely related to the present invention, as shown in FIG. Storage unit 21, landmark information storage unit 22, landmark search result storage unit 23, camera parameter storage unit 24, included angle calculation result storage unit 25, constrained circle calculation result storage unit 26, likelihood model A storage unit 27 and a voting process result storage unit 28 are provided. Each part will be described in detail later.

  The processing unit 30 executes various types of processing based on the image data transferred from the input / output control I / F unit 13, and particularly as closely related to the present invention, as shown in FIG. A unit 31, a landmark search processing unit 32, an included angle calculation processing unit 33, a constrained circle calculation processing unit 34, a voting processing unit 35, and a self-position estimation unit 36. Here, the included angle calculation processing unit 33 corresponds to “an included angle calculation procedure” described in the claims, and the constraint circle calculation processing unit 34 also corresponds to the “restricted circle calculation procedure”. 35 corresponds to the “score calculation procedure”, and the self-position estimation unit 36 corresponds to the “self-position estimation procedure”. Each part will be described in detail below.

  The landmark detection processing unit 31 detects a landmark included in the input image data. Specifically, using a known pattern recognition technique, landmarks such as buildings and signs are detected from the image data, and the coordinate position, width, height, image feature amount, etc. of the landmarks in the image data are detected. Calculate information. “Coordinate position (X coordinate and Y coordinate), width and height” indicates the range occupied by the landmark in the image data on the screen. In particular, the X coordinate indicates the coordinate position of the left end of the landmark. It is shown. The “image feature amount” may be either the detected landmark image itself or information calculated by the pattern recognition method used (for example, gradient histogram, DCT (Discrete Cosine Transform) coefficient, etc.). , Information given to identify the detected landmark. These calculation results are stored in a landmark detection result storage unit 21 described later.

  The landmark detection result storage unit 21 stores the processing result by the landmark detection processing unit 31 described above. Specifically, as shown in FIG. 3, “landmark area ID”, “X coordinate, Y coordinate”, “width”, “height”, and “image feature amount” are stored in association with each other. Here, the “landmark area ID” in FIG. 3 is an ID given to each detected landmark. The “X coordinate, Y coordinate”, “width”, “height”, and “image feature amount” are calculated from the landmark image detected by the landmark detection processing unit 31 described above. is there.

  The landmark information storage unit 22 stores data used for processing by a landmark search processing unit 32 described later. Specifically, the landmark information storage unit 22 captures information obtained by positioning a building or a sign that can be used as a landmark with a high-precision GPS (Global Positioning Systems) device in advance and an imaging device. The information obtained by doing is memorize | stored. That is, as shown in FIG. 4, “landmark ID”, “X coordinate, Y coordinate”, “image data”, and “image feature amount” are stored in association with each other. Here, the “landmark ID” in FIG. 4 is an ID assigned to each landmark held in the landmark information storage unit 22. The “X coordinate, Y coordinate” is landmark positioning information obtained by a high-accuracy GPS device. For example, latitude and longitude correspond to these. The “image data” is landmark image data obtained by the imaging apparatus, and the “image feature amount” is information (for example, a gradient histogram) calculated from the “image data” by a known pattern recognition method. Or a DCT (Discrete Course Transform) coefficient, etc.), which is the same quality as the image feature quantity of the landmark stored in the landmark detection result storage unit 21 described above.

  The landmark search processing unit 32 searches for a known landmark from landmarks captured in the input image data. Specifically, the image feature quantity stored in the landmark detection result storage unit 21 and the image feature quantity stored in the landmark information storage unit 22 are collated, and the image feature quantity matches. Search and associate “landmarks captured in data” with “known landmarks”. The result is stored in a landmark search result storage unit 23 described later.

  The landmark search result storage unit 23 stores the processing result of the landmark search processing unit 32. Specifically, as shown in FIG. 5, “landmark area ID” and “landmark ID” are stored in association with each other. Here, the “landmark region ID” is an ID assigned to each detected landmark held in the landmark detection result storage unit 21 and corresponds to the “landmark region ID” in FIG. The “landmark ID” is an ID assigned to each landmark held in the landmark information storage unit 22 and corresponds to the “landmark ID” in FIG.

  The camera parameter storage unit 24 stores parameters defined by camera lenses used by a camera that is an imaging device. Specifically, it is an angle per pixel defined by a camera lens used by a camera that is an imaging device, and is used for a narrow angle calculation process of the narrow angle calculation processing unit 33 described later.

  The included angle calculation processing unit 33 determines, for each of a plurality of landmarks, the maximum included angle indicating the maximum value of the included angle between the landmarks and the minimum of the included angles from the farthest distance and the nearest distance between the landmarks. The minimum included angle indicating the value is calculated, and the result is stored in the narrow angle calculation result storage unit 25 described later. Specifically, based on the “X coordinate and width” of the landmark stored in the landmark detection result storage unit 21 and the “camera parameter” stored in the camera parameter storage unit 24, the land included in the image is displayed. The maximum and minimum included angles between the marks are calculated. That is, as shown in FIG. 6A, the X coordinate “x_i” and the width “w_i” of the landmark i stored in the landmark detection result storage unit 21, the X coordinate “x_j” of the landmark j, and From the width “w_j”, “θ_ij_max” that is the maximum included angle between the landmark i and the landmark j and “θ_ij_min” that is the minimum included angle are calculated. Here, in FIG. 6B, an angle Δθ (delta theta) per pixel which is a parameter defined by a camera lens used by a camera which is an imaging device stored in the camera parameter storage unit 24. 2 shows a mathematical formula for calculating the maximum included angle “θ_ij_max” and the minimum included angle “θ_ij_min”.

  The included angle calculation result storage unit 25 stores the processing result of the included angle calculation processing unit 33 described above. Specifically, as shown in FIG. 7, “landmark area ID combinations (L, R)”, “minimum included angle (°)”, and “maximum included angle (°)” are stored in association with each other. Here, “Landmark area ID combination (L, R)” in FIG. 7 means that “Landmark area ID” of two detected landmarks being processed is relatively left (L) in the input image. The landmark area ID is located in the order of the landmark area ID located on the right (R) in the input image. The “minimum included angle (°)” and the “maximum included angle (°)” are the minimum included angle calculated by the included angle calculation processing unit 33 for the “landmark area ID combination (L, R)”. Corresponds to the maximum included angle.

  The constraining circle calculation processing unit 34 calculates the maximum narrow angle constraining circle determined from the maximum narrow angle and the minimum narrow angle based on the maximum included angle and the minimum included angle calculated above for each of a plurality of landmarks. The fixed minimum narrow angle constraint circle is calculated, and the result is stored in the constraint circle calculation result storage unit 26 described later. More specifically, the “X coordinate, Y coordinate (for example, latitude and longitude)” of the landmark stored in the landmark search result storage unit 23 and the “maximum included angle” stored in the included angle calculation result storage unit 25. And the minimum included angle constrained circle and the minimum included angle constrained circle are calculated based on the “minimum included angle”, and an area where the self-position may exist is calculated.

  That is, as shown in FIG. 8A, if the position information and the included angle of two landmarks are given, a constraining circle in which the self position exists can be calculated. More specifically, as shown in FIG. 8A, “Loc_i = (X_i, X_i, which is absolute position information whose“ landmark area ID ”stored in the landmark search result storage unit 23 is“ IL_i1 ”. Y_i) ”,“ Loc_j = (X_j, Y_j) ”, which is absolute position information with“ landmark region ID ”“ IL_i1 ”, and“ clipping angle (θ_ij) ”, a constraint circle satisfying these conditions is obtained. The “constrained circle parameter (λ_ij)” is calculated. The “constrained circle parameter (λ_ij)” is a parameter composed of two elements: “vector O (θ_ij)” that is the center of the circle and “r (θ_ij)” that is the radius. FIG. 8B shows mathematical formulas for calculating two elements of the constraint circle parameter. “Θ_ij” is an angle between the maximum included angle “θ_ij_max” held in the included angle calculation result storage unit 25 and the minimum included angle “θ_ij_min”. Thus, the constraint circle calculation processing unit 34 calculates the maximum included angle constraint circle determined from the maximum included angle and the minimum included angle constraint circle determined from the minimum included angle. The constraining circle calculation processing unit 34 is used when calculating a score in a voting processing unit 35 to be described later, and is a constraining circle for a central constraining angle constraining circle that is determined from the central constraining angle that is the central value of the maximum constraining angle and the minimum conclusive angle. Parameters are also calculated in advance.

  The constraint circle calculation result storage unit 26 stores the result calculated by the constraint circle calculation processing unit 34. Specifically, as shown in FIG. 9, "radius at minimum value", "center at minimum value", "radius at maximum value", "center at maximum value", "radius at center value" and "center value" “Time center” is stored in association with each other. Here, “minimum value radius” and “minimum value center” in FIG. 9 are constraining circle parameters of the minimum included angle constraint circle, and “maximum value radius” and “maximum value center” The constrained circle parameters of the corner constrained circle, and “radius at the center value” and “center at the center value” are constrained circle parameters of the center included angle constrained circle. Here, the constraint circle parameter of the center included angle constraint circle calculated in advance is stored together with the constraint circle parameter of the minimum included angle constraint circle and the constraint circle parameter of the maximum included angle constraint circle.

  The likelihood model storage unit 27 stores a likelihood model applied to score calculation of the voting processing unit 35 described later. Specifically, the likelihood model storage unit 27 stores a likelihood model defined by a single-release curve as shown in FIG. This will be described in detail. FIG. 10A is the same as that shown in FIG. 6A, and is the maximum included angle “θ_ij_max” and the minimum included angle based on the width of the landmark. “Θ_ij_min” is given. FIG. 10B is a graph showing a likelihood model including a single-release curve in which “θ_ij_μ” that is the central included angle is the maximum likelihood for each angle between the maximum included angle and the minimum included angle. Is shown. FIG. 10C shows a mathematical formula for calculating the likelihood model function (P (θ_ij)).

  The voting processing unit 35 sets each angle between the maximum included angle and the minimum included angle in a region surrounded by the maximum included angle restricted circle and the minimum included angle restricted circle calculated above for each of a plurality of landmarks. Applying a likelihood model that prescribes the likelihood as a narrow angle in, a score indicating the possibility as a self-position is calculated for each coordinate position in the region. Specifically, as shown in FIG. 11, the “constraint circle parameter of the minimum included angle restraint circle”, the “constraint circle parameter of the maximum included angle restraint circle” and the “center” stored in the restraint circle calculation result storage unit 26 are stored. Based on the “constraint circle parameter of the included angle constraint circle” and the “likelihood model” stored in the likelihood model storage unit 27, the entire region surrounded by the maximum included angle constraint circle and the minimum included angle constraint circle is A score calculated from the likelihood model is calculated. That is, in FIG. 11A, a line segment composed of an intersection point A, which is an intersection point between a straight line passing through the center of the center included angle constraint circle and the maximum included angle constraint circle, and an intersection point B, which is the intersection of the minimum included angle constraint circle. It shows that the score calculated by applying the likelihood model memorize | stored in the likelihood model memory | storage part 27 is voted along AB. In FIG. 11 (B), the shape of the score calculated by applying the likelihood model shown in FIG. 10 (B) to the line segment AB is shown in a graph, and in FIG. 11 (C), the line segment is shown. A function P ′ (u) for calculating a score at a distance u from AB is shown.

  Here, the voting process described above is performed on the entire region surrounded by the maximum included angle constraint circle and the minimum included angle constraint circle, and the procedure is performed according to the following steps. First, the points are given at equal intervals around the circumference of the center included angle constraining circle. Subsequently, the intersection point A, which is the intersection of the center of the center included angle constraint circle, the straight line composed of the above points, and the maximum included angle constraint circle and the intersection point of the minimum included angle constraint circle is calculated. Subsequently, for the points on the line segment AB, a score is calculated by the function shown in FIG. Subsequently, a grid (GRID (x, y)) where the line segment AB exists is calculated, and a corresponding score is associated with each grid. For example, if 100 points are given at equal intervals on the circumference of the center included angle constraining circle, the processing from step 2 to step 4 is performed for all 100 points. However, for example, if the grid that has already been voted by the process of the 34th point can be voted by the process of the 35th point, the vote is not made. This is to prevent duplicate scores from being voted in a process between one landmark. However, voting is allowed for processing between different landmarks. By performing the score voting process as described above, a voting score as shown in FIG. 12 is obtained between one landmark.

  In addition, the voting processing unit 35 adds the score calculated for each of the plurality of landmarks for each coordinate position. Specifically, as shown in FIG. 13, a score calculated for each landmark and associated with the grid is added again for each grid.

  The voting processing result storage unit 28 stores the processing result of the voting processing unit 35. Specifically, as shown in FIG. 12 and FIG. 13B, the score voted for the grid corresponding to the entire region surrounded by the maximum included angle restricted circle and the minimum included angle restricted circle is stored. To do. In the “voting score for landmarks 1 and 2” in FIG. 12 and the “voting score for landmarks 1 and 2” and the “voting score for landmarks 2 and 3” in FIG. The correspondence is shown in tabular form.

  Based on the score calculated and added as described above, the self-position estimation unit 36 estimates the coordinate position where the score is maximum as the self-position. Specifically, based on the result after addition stored in the voting process result storage unit 28, the position indicating the maximum value is output as the self-estimated position. FIG. 13 shows a case where “landmark 1 and landmark 2” and “landmark 2 and landmark 3” are processed and the self-position is estimated. As shown in FIG. 13 (A), by the processing of the voting processing unit 35, “Landmark 1 and Landmark 2” and “Landmark 2 and Landmark 3” are respectively subjected to the maximum included angle restriction circle and the minimum included angle. A score is calculated and added to the grid corresponding to the region defined by the corner constraint circle. Here, the self-position estimation unit 36 is a grid that indicates the maximum value among the results of “score addition for each landmark” in FIG. 13B (stored in the voting process result storage unit 28). Is estimated as a self-position. Here, the coordinate position of GRID (X, Y) = (3,4) is estimated as the self position. In addition, the said coordinate position may be specifically converted into the latitude and the longitude from the absolute position information of the landmark.

  The car navigation output device 40 outputs the output result of the self-estimated position output unit 12 to a car navigation device (also referred to as a car navigation device). For example, the latitude and longitude information calculated as the self-estimated position is displayed on the screen of the car navigation device together with the existing map information.

  The collision avoidance vehicle control device 50 controls the operation of the vehicle. For example, when the car navigation apparatus holds road position information and shape information, the road position information and shape information are acquired from the car navigation apparatus, and the output result of the self-estimated position output unit 12 is acquired. Based on the information of the information and the shape information and the output result of the self-estimated position output unit 12, it is determined whether or not the vehicle may enter the outside of the road, and it is determined that the vehicle may enter the outside of the road. In such a case, the vehicle operation stop control is performed in order to avoid entering the road.

[Procedure for Processing by Self-position Estimation Device in Embodiment 1]
Next, processing performed by the self-position estimation apparatus according to the first embodiment will be described with reference to FIG. FIG. 14 is a flowchart illustrating a processing procedure of the self-position estimation apparatus according to the first embodiment.

  First, when the self-position estimation apparatus 10 according to the first embodiment receives a self-position estimation request from a keyboard or a touch panel (Yes in step S1401), the landmark detection processing unit 31 detects a landmark included in the input image data. (Step S1402).

  Then, the landmark search processing unit 32 searches for a known landmark from the landmarks captured in the input image data (step S1403).

  Subsequently, the included angle calculation processing unit 33 determines the maximum value indicating the maximum value of the included angle between the landmarks from the farthest distance and the nearest distance between the landmarks for each of the plurality of landmarks searched above. An included angle calculation process for calculating the included angle and the minimum included angle indicating the minimum value of the included angle is performed (step S1404). That is, based on the “X coordinates and width” of the landmarks stored in the landmark detection result storage unit 21 and the “camera parameters” stored in the camera parameter storage unit 24, the landmarks included in the image are displayed. The maximum included angle and the minimum included angle are calculated.

  Then, the constrained circle calculation processing unit 34 determines the maximum narrow angle constrained circle and the minimum narrow angle determined from the maximum narrow angle based on the maximum sandwich angle and the minimum sandwich angle calculated above for each of the plurality of landmarks. The minimum narrow-angle constraint circle determined from the corner is calculated (step S1405). That is, the “X coordinate, Y coordinate (for example, latitude and longitude)” of the landmark stored in the landmark search result storage unit 23 and the “maximum included angle and minimum included angle” stored in the included angle calculation result storage unit 25. Based on the “angle”, a maximum included angle constrained circle and a minimum included angle constrained circle are calculated, and an area where the self-position may exist is calculated.

  Subsequently, the voting processing unit 35 determines the interval between the maximum included angle and the minimum included angle in the region surrounded by the maximum included angle restricted circle and the minimum included angle restricted circle calculated above for each of the plurality of landmarks. Applying a likelihood model that defines the likelihood as a narrow angle at each angle, a score calculation process is performed for calculating a score indicating the possibility of the self position for each coordinate position in the region (step S1406). That is, the “constraint circle parameter of the minimum included angle restraint circle”, the “constraint circle parameter of the maximum included angle restraint circle”, and the “constrained circle parameter of the center included angle restraint circle” stored in the restraint circle calculation result storage unit 26, Based on the “likelihood model” stored in the likelihood model storage unit 27, the score calculated from the likelihood model is calculated for the entire region surrounded by the maximum included angle restricted circle and the minimum included angle restricted circle. To do.

  Then, the voting processing unit 35 adds the score calculated for each of the plurality of landmarks for each coordinate position (step S1407). That is, the score calculated for each landmark and associated with the grid is added again for each grid.

  Then, based on the score calculated and added as described above, the self-position estimating unit 36 estimates the coordinate position where the score is the maximum as the self-position (step S1408), and ends the process. That is, among the results of “score addition for each landmark” stored in the voting process result storage unit 28, the grid indicating the maximum value is estimated as the self position, and the position is output as the self position. finish.

[Effect of Example 1]
As described above, according to the first embodiment, for each of a plurality of landmarks, from the farthest distance and the nearest distance between the landmarks, The minimum included angle indicating the minimum included angle is calculated, and the maximum narrowness determined from the maximum narrow angle based on the maximum included angle and the minimum included angle calculated by the included angle calculation procedure for each of the plurality of landmarks. Calculates the minimum angle and the minimum angle determined from the minimum angle, and the maximum angle and the minimum angle in the area surrounded by the maximum angle and the minimum angle constraint circle between multiple landmarks. Applying a likelihood model that defines the likelihood as a narrow angle at each angle between the included angle and calculating a score indicating the possibility of self-position for each coordinate position in the area, and a plurality of lands Calculated for each mark Since the score is added for each coordinate position and the coordinate position where the score is maximum is estimated as the self position, the influence of the calculation error of the included angle can be reduced and the self position estimation accuracy can be improved. Become.

  Further, according to the first embodiment, a likelihood model that defines the likelihood as a narrow angle at each angle between the maximum included angle and the minimum included angle is applied, and the self position is determined for each coordinate position in the region. In the case of a likelihood model in which the likelihood is defined by a single-release curve between the maximum included angle and the minimum included angle applied in the first embodiment, for example, two landmarks are calculated. The circumferential portion of the maximum included angle constrained circle or the minimum included angle constrained circle defined by the end of the position is not self-positioned, and the accuracy of self-position estimation can be improved.

  Further, according to the first embodiment, for example, even in a vehicle moving in a place where radio waves from an artificial satellite do not reach the GPS device, highly accurate self-position estimation is performed only with input image data from the imaging device. It becomes possible to do.

  Further, according to the first embodiment, since the self-position estimation is performed with higher resolution than the self-position estimation by the GPS apparatus using only the input image data from the imaging device, for example, it is possible to avoid the entry to the outside of the road. become.

  In the first embodiment described above, one likelihood model prepared in advance is applied, but in the second embodiment, a likelihood model is selected from a plurality of likelihood models according to the imaging state of the landmark. A self-position estimation apparatus that applies the selected likelihood model will be described.

[Outline and Features of Self-Location Estimation Device in Embodiment 2]
First, the main features of the self-position estimation apparatus according to the second embodiment will be specifically described with reference to FIG. FIG. 15 is a diagram for explaining the outline and features of the self-position estimation apparatus according to the second embodiment.

  The self-position estimation apparatus according to the second embodiment calculates a score by applying a likelihood model in which the likelihood is defined by a single-release curve between the maximum narrow angle and the minimum narrow angle, and the landmark in the input image is calculated. When the imaging state is not good, a score is calculated by applying a likelihood model in which the likelihood is defined by a uniform straight line between the maximum narrow angle and the minimum narrow angle.

  Specifically, as shown in FIG. 15, when the imaging state of the landmark detected in the input image is evaluated and the evaluation of the two landmarks for which score calculation processing is both “good”, the maximum narrow angle The score is calculated by applying a likelihood model in which the likelihood is defined by a single-release curve between the angle and the minimum narrow angle, and one of the two landmark evaluations that perform score calculation processing is “bad” In the case of, the score is calculated by applying a likelihood model in which the likelihood is defined by a uniform straight line between the maximum narrow angle and the minimum narrow angle.

  FIG. 15 shows an example in which three landmarks of landmark 1, landmark 2, and landmark 3 are detected in the input image input to the self-position estimation device from the imaging device mounted on the vehicle. The image data of the landmark 1, the landmark 2, and the landmark 3 detected in the input image, and the landmark 1, the landmark 2, and the landmark 3 stored in the landmark information storage unit are described. Each image data is compared and evaluated. Here, the evaluation of the landmark 1 and the landmark 2 is “good”, and the evaluation of the landmark 3 is “bad”. Like Landmark 1 and Landmark 2, when processing is performed between landmarks that are evaluated as “good”, the likelihood model P1 defined by the single-release curve is applied and score calculation is continued. Process. Like the landmark 2 and the landmark 3, when processing is performed between landmarks having at least one evaluation of “bad”, the likelihood model P2 defined by a uniform straight line is applied and the score is continuously calculated. .

  For this reason, the self-position estimation apparatus according to the second embodiment can automatically select a likelihood model according to the imaging state, and can improve self-position estimation accuracy.

[Configuration of Self-Position Estimation Device in Embodiment 2]
Next, the self-position estimation apparatus according to the second embodiment will be described with reference to FIGS. FIG. 16 is a block diagram illustrating a configuration of the self-position estimation apparatus according to the second embodiment, and FIG. 17 is a diagram for explaining a selection likelihood model stored in the likelihood model storage unit. These are the figures for demonstrating the landmark detection image evaluation result and selection likelihood model which are memorize | stored in a landmark detection image evaluation result memory | storage part and a selection likelihood model memory | storage part.

  As shown in FIG. 16, the self-position estimation apparatus in the second embodiment is basically the same as the self-position estimation apparatus in the first embodiment shown in FIG. 2, and includes a landmark detection image evaluation result storage unit 29, a selection The difference is that a likelihood model storage unit 210, a landmark detection image evaluation unit 37, and a likelihood model selection unit 38 are newly provided. Hereinafter, these will be mainly described.

  The likelihood model storage unit 27 sets the likelihood between the maximum narrow angle and the minimum narrow angle, and the likelihood model in which the likelihood is defined by a single-release curve between the maximum narrow angle and the minimum narrow angle. The likelihood model defined by the straight line is stored. Specifically, as shown in FIG. 17, the likelihood model storage unit 27 is a likelihood model likelihood model P1 (1) in which the likelihood is defined by a single-release curve between the maximum narrow angle and the minimum narrow angle. FIG. 17A) and a likelihood model P2 (FIG. 17B) in which the likelihood is defined by a uniform straight line between the maximum narrow angle and the minimum narrow angle are stored. Here, a mathematical expression for deriving the likelihood model P1 and a mathematical expression for deriving the likelihood model P2 are shown together.

  The landmark detection image evaluation unit 37 evaluates the imaging state of the landmark detected in the input image. Specifically, the landmark detection image data stored in the landmark detection result storage unit 21 is compared with the landmark image data stored in the landmark information storage unit 22, and the landmark detection image data is stored. Evaluate the imaging state. That is, as shown in FIG. 18, an evaluation function “dup” that quantifies the degree of overlap between the landmark image data “Crop” and the landmark detection image “IL” is used, and the numerical value is set to a predetermined threshold (TH ) If it is larger, it is evaluated as “good”, and if it is smaller, it is evaluated as “bad”.

  The landmark detection image evaluation result storage unit 29 stores the evaluation result of the landmark detection image evaluated by the landmark detection image evaluation unit 37. Specifically, as shown in FIG. 18, “Landmark area ID combination (L, R)”, “Minimum included angle (°)”, “Maximum included angle (°)”, “Detected state (L, R, R)” R) ”and“ selected model ”are stored in association with each other. Here, “Landmark area ID combination (L, R)”, “Minimum included angle (°)” and “Maximum included angle (°)” in FIG. 18 are “Landmark area ID combinations ( L, R) "," minimum included angle (°) "and" maximum included angle (°) ", and" detected state (L, R) "is the two detected landmarks being processed Among these, the evaluation result of the image data of the landmark area ID relatively positioned on the left “L” and the evaluation result of the image data of the landmark area ID relatively on the right “R” in the input image are sequentially displayed. It is written. Here, in order to easily understand the results stored in the landmark detection image evaluation result storage unit 29, the results are shown together with the included angle calculation results stored in the included angle calculation result storage unit 25. The “selected model” will be described later in the selection likelihood model storage unit 210.

  The likelihood model selection unit 38 selects a likelihood model to be applied from the likelihood model storage unit 27 based on the landmark detection image evaluation result stored in the landmark detection image evaluation result storage unit 29. That is, in the landmark detection image evaluation result stored in the landmark detection image evaluation result storage unit 29, if both of the two landmarks that perform score calculation are “good”, the maximum narrow angle and the minimum narrow angle Likelihood model P1 in which the likelihood is defined by a single-release curve is selected, and if at least one is “bad”, the likelihood is a straight line between the maximum narrow angle and the minimum narrow angle. The specified likelihood model P2 is selected.

  The selection likelihood model storage unit 210 stores the likelihood model selected by the likelihood model selection unit 38. Specifically, as shown in FIG. 18, “Landmark area ID combination (L, R)”, “Minimum included angle (°)”, “Maximum included angle (°)”, “Detected state (L, R, R)” R) ”and“ selected model ”are stored in association with each other. In FIG. 18, “Landmark area ID combination (L, R)”, “Minimum included angle (°)”, “Maximum included angle (°)”, and “Detected state (L, R)” are This is the same as that described in the landmark detection image evaluation result storage unit 29, and “selected model” indicates the likelihood model selected by the likelihood model selection unit 38. For example, as shown in FIG. 18, it is stored that the likelihood model P1 is selected in “IL_i1” and “IL_j1”, and the likelihood model P2 is selected in “IL_i2” and “IL_j2”. Here, in order to easily understand the result stored in the selection likelihood model storage unit 210, the result is shown together with the included angle calculation result stored in the included angle calculation result storage unit 25.

  The voting processing unit 35 applies a likelihood model in which the likelihood is defined by a single-release curve between the maximum narrow angle and the minimum narrow angle when the landmark imaging state in the input image is good. The score is calculated by applying a likelihood model in which the likelihood is defined as a uniform straight line between the maximum narrow angle and the minimum narrow angle when the landmark imaging state in the input image is not good. calculate. Specifically, the likelihood model P1 or the likelihood model P2 is automatically selected according to the imaging state of the two landmarks between the landmarks, and the score corresponding to the applied likelihood model is calculated. Is done.

[Procedure of Processing by Self-position Estimation Device in Embodiment 2]
Next, processing performed by the self-position estimation apparatus according to the second embodiment will be described with reference to FIG. FIG. 19 is a flowchart illustrating a processing procedure of the self-position estimation apparatus according to the second embodiment.

  First, when the self-position estimation apparatus 10 according to the second embodiment receives a self-position estimation request from a keyboard or a touch panel (Yes at step S1901), the same as the first embodiment (similar to the processing at steps S1402 to S1405 shown in FIG. 14). In addition, “landmark detection”, “landmark search”, “squeeze angle calculation”, and “constrained circle calculation” are performed (steps S1902 to S1905).

  Subsequently, the landmark detection image evaluation unit 37 evaluates the imaging state of the landmark detected in the input image (step S1906). That is, the landmark detection image data stored in the landmark detection result storage unit 21 and the landmark image data stored in the landmark information storage unit 22 are compared to determine the imaging state of the landmark detection image. evaluate.

  Then, the likelihood model selection unit 38 selects a likelihood model to be applied from the likelihood model storage unit 27 based on the landmark detection image evaluation result stored in the landmark detection image evaluation result storage unit 29 (step). S1907). That is, in the landmark detection image evaluation result stored in the landmark detection image evaluation result storage unit 29, if both of the two landmarks that perform score calculation are “good”, the maximum narrow angle and the minimum narrow angle Likelihood model P1 in which the likelihood is defined by a single-release curve is selected, and if at least one is “bad”, the likelihood is a straight line between the maximum narrow angle and the minimum narrow angle. The specified likelihood model P2 is selected.

  Subsequently, the voting processing unit 35 determines the interval between the maximum included angle and the minimum included angle in the region surrounded by the maximum included angle restricted circle and the minimum included angle restricted circle calculated above for each of the plurality of landmarks. In step S1908, a likelihood model that defines the likelihood as a narrow angle at each angle is applied to calculate a score indicating the possibility of a self-position for each coordinate position in the region (step S1908). That is, the “constraint circle parameter of the minimum included angle restraint circle”, the “constraint circle parameter of the maximum included angle restraint circle”, and the “constrained circle parameter of the center included angle restraint circle” stored in the restraint circle calculation result storage unit 26, Based on the “likelihood model” stored in the selection likelihood model storage unit 210, a score calculated from the likelihood model is applied to the entire region surrounded by the maximum included angle constraint circle and the minimum included angle constraint circle. calculate.

  Then, the voting processing unit 35 adds the score calculated for each of the plurality of landmarks for each coordinate position (step S1909). That is, the score calculated for each landmark and associated with the grid is added again for each grid.

  Then, based on the score calculated and added as described above, the self-position estimating unit 36 estimates the coordinate position where the score is maximized as the self-position (step S1910), and ends the process. That is, among the results of “score addition for each landmark” stored in the voting process result storage unit 28, the grid indicating the maximum value is estimated as the self position, and the position is output as the self position. finish.

[Effect of Example 2]
As described above, according to the second embodiment, since the likelihood model can be automatically selected according to the imaging state of the landmark, the self-position estimation accuracy can be improved.

  Further, according to the second embodiment, it is possible to calculate a score corresponding to a single-release curve with the central likelihood that is the central value of the maximum included angle and the minimum included angle as the maximum likelihood. When the landmark is imaged well, it is possible to improve the self-position estimation accuracy. Specifically, when the imaging state of two landmarks is good, the center included angle consisting of the center of each landmark gives the maximum likelihood indicating its own position, so it is specified by a single-release curve. By applying the likelihood model, it is possible to improve the self-position estimation accuracy.

  Further, according to the second embodiment, the score can be calculated as a uniform straight line with respect to the angle between the maximum included angle and the minimum included angle. For example, when the imaging state of two landmarks is not good In particular, the self-position estimation accuracy can be improved. Specifically, when the imaging state of the two landmarks is not good, it is impossible to determine which angle between the calculated maximum and minimum included angles gives the maximum likelihood. By applying the likelihood model defined in (2), it is possible to improve the self-position estimation accuracy.

  In the above-described second embodiment, the likelihood model is selected from the plurality of likelihood models according to the imaging state of the landmark, and the score as the self-position is calculated by applying the selected likelihood model. In the third embodiment, a self-position estimation apparatus that calculates a score by applying a selected likelihood model with weighting according to the imaging state according to the imaging state of the landmark will be described.

[Outline and Features of Self-Location Estimation Device in Embodiment 3]
First, the main features of the self-position estimation apparatus according to the third embodiment will be specifically described with reference to FIG. FIG. 20 is a diagram for explaining the outline and features of the self-position estimation apparatus according to the third embodiment.

  In the self-position estimation apparatus according to the third embodiment, when the landmark imaging state in the input image is good, the likelihood defined by the single-release curve between the maximum included angle and the minimum included angle When the score is calculated by applying the model, and the landmark imaging state in the input image is not good, the likelihood model that defines the likelihood as a uniform straight line between the maximum and minimum included angles is applied Then, a score is calculated by applying a likelihood model with a weighting according to the imaging state according to the imaging state of the landmark in the input image.

  Specifically, as shown in FIG. 20, the score is calculated by further applying weighting to the score calculated from the selected likelihood model, based on the evaluation value calculated by the landmark detection image evaluation. Note that FIG. 20 shows an example in which three landmarks of landmark 1, landmark 2, and landmark 3 are detected in the input image input to the self-position estimation device from the imaging device mounted on the vehicle. The image data of the landmark 1, the landmark 2 and the landmark 3 detected in the input image and the landmark 1, the landmark 2 and the landmark 3 held in the landmark information storage unit Each image data is compared and evaluated.

  Here, since the evaluation values of the landmark 1 and the landmark 2 are values higher than the “threshold value TH 0.5” set here, both of them are the same “good”, and the likelihood model is P1. Is selected. However, since the evaluation value of the landmark 1 is the maximum evaluation value “1”, the evaluation value of the landmark 2 is as low as “0.65”. A score calculated by further applying weighting to the score calculated by applying P1 is calculated. In the combination of the landmark 2 and the landmark 3, since the evaluation of the landmark 2 is “good” and the evaluation of the landmark 3 is “bad”, the likelihood model P2 is selected and the score is calculated. However, a score to which weighting is applied is calculated based on these evaluation values.

  For this reason, the self-position estimation apparatus according to the third embodiment automatically selects the likelihood model according to the imaging state and is automatically selected according to the imaging state, as described above. Weighting can be applied to the score calculated from the likelihood model, and the self-position estimation accuracy can be improved.

[Configuration of Self-Position Estimation Device in Embodiment 3]
Next, the self-position estimation apparatus according to the third embodiment will be described with reference to FIG. 21 and FIG. FIG. 21 is a block diagram illustrating a configuration of the self-position estimation apparatus according to the third embodiment, and FIG. 22 is a diagram for explaining a weighted voting processing unit.

  As shown in FIG. 21, the self-position estimation apparatus in the third embodiment is basically the same as the self-position estimation apparatus in the second embodiment shown in FIG. 16, and a weighted voting processing section 39 is used instead of the voting processing section 35. Is different. Hereinafter, this will be mainly described.

  The weighting voting processing unit 39 calculates a score by applying a likelihood model with weighting according to the imaging state according to the imaging state of the landmark in the input image. Specifically, further weighting is performed based on the evaluation value stored in the landmark detection image evaluation result storage unit 29 and the score calculated from the selected likelihood model stored in the selection likelihood model storage unit. The score to which is applied is calculated. For example, as illustrated in FIG. 22, the “selected likelihood model P” is derived by multiplying the “function P ′ for calculating the score” by the “evaluation value s” calculated from the “evaluation function dup”. A score to which weighting is applied is calculated by the function P ″ ”.

[Procedure of processing by self-position estimation apparatus in Embodiment 3]
Next, processing performed by the self-position estimation apparatus according to the third embodiment will be described with reference to FIG. FIG. 23 is a flowchart illustrating a processing procedure of the self-position estimation apparatus according to the third embodiment.

  First, when the self-position estimation apparatus 10 according to the third embodiment receives a self-position estimation request from the keyboard or the touch panel (Yes at step S2301), the same as the second embodiment (similar to the processing at steps S1902 to S1907 shown in FIG. 19). In addition, “landmark detection”, “landmark search”, “clipping angle calculation”, “constrained circle calculation”, “landmark detection image evaluation”, and “likelihood model selection” are performed (steps S2302 to S2307).

  Subsequently, the weighting voting processing unit 39 calculates a score by applying the likelihood model with weighting according to the imaging state in accordance with the imaging state of the landmark in the input image (step S2308). Specifically, based on the evaluation value stored in the landmark detection image evaluation result storage unit 29 and the score calculated from the selected likelihood model stored in the selection likelihood model storage unit 210, further A score with weighting applied is calculated.

  Subsequently, the weighted voting processing unit 39 adds the score calculated for each of the plurality of landmarks for each coordinate position (step S2309). That is, the score calculated for each landmark and associated with the grid is added again for each grid.

  Then, based on the score calculated and added as described above, the self-position estimating unit 36 estimates the coordinate position where the score is maximized as the self-position (step S2310), and ends the process. That is, among the results of “score addition for each landmark” stored in the voting process result storage unit 28, the grid indicating the maximum value is estimated as the self position, and the position is output as the self position. finish.

[Effect of Example 3]
As described above, according to the third embodiment, the imaging state of the input image is evaluated, the likelihood model is automatically selected, the score is calculated, and the weight is further set according to the evaluation value that is the evaluation reference. Since the score is calculated by application, self-position estimation accuracy can be improved. That is, even if the evaluation of the detected landmark image data is the same, weighting can be further applied to the score according to the degree of the evaluation value, so that self-position estimation accuracy can be improved.

  Now, although the self-position estimation apparatus in Examples 1-3 was demonstrated so far, this invention may be implemented with a various different form other than the Example mentioned above. Therefore, in the following, various different embodiments will be described as being divided into (1) to (5) as the self-position estimation apparatus according to the fourth embodiment.

(1) Weighting In the third embodiment, a likelihood model is selected from a plurality of likelihood models according to the imaging state of the landmark, and the selected likelihood model is applied with weighting according to the imaging state. However, the present invention is not limited to this, and the present invention is not limited to this. The given likelihood model is applied with weighting according to the imaging state of the landmark, and the score is calculated. It may be a self-position estimation device that calculates. That is, the “landmark detection image evaluation unit 37” and the “weighted voting process 39” may be the self-position estimation apparatus 10 further provided in the processing unit 30 of the first embodiment.

(2) Likelihood model In the first embodiment, the likelihood model storage unit 27 stores a likelihood model in which the likelihood is defined by a single-release curve between the maximum narrow angle and the minimum narrow angle. Although the position estimation device 10 has been described, the present invention is not limited to this, and even if the likelihood model defines the likelihood with a uniform straight line between the maximum narrow angle and the minimum narrow angle, It may be a likelihood model.

  In the second and third embodiments, a likelihood model P1 in which the likelihood is defined by a single-release curve between the maximum narrow angle and the minimum narrow angle in the likelihood model storage unit 27, and the maximum narrow angle. Although the self-position estimation apparatus 10 in which two likelihood models of the likelihood model P2 in which the likelihood is defined by a uniform straight line between the minimum angle and the minimum narrow angle has been described, the present invention is limited to this. Instead, two or more likelihood models replaced with or added to another likelihood model may be stored. Here, another likelihood model is the shape of a landmark (such as a rounded building or a square-pyramid TV tower) and how the landmark looks (such as the right half of the tree being covered by a shield such as a tree) It is a likelihood model according to.

(3) Applicable object In the above-described embodiment, the vehicle equipped with the self-position estimating device configured to include the self-position estimating program according to the present invention has been described as an applicable object. However, the present invention is not limited to this. For example, it may be used as a self-position estimation device that processes a photograph in which it is unknown where the image was taken and estimates the place where the photograph was taken.

(4) System configuration etc. In addition, among the processes described in the above embodiments, all or a part of the processes described as being automatically performed can be performed manually (for example, from the keyboard or the touch panel). Instead of accepting a position estimation request, a self-position estimation request is automatically generated at regular intervals), or all or part of the processing described as being performed manually is automatically performed by a known method. You can also. For example, in addition to this, the processing procedures, specific names, and information including various data and parameters shown in the drawings (for example, information shown in FIG. 9) are arbitrarily changed unless otherwise specified. be able to.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. That is, the specific form (for example, the form of FIG. 2) of distribution and integration of each processing unit and each storage unit is not limited to the illustrated one. For example, the voting processing unit 35 and the self-position estimation unit 36 are integrated. All or some of them can be configured to be functionally or physically distributed / integrated in arbitrary units according to various loads or usage conditions. Further, all or any part of each processing function performed in each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

(5) Self-position estimation program In the above first to third embodiments, the case where various processes are realized by hardware logic has been described. However, the present invention is not limited to this, and a program prepared in advance. May be executed on a computer. Therefore, in the following, an example of a computer that executes a self-position estimation program having the same function as the self-position estimation apparatus 10 shown in the first embodiment will be described with reference to FIG. FIG. 24 is a diagram illustrating the computer that executes the self-position estimation program according to the first embodiment.

  As shown in FIG. 24, a computer 240 serving as an information processing device is configured by connecting a keyboard 241, a display 242, a CPU 243, a ROM 244, an HDD 245, and a RAM 246 via a bus 247, and further, the car navigation output device 40 and the collision avoidance vehicle control. Connected to device 50.

  In the ROM 244, a self-position estimation program that exhibits the same function as the self-position estimation apparatus 10 shown in the first embodiment, that is, as shown in FIG. 24, a landmark detection program 244a, a landmark search program 244b, A narrow angle calculation program 244c, a constrained circle calculation program 244d, a voting program 244e, and a self-position estimation program 244f are stored in advance. Note that these programs 244a to 244f may be appropriately integrated or distributed in the same manner as each component of the self-position estimation apparatus 10 shown in FIG.

  Then, the CPU 243 reads out these programs 244a to 244f from the ROM 244 and executes them, so that each program 244a to 244f has a landmark detection process 243a, landmark search process 243b, narrow angle as shown in FIG. It functions as a calculation process 243c, a constraint circle calculation process 243d, a voting process 243e, and a self-position estimation process 243f. Each of the processes 243a to 243f includes the landmark detection processing unit 31, the landmark search processing unit 32, the narrow angle calculation processing unit 33, the constrained circle calculation processing unit 34, the voting processing unit 35, the self position shown in FIG. Each corresponds to the estimation unit 36.

  Further, as shown in FIG. 24, the HDD 245 is provided with a landmark information table 245a, a camera parameter table 245b, and a likelihood model table 245c. The landmark information table 245a corresponds to the landmark information storage unit 22 used in FIG. 2, the camera parameter table 245b corresponds to the camera parameter storage unit 24, and the likelihood model table 245c is stored in the likelihood model storage unit 27. Correspond. The CPU 243 registers the landmark information data 246b in the landmark information table 245a, registers the camera parameter data 246d in the camera parameter table 245b, and sets the likelihood model data 246g in the likelihood model table 245c. The landmark information data 246b, the camera parameter data 246d, and the likelihood model data 246g are read out and stored in the RAM 246. The landmark detection data 246a, the landmark search data 246c stored in the RAM 246, The self-position estimation process is executed based on the corner calculation data 246e, the constraint circle calculation data 246f, and the voting process data 246h.

  The above-described programs 244a to 244f are not necessarily stored in the ROM 244 from the beginning. For example, a flexible disk (FD), a CD-ROM, an MO disk, a DVD disk, a magneto-optical disk inserted into the computer 240, or the like. A “portable physical medium” such as a disk or an IC card, or a “fixed physical medium” such as an HDD provided inside or outside the computer 240, and further via a public line, the Internet, a LAN, a WAN, etc. Each program may be stored in “another computer (or server)” connected to the computer, and the computer 240 may read and execute each program from these programs.

(Supplementary Note 1) A narrow angle between landmarks centered on a self-position that is an imaging position of the input image is calculated for each of the plurality of landmarks included in the input image, and the calculated narrow angle is used to calculate the narrow angle. A self-position estimation program that causes a computer to execute a self-position estimation method that calculates a restraint circle constrained as a self-position and estimates the self-position,
For each of the plurality of landmarks, the maximum included angle indicating the maximum value of the included angle between the landmarks and the minimum included angle indicating the minimum value of the included angle from the farthest distance and the nearest distance between the landmarks. A procedure for calculating the included angle,
For each of the plurality of landmarks, based on the maximum included angle and the minimum included angle calculated by the included angle calculation procedure, the maximum narrow angle constrained circle determined from the maximum narrow angle and the minimum narrow angle determined from the minimum narrow angle. A constraint circle calculation procedure for calculating an angle constraint circle;
Each angle between the maximum included angle and the minimum included angle in a region surrounded by the maximum included angle restricted circle and the minimum included angle restricted circle calculated by the restricted circle calculation procedure for each of the plurality of landmarks. Applying a likelihood model that defines the likelihood as the narrow angle in, to calculate a score indicating the possibility as the self position for each coordinate position in the region, and for each of the plurality of landmarks A score calculation procedure for adding the calculated score for each coordinate position;
Based on the score calculated and added by the score calculation procedure, a self-position estimation procedure for estimating the coordinate position where the score is maximum as the self-position,
A self-position estimation program that causes a computer to execute.

(Additional remark 2) The said score calculation procedure calculates the said score by applying the likelihood model which prescribed | regulated the said likelihood with the single-release curve between the said maximum included angle and the minimum included angle, It is characterized by the above-mentioned. The self-position estimation program according to appendix 1.

(Additional remark 3) The said score calculation procedure calculates the said score by applying the likelihood model which prescribed | regulated the said likelihood with the uniform straight line between the said maximum included angle and the minimum included angle. The self-position estimation program according to 1.

(Additional remark 4) The said score calculation procedure selects the likelihood model according to the said imaging state from the some likelihood models according to the imaging state of the said landmark in the said input image, The said selected likelihood The self-position estimation program according to appendix 1, wherein the score is calculated by applying a model.

(Supplementary Note 5) When the landmark is captured in the input image in the score calculation procedure, the likelihood is expressed as a single-release curve between the maximum and minimum included angles. The score is calculated by applying a prescribed likelihood model, and the likelihood is uniform between the maximum included angle and the minimum included angle when the imaging state of the landmark in the input image is not good. The self-position estimation program according to appendix 1, wherein the score is calculated by applying a likelihood model defined by a straight line.

(Additional remark 6) The said score calculation procedure applies the said likelihood model with the weighting according to the said imaging state according to the imaging state of the said landmark in the said input image, It is characterized by the above-mentioned. The self-position estimation program according to attachment 1.

(Additional remark 7) The narrow angle between each landmark centering | focusing on the self position which is the imaging position of the said input image is calculated for every some landmark included in an input image, respectively, and the said narrow angle is calculated from each said calculated narrow angle. A self-position estimation method for estimating a self-position by calculating a restraint circle constrained as a self-position,
For each of the plurality of landmarks, the maximum included angle indicating the maximum value of the included angle between the landmarks and the minimum included angle indicating the minimum value of the included angle from the farthest distance and the nearest distance between the landmarks. An included angle calculating step to calculate,
For each of the plurality of landmarks, based on the maximum included angle and the minimum included angle calculated by the included angle calculating step, the maximum narrow angle constrained circle determined from the maximum narrow angle and the minimum narrow angle determined from the minimum narrow angle. A constraint circle calculation step for calculating an angle constraint circle;
Each angle between the maximum included angle and the minimum included angle in a region surrounded by the maximum included angle restricted circle and the minimum included angle restricted circle calculated by the restricted circle calculating step for each of the plurality of landmarks. Applying a likelihood model that defines the likelihood as the narrow angle in, to calculate a score indicating the possibility as the self position for each coordinate position in the region, and for each of the plurality of landmarks A score calculation step of adding the calculated score for each coordinate position;
Based on the score calculated and added by the score calculation step, a self-position estimation step for estimating the coordinate position where the score is maximum as the self-position,
A self-position estimation method comprising:

(Additional remark 8) The narrow angle between each landmark centering on the self-position which is the imaging position of the said input image is calculated for every some landmark included in an input image, respectively, and the said narrow angle is calculated from each said calculated narrow angle. A self-position estimation device that calculates a restraint circle constrained as a self-position and estimates the self-position,
For each of the plurality of landmarks, the maximum included angle indicating the maximum value of the included angle between the landmarks and the minimum included angle indicating the minimum value of the included angle from the farthest distance and the nearest distance between the landmarks. An included angle calculating means for calculating;
Based on the maximum included angle and the minimum included angle calculated by the included angle calculating means for each of the plurality of landmarks, the maximum narrow angle constrained circle determined from the maximum narrow angle and the minimum narrow angle determined from the minimum narrow angle A constraint circle calculation means for calculating an angle constraint circle;
Each angle between the maximum included angle and the minimum included angle in an area surrounded by the maximum included angle restricted circle and the minimum included angle restricted circle calculated by the restricted circle calculating means for each of the plurality of landmarks. Applying a likelihood model that defines the likelihood as the narrow angle in, to calculate a score indicating the possibility as the self position for each coordinate position in the region, and for each of the plurality of landmarks Score calculating means for adding the calculated score for each coordinate position;
Based on the score calculated and added by the score calculation means, self-position estimation means for estimating the coordinate position where the score is maximized as the self-position,
A self-position estimation apparatus comprising:

  As described above, the self-position estimation program, the self-position estimation method, and the self-position estimation apparatus according to the present invention center around the self-position that is the imaging position of the input image for each of the plurality of landmarks included in the input image. It is useful for calculating the narrow angle between each landmark and estimating the self position by calculating the constrained circle constrained as the self position from each calculated narrow angle. Suitable for improving.

It is a figure for demonstrating the outline | summary and the characteristic of the self-position estimation apparatus in Example 1. FIG. It is a block diagram which shows the structure of the self-position estimation apparatus in Example 1. FIG. It is a figure for demonstrating a landmark detection result memory | storage part. It is a figure for demonstrating a landmark information storage part. It is a figure for demonstrating a landmark search result memory | storage part. It is a figure for demonstrating an included angle calculation process part. It is a figure for demonstrating an included angle calculation result memory | storage part. It is a figure for demonstrating a constrained circle calculation process part. It is a figure for demonstrating a restricted circle calculation result storage part. It is a figure for demonstrating a likelihood model memory | storage part. It is a figure for demonstrating a voting process part. It is a figure for demonstrating a voting process result memory | storage part. It is a figure for demonstrating a self-position estimation part. It is a figure for demonstrating the process of the self-position estimation apparatus in Example 1. FIG. It is a figure for demonstrating the outline | summary and the characteristic of the self-position estimation apparatus in Example 2. FIG. It is a block diagram which shows the structure of the self-position estimation apparatus in Example 2. FIG. It is a figure for demonstrating the selection likelihood model stored in the likelihood model memory | storage part. It is a figure for demonstrating a landmark detection image evaluation result memory | storage part and a selection likelihood model memory | storage part. It is a figure for demonstrating the process of the self-position estimation apparatus in Example 2. FIG. It is a figure for demonstrating the outline | summary and the characteristic of the self-position estimation apparatus in Example 3. FIG. It is a block diagram which shows the structure of the self-position estimation apparatus in Example 3. It is a figure for demonstrating a weighting vote process part. It is a figure for demonstrating the process of the self-position estimation apparatus in Example 3. FIG. It is a figure which shows the computer which performs the self-position estimation program of Example 1. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Self-position estimation apparatus 11 Camera image input part 12 Self-estimation position output part 13 Input / output control I / F part 20 Storage part 21 Landmark detection result storage part 22 Landmark information storage part 23 Landmark search result storage part 24 Camera parameter Storage unit 25 Angle calculation result storage unit 26 Constrained circle calculation result storage unit 27 Likelihood model storage unit 28 Voting process result storage unit 30 Processing unit 31 Landmark detection processing unit 32 Landmark search processing unit 33 Angle calculation processing unit 34 Constraint circle calculation processing unit 35 Voting processing unit 36 Self-position estimation unit 40 Car navigation output device 50 Collision avoidance vehicle control device

Claims (5)

For each landmark included in the input image, a narrow angle between each landmark centered on the self-position that is the imaging position of the input image is calculated, and constrained as the self-position from each calculated narrow angle A self-position estimation program for causing a computer to execute a self-position estimation method for calculating a restrained circle to be estimated and estimating the self-position,
For each of the plurality of landmarks, the maximum included angle indicating the maximum value of the included angle between the landmarks and the minimum included angle indicating the minimum value of the included angle from the farthest distance and the nearest distance between the landmarks. A procedure for calculating the included angle,
For each of the plurality of landmarks, based on the maximum included angle and the minimum included angle calculated by the included angle calculation procedure, the maximum narrow angle constrained circle determined from the maximum narrow angle and the minimum narrow angle determined from the minimum narrow angle. A constraint circle calculation procedure for calculating an angle constraint circle;
Each angle between the maximum included angle and the minimum included angle in a region surrounded by the maximum included angle restricted circle and the minimum included angle restricted circle calculated by the restricted circle calculation procedure for each of the plurality of landmarks. Applying a likelihood model that defines the likelihood as the narrow angle in, to calculate a score indicating the possibility as the self position for each coordinate position in the region, and for each of the plurality of landmarks A score calculation procedure for adding the calculated score for each coordinate position;
Based on the score calculated and added by the score calculation procedure, a self-position estimation procedure for estimating the coordinate position where the score is maximum as the self-position,
A self-position estimation program that causes a computer to execute.   The score calculation procedure is characterized in that the score is calculated by applying a likelihood model in which the likelihood is defined by a single-release curve between the maximum included angle and the minimum included angle. The self-position estimation program described in 1.   The score calculation procedure calculates the score by applying a likelihood model in which the likelihood is defined by a uniform straight line between the maximum included angle and the minimum included angle. Self-localization program.   In the score calculation procedure, when the imaging state of the landmark in the input image is good, the likelihood defined by the single-release curve between the maximum included angle and the minimum included angle The score is calculated by applying a model, and when the landmark imaging state in the input image is not good, the likelihood is defined by a uniform straight line between the maximum included angle and the minimum included angle The self-position estimation program according to claim 1, wherein the score is calculated by applying a likelihood model.   2. The score calculation procedure according to claim 1, wherein the score is calculated by applying the likelihood model with weighting according to the imaging state in accordance with an imaging state of the landmark in the input image. The described self-location estimation program.

Images