JP2011215972A - Image processing system and position measurement system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system suitable for creation of effective image data for reference for a scenery image recognition technology for positioning.SOLUTION: The image processing system includes: a feature point extraction part for extracting an image feature point from a captured image of a scene viewed from a vehicle; an image feature point data generation part for generating the image feature point data of each captured image from the image feature point based on a weighting factor assigned to the image feature point; an adjustment factor setting part for setting an adjustment factor for adjusting the weighting factor applied to the image feature point in accordance with the distribution state of the image feature point data in the captured image; and a reference data database generation part for generating the database of reference data to be used for scenery image recognition by associating the image feature point data with image-capturing attribute information including the image-capturing position of the captured image corresponding to the image feature point data.

Description

  The present invention relates to an image processing system, and more particularly to an image processing system for creating reference data for landscape image recognition processing, and a position positioning system using the reference data.

  Conventionally, in a car navigation apparatus, as a method of calculating the current position of a vehicle, a method using information acquired from a sensor such as a gyroscope or a geomagnetic sensor (autonomous navigation), a method using a signal from a GPS satellite, or an autonomous vehicle A method of combining navigation and GPS is adopted. Furthermore, in order to calculate the current position with high accuracy, the provisional current position is obtained using a signal from a positioning satellite, and the provisional current position is used as a reference using the captured image in front of the vehicle. The coordinate of the feature point of the road marking (automotive coordinate system feature point) in the coordinate system (car coordinate system) is calculated, and the calculated coordinate point of the car coordinate system and the coordinate of the feature point of the stored road sign (world 2. Description of the Related Art Position positioning devices configured to calculate the current position of a vehicle using a coordinate system) are known (for example, see Patent Document 1). In this apparatus, even if positioning based on signals from positioning satellites and signals from various sensors includes an error, it is possible to calculate a current position with high accuracy.

JP 2007-208043 (paragraph number 0009-0013, FIG. 1)

  However, in the positioning device according to Patent Document 1, the spatial coordinates of the feature points of the road marking on the road are obtained from the stereo image, and are obtained from the latitude / longitude of the road marking having the feature point stored in the road marking information database. Since the vehicle position is calculated using the coordinates obtained, it cannot be used in a place where there is no road marking. Further, since it is necessary to calculate the spatial coordinates of the feature points recognized by the image processing, the apparatus is required to have a high calculation capability, which causes a cost increase.

  Therefore, it is possible to use the landscape image recognition technique as a positioning that can be used on a road without a road sign or in a specific site, and does not have to calculate the spatial coordinates of each feature point for each position calculation. At that time, since it is important to create reference data that is effective reference image data used in landscape image recognition technology, an image processing system suitable for creating such effective reference data and such Realization of a positioning system using reference data is desired.

  A feature configuration of the image processing system according to the present invention includes a data input unit that inputs a captured image obtained by capturing a landscape from a vehicle, a feature point extraction unit that extracts an image feature point from the input captured image, and the image A weighting unit for assigning a weighting factor to a feature point; an image feature point data generating unit for generating image feature point data for each captured image from the image feature point based on the weighting factor; and the image feature point data for the image A reference data database creation unit that creates a database as reference data used for landscape image recognition in association with photographing attribute information including a photographing position of the photographed image corresponding to feature point data; and An adjustment coefficient setting unit for setting an adjustment coefficient for adjusting a weighting coefficient given to the image feature point according to a distribution state of the image feature point data. It lies in the fact that.

  According to this feature configuration, the image feature point data composed of the image feature points extracted for each photographed image from the photographed image that is a landscape from the vehicle is associated with the photographing position that is also the own vehicle position at the time of photographing. Therefore, the image feature point data is used as reference data for landscape image recognition. At that time, different weighting factors are assigned to the extracted image feature points according to the importance of the image feature points. The degree of importance for image feature points varies depending on the type of landscape being photographed and the positional relationship with the traveling lane, and it is difficult to determine accurately. Therefore, there is a possibility that a low weighting coefficient is assigned to an image feature point that is originally highly important. According to this feature configuration, the weighting coefficient assigned by the weighting unit is adjusted by the adjustment coefficient setting unit according to the distribution state of the image feature point data in the captured image. In other words, from the knowledge of the present inventor that some of the image feature points with high importance are likely to be randomly located in the captured image, the weight that increases the weight because of the high importance. In order to suppress inconvenience due to the uneven distribution of image feature points to which coefficients are assigned, the weight coefficients are adjusted according to the distribution state of the image feature points in the captured image. Then, the image feature points are organized based on the weighting coefficient adjusted by the adjustment coefficient, and image feature point data for each captured image is generated. Further, the image feature point data is associated with shooting attribute information including the shooting position of the shot image corresponding to the image feature point data, and reference data used for landscape image recognition is created. As a result, the construction of a reference data database composed of reference data groups for realizing effective landscape image recognition is realized.

  In order to simplify the process of assigning weighting factors to image feature points, instead of assigning weighting factors individually to image feature points, the captured image is divided into a plurality of sections, and the weighting coefficients for each section It is preferable that the image feature points belonging to each section have the same weight coefficient. In order to achieve this, in one of the preferred embodiments of the present invention, the shooting area of the shot image is further divided into a plurality of image sections, and the distribution state of the image feature points is the minimum unit of the image sections. It is configured to be evaluated as That is, it is intended that the image feature points belonging to the same image section are grouped as an image feature point group and handled in a unified manner, and the grouped image feature point group exists in all image sections as much as possible.

  As one specific embodiment for avoiding uneven distribution of image feature points to which weighting factors that increase in weight due to high importance are assigned, the weighting factor after adjustment by the adjustment factor is predetermined. It is proposed that the adjustment coefficient setting unit sets the adjustment coefficient so that the distribution of the image feature point data that is greater than or equal to the value becomes uniform. With this configuration, it is possible to disperse image feature points having a weighting coefficient equal to or larger than a predetermined value as uniformly as possible without being unevenly distributed in a region corresponding to a captured image.

  Furthermore, since image feature points that can be stably detected at points on the image are preferable, generally edge points that can be detected using an edge detection filter or a Harris operator are used. In the present invention, considering points of feature points in a landscape image, a group of edge points connected on a straight line indicating the outline of a building, the window of a building, and the outlines of various signs are suitable. Therefore, in a preferred embodiment of the present invention, when the image feature point extracted by the feature point extraction unit is an edge point, and the edge point is a linear component edge point forming a straight line, the linear component The edge point is configured to be given a higher weighting coefficient than edge points other than the linear component edge point. According to this configuration, it is possible to create reference data for accurately and simply recognizing a specific artifact such as a building or a signboard that characterizes a landscape. At this time, it is preferable that an intersection edge point as an intersection of two linear components among the linear component edge points is given a higher weighting factor than linear component edge points other than the intersection edge point. . As a result, the image feature points included in the reference data can be limited to only the corner points, which are the most important feature points of buildings, bridges, signboards, etc., that is, the intersection edge points, thereby reducing the computational burden in image recognition. It becomes. A Harris operator or the like can be used to detect a corner point.

  The weighting coefficient assigned to the image feature points by the weighting unit can be discarded after being used for organizing the image feature points when generating the image feature point data. However, when performing landscape image recognition through matching processing with image feature point data extracted from captured images acquired in real time using reference data constructed as a database by this image processing system, the reference data Depending on the usage form, it may be more convenient to attach a weighting coefficient to the reference data, that is, to associate them with each other. For this reason, in one embodiment of the present invention, each of the image feature points constituting the reference data is associated with a weighting factor given to the image feature point.

When the shooting direction is included in the shooting attribute information including the shooting position of the shot image from which the image feature point is extracted and finally related to the image feature point data, the shooting direction is specified and It is convenient that the image feature point data can be narrowed down.
Furthermore, the present invention also includes a position positioning system that determines the vehicle position through matching processing using reference data created by the above-described image processing system. Such a positioning system includes a reference data database storing the reference data, a data input unit for inputting a photographed image obtained by photographing a landscape from a vehicle, and extracting image feature points from the inputted photographed image. A feature point extraction unit, a captured image processing unit that generates image feature point data for each captured image from the image feature points, and outputs the image feature point data as matching data; reference data extracted from the reference data database; A scenery matching unit that performs matching with data and determines a vehicle position based on a shooting position related to the reference data that has been successfully matched. In this position positioning system, as described above, since the reference data effective for landscape matching is used, the vehicle position can be determined satisfactorily.

  Further, in such a position positioning system, the landscape matching unit may select the extracted reference data based on the weighting coefficient after adjustment by the adjustment coefficient and use it for matching with the matching data. Is preferred. With this adjustment coefficient, the weight coefficient is appropriately adjusted according to the distribution of the image feature point data in the photographed image. Therefore, using the weight coefficient after adjustment by this adjustment coefficient, an image suitable for matching from the reference data Feature point data can be selected. Thereby, more efficient matching processing is realized.

It is a schematic diagram explaining the basic concept of the positioning technology which determines the own vehicle position through preparation of the reference data by the image processing system by this invention, and the matching process using the reference data. It is a functional block diagram which shows the main functions in an example of the image processing system by this invention. It is a schematic diagram which shows typically the adjustment of the weighting coefficient using an adjustment coefficient. It is a functional block of the car navigation system using the reference data DB created by the image processing system according to the present invention.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 shows a position where a landscape image is photographed by recognizing a landscape image from an in-vehicle camera through a matching process using reference data created by the image processing system according to the present invention, that is, the position of the vehicle. The basic concept of the positioning technology to be determined is schematically shown.

  First, the construction procedure of the reference data database 92 (hereinafter simply referred to as “reference data DB”) will be described. As shown in FIG. 1, a shooting image obtained by shooting a landscape from a vehicle in the middle of traveling and shooting attribute information including a shooting position and shooting direction at the time of shooting are input (# 01). A feature point detection process for detecting an image feature point, for example, an edge detection process is performed on the input photographed image (# 02). Here, an edge point corresponding to one pixel or a plurality of pixels constitutes a line segment such as an outline is called an edge, and an intersection point where a plurality of edges intersect is called a corner. However, an example of the image feature point is the edge and the corner. An edge including a corner is extracted as an image feature point from the edge detection image obtained by the edge detection process (# 03).

  In consideration of being used for landscape image recognition, not all extracted image feature points have the same importance. For example, the importance of each image feature point may differ depending on the coordinate position in the corresponding captured image. For this reason, it is important for obtaining image capturing information including such condition information (# 04) to reduce the importance of image feature points that are not suitable for landscape image recognition and for landscape image recognition. A rule for increasing the importance of the image feature point is selected based on the shooting situation information, and the importance of each image feature point is determined using the selected rule (# 05). When the importance of each image feature point is determined, a weighting coefficient matrix that defines the assignment of the weighting coefficient to each image feature point according to the importance is generated (# 06a). Further, the adjustment coefficient for adjusting the weighting coefficient set in this weighting coefficient matrix is set by another rule created from a viewpoint different from the rule for determining the importance level (# 06b). .

  Subsequently, the weighting coefficient matrix is adjusted based on the adjustment coefficient to adjust the weighting coefficient (# 06c), and the image feature points are organized using the weighting coefficient adjusted in this way, so that an image for each captured image is obtained. Feature point data is generated (# 07). In this image feature point data generation process, image feature points having a weighting factor equal to or lower than a predetermined threshold level are discarded, or image feature points having a weighting factor equal to or higher than a predetermined threshold level and surrounding image feature points Sorting process to discard other than is performed. Since the image feature point data generated here is used as a pattern when pattern matching is adopted for the purpose of landscape image recognition, it is necessary to provide only image feature points effective in pattern matching of landscape images. Important for speed and accuracy. The generated image feature point data is associated with the shooting position and shooting direction of the shot image corresponding to the image feature point data, and becomes database data that can use the shooting position and shooting direction as a search key (# 08). ). That is, the image feature point data is stored in the reference data DB 92 as reference data used for landscape image recognition, for example, a pattern matching pattern (# 09).

  Next, a procedure for determining the position of the vehicle (vehicle position) during actual vehicle travel using the reference data DB 92 constructed in the above-described procedure will be described. As shown in FIG. 1, an actual captured image obtained by capturing a landscape with an in-vehicle camera, and an imaging position and an imaging direction used for extracting reference data from the reference data DB 92 are input (# 11). ). The photographing position here is an estimated own vehicle position estimated using a GPS measurement unit or the like. Data for matching, which is image feature point data, is generated from the input photographed image through the processing procedures of steps # 02 to # 07 (# 12). At the same time, using the input shooting position and shooting direction as search conditions, the reference data of the corresponding shooting position (estimated vehicle position) and the reference data before and after the shooting position (estimated vehicle position) are matching candidate reference data. (# 13).

  Reference data is set as a pattern one by one from the extracted matching candidate reference data sets, and pattern matching processing with the matching data generated now is executed as landscape image recognition (# 14). If the matching is successful, the shooting position associated with the reference data that is the target is read (# 15), and this shooting position is determined as the official vehicle position that replaces the estimated vehicle position (#). 16).

Next, an example of an image processing system according to the present invention that creates reference data from a captured image based on the basic concept of the positioning technique described above will be described. The function block diagram of FIG. 2 schematically shows functions of such an image processing system, particularly related to the present invention.
This image processing system includes a data input unit 51, a feature point extraction unit 52, a feature point importance degree determination unit 53, a weighting unit 55, an adjustment coefficient setting unit 54, an image feature point data generation unit 56, and a reference data database generation unit 57. , Etc., and each of these functions can be created by hardware, software, or a combination thereof.

  The data input unit 51 includes a photographic image obtained by photographing a landscape by a camera mounted on a vehicle traveling for reference data creation, photographing attribute information including a photographing position and a photographing direction at the time of photographing, and a photographing situation. Information is entered. In the form in which the image processing system is mounted on the traveling vehicle, the captured image, the photographing attribute information, and the photographing situation information are input to the input unit 51 in real time. The image processing system is a data processing center or the like. In the form installed in the system, the photographed image, the photographing attribute information, and the photographing state information are temporarily recorded on a recording medium, and these data inputs are performed in a batch process. Since the method for generating the photographed image and the photographing attribute information is well known, the description thereof is omitted.

The shooting situation information is information indicating the possibility that a specific subject is included in the shot image, and the contents included in the shooting situation information of this embodiment include travel lane data, moving object data, and area attribute data. It is. The traveling lane data is data indicating the traveling lane region of the own vehicle and the region outside the road in the photographed image obtained from the recognition result of the white line, the guide rail, and the safety zone obtained through image processing on the photographed image. The moving object data is data indicating an existing area in a captured image of a moving object present around the vehicle that is recognized by an in-vehicle sensor that detects an obstacle such as a radar. Area attribute data is data indicating area attributes such as mountain area, suburban area, urban area, high-rise city area, etc. is there.
The subject handled in the shooting situation information can be recognized by various known methods. For example, it can be detected from a shot image through an image recognition process, and various in-vehicle sensors (distance sensors, obstacle detection sensors). To process and detect the sensor signal from VICS (registered trademark) (Vehicle Information and Communication System), etc. Are known.

  The feature point extraction unit 52 extracts edge points as image feature points from the captured image using an appropriate operator. The feature point importance determination unit 53 determines the importance of the image feature points extracted by the feature point extraction unit 52 based on the contents of each data included in the shooting situation information. For example, when the content of travel lane data is used, the importance given to the region in the travel lane with respect to the image feature point belonging to the region further away from the travel lane closer to the shoulder in the captured image Give high importance. When moving object data is used, an importance level lower than an importance level assigned to a region where the moving object does not exist is assigned to an image feature point belonging to a region where the moving object exists in the captured image. Further, when the contents of the area attribute data are used, the importance assigning rule corresponding to the position in the captured image is changed according to the area attribute. For example, in a shot image of a mountain area, there is a high possibility that the upper part of the photographing center optical axis is empty and the left and right are forests. Therefore, the center area around the photographing central optical axis is higher than the other areas. Set importance. In the photographed image of the suburban area, there is little traffic of cars, and structures such as houses spread around, so a high importance is set for the area below the photographing center optical axis. In the captured image of the urban area, since there is a lot of traffic, a high importance is set for the area above the optical axis of the image capturing center. In the photographed image of the high-rise city area, since there are many elevated roads and viaducts, high importance is set for the region above the photographing center optical axis.

The weighting unit 55 assigns a weighting factor to the image feature point according to the importance determined by the feature point importance determining unit 53. High importance is set for image feature points that are considered important for accurate image recognition (pattern matching), so a large weighting factor is assigned to image feature points with high importance. Considering that image feature points with low weighting factors are not likely to be used in actual image recognition or deleted from reference data, they can be used to determine the selection of image feature points Calculated.
The adjustment coefficient setting unit 54 calculates an adjustment coefficient for changing the weighting coefficient assigned by the weighting unit 55 from the viewpoint of the distribution state in the corresponding captured image region. In other words, the importance determined based on the shooting situation information for the image feature points extracted by the feature point extraction unit 52 includes a certain degree of error, and image feature points having a certain degree of importance are also randomly generated. It is also possible that For this reason, when there is an uneven distribution of image feature points, in other words, an uneven distribution of the weighting coefficient assigned by the weighting unit 55, the adjustment coefficient setting unit 54 is used to moderate the uneven distribution. When the distribution of image feature points obtained by calculation processing indicates the uneven distribution of image feature points, the adjustment coefficient is set so that the weighting factor of image feature points belonging to the region where the density of image feature points is small is increased. The adjustment coefficient is set so that the weighting coefficient of the image feature point belonging to the region where the density of the image feature points is large is small.

  The image feature point data generation unit 56 organizes the image feature points based on the weighting coefficient assigned by the weighting unit 55 and the adjustment coefficient assigned in some cases, and obtains image feature point data for each recorded image. Generate. At this time, the image feature points can be narrowed down so that the matching process is efficiently performed by deleting the image feature points having a weighting coefficient equal to or less than a predetermined threshold value. In addition, this weighting factor is attached to the image feature point data so that it can be related to each image feature point in the reference data as it is, and for weighting similarity calculation at the time of matching processing using the reference data with the weighting factor. It may be used for.

  Here, a process of spreading image feature points as widely as possible over the entire photographed image area using the above-described adjustment coefficient will be described with reference to a schematic explanatory diagram shown in FIG. A feature point image (FIG. 3B) is generated by extracting image feature points from the photographed image (FIG. 3A), and importance is given to each image feature point of the feature point image. 3C, the importance corresponding to each image feature point is shown in the form of the importance layer corresponding to the feature point image so that the degree to which the degree is given can be schematically understood. 3D, a weighting factor is assigned to each image feature point, and in FIG.3 (d), the weighting factor is in the form of a feature point image in which image feature points are drawn such that the larger the weighting factor, the larger the point. Here, when image feature points are arranged such that image feature points assigned weighting factors equal to or less than a predetermined threshold are removed, for example, FIG. Only image feature points that are large in 3 (d) are selected. Then, the image feature points located in the lower region of the feature point image are excluded, and the distribution of the remaining image feature points is greatly uneven. The adjustment coefficient is set so as to increase the weighting coefficient of the image feature points in the region where the density of the image feature points to be selected is reduced as a result. As can be understood, in FIG. 3 (e), the adjustment coefficient group is shown in the form of an adjustment coefficient layer arranged in a matrix (here, in units of a plurality of pixel areas) so as to correspond to the feature point image. The image feature point data generation unit 56 organizes each image feature point using the weighting coefficient finally set based on such weighting coefficient and adjustment coefficient, and is shown in FIG. Image feature point data It is generated for each shadow image.

  Here, the setting of the adjustment coefficient will be described. This adjustment coefficient is a remedy in the case where a large distribution of image feature points occurs, and in order to reduce this uneven distribution, the value of the weight coefficient of the image feature points belonging to the region where the image feature points are concentrated Is made relatively small, and the value of the weight coefficient of the image feature point belonging to the region where the image feature points are sparse is made relatively large. When image feature points are selected such that image feature points with a weight coefficient equal to or greater than a predetermined threshold are left and image feature points below the predetermined threshold are discarded, the weight coefficient after adjustment with the adjustment coefficient is performed. It is necessary to make the distribution of image feature points having a value equal to or greater than a predetermined value uniform. For this reason, an adjustment coefficient for increasing the value of the weighting factor is given so that image feature points belonging to the sparse region remain as much as possible, and the weighting factor value of the image feature point belonging to the region where the image feature points are concentrated is given. Gives the adjustment factor to decrease. In order to calculate the degree of concentration of image feature points or the degree of quietness of image feature points, it is possible to use a scatter degree that is a statistical value. Scattering degree includes variance, average deviation, etc. Iterative calculation is effective by repeating selection of each image feature point and scattering degree calculation so that this spreading degree increases and the image feature point varies. It is. A simpler method for eliminating the local concentration of image feature points or the local quietness of image feature points is to divide the area where image feature points are scattered, that is, the captured image area into multiple small areas. It is proposed to divide and adjust the weighting coefficient so that the image feature points can be selected so that the density of the image feature points in each small region is equal.

  The reference data database unit 57 associates the image feature point data generated by the image feature point data generation unit 56 with the shooting attribute information of the shot image corresponding to the image feature point data, and recognizes the landscape image. Create a database as reference data to be used. The reference data stored in a database is stored in the reference data DB 92.

  In the above description, the degree of importance is determined for each image feature point, and as a result, a weighting factor is set for each image feature point. However, these processes can be performed in units of groups. is there. In that case, for example, the captured image area that is the captured area of the captured image is divided into a plurality of image sections, and the feature point importance determination unit 53 uses the image feature points belonging to the same image section as the image feature point group. The image feature points included in the group of image feature points are given the same importance, and the weighting unit 55 may similarly set the weight coefficient in units of image feature points. In addition, the image sections handled here may be handled in units of one pixel constituting the captured image, but may be handled in units of a plurality of pixels. Therefore, in the present invention, the image section is composed of one or a plurality of pixels.

  Next, an in-vehicle car navigation system that corrects the vehicle position by landscape image recognition (image feature point pattern matching) using the reference data DB 92 created by the above-described image processing system will be described. FIG. 3 is a functional block showing such a car navigation system incorporated in an in-vehicle LAN. This car navigation system includes an input operation module 21, a navigation control module 3, a vehicle position detection module 4, a photographing situation information generation unit 7, a road map database 91 (hereinafter referred to as a road map database 91) that stores the reference data DB 92 and car navigation road map data. And a database 9 having simply a road map DB).

  The navigation control module 3 includes a route setting unit 31, a route search unit 32, and a route guide unit 33. The route setting unit 31 sets, for example, a departure point such as the vehicle position, an input destination, a passing point, and traveling conditions (whether or not an expressway is used). The route search unit 32 is a processing unit that performs arithmetic processing for searching for a guide route from the departure point to the destination based on the conditions set by the route setting unit 31. The route guidance unit 33 provides appropriate route guidance to the driver by guidance display on the display screen of the monitor 12 or voice guidance by the speaker 13 according to the route from the departure place to the destination searched by the route search unit 32. It is a process part which performs the arithmetic processing for performing.

  The own vehicle position detection module 4 corrects the estimated own vehicle position obtained by the conventional position calculation by GPS and the position calculation by dead reckoning with the own vehicle position determined by landscape image recognition using the estimated own vehicle position. It has the function to do. The own vehicle position detection module 4 includes a GPS processing unit 41, dead reckoning processing unit 42, own vehicle position coordinate calculation unit 43, map matching unit 44, own vehicle position determination unit 45, captured image processing unit 5, and landscape matching unit 6. I have. A GPS measurement unit 15 that receives GPS signals from GPS satellites is connected to the GPS processing unit 41. The GPS processing unit 41 analyzes the signal from the GPS satellite received by the GPS measurement unit 15, calculates the current position (latitude and longitude) of the vehicle, and sends it to the own vehicle position coordinate calculation unit 43 as GPS position coordinate data. The dead reckoning processing unit 42 is connected to the distance sensor 16 and the azimuth sensor 17. The distance sensor 16 is a sensor that detects a vehicle speed or a moving distance of the vehicle. For example, a vehicle speed pulse sensor that outputs a pulse signal every time a drive shaft, a wheel, or the like of the vehicle rotates by a certain amount detects the acceleration of the host vehicle C. And a circuit that integrates the detected acceleration. The distance sensor 16 outputs information on the vehicle speed and moving distance as the detection result to the dead reckoning processing unit 42. The direction sensor 17 includes, for example, a gyro sensor, a geomagnetic sensor, an optical rotation sensor attached to the rotating part of the handle, a rotary resistance volume, an angle sensor attached to the wheel part, and the like, and the direction of the detection result Information is output to dead reckoning processing unit 42. The dead reckoning processing unit 42 calculates dead reckoning position coordinates based on the moving distance information and the direction information sent every moment, and sends the dead reckoning position coordinate data to the own vehicle position coordinate calculating unit 43 as dead reckoning position coordinate data. The own vehicle position coordinate calculation unit 43 performs an operation for specifying the position of the vehicle from the GPS position coordinate data and the dead reckoning position coordinate data by a known method. The calculated vehicle position information is information including measurement errors and the like, and in some cases, the vehicle position information is off the road. Therefore, the map matching unit 44 determines the vehicle position on the road indicated on the road map. Correction is performed. The vehicle position coordinates are sent to the vehicle position determination unit 45 as the estimated vehicle position.

  The photographed image processing unit 5 substantially includes most of the functional units that constitute the image processing system shown in FIG. The captured image processing unit 5 includes a data input unit 51, a feature point extraction unit 52, a feature point importance degree determination unit 53, a weighting unit 55, an adjustment coefficient setting unit 54, and an image feature point data generation unit 56. When a forward landscape photographed image from a vehicle photographed by the in-vehicle camera 14 is input to the data input unit 51, the feature point extraction unit 52 extracts an image feature point from the input forward landscape photographed image. The image feature point data generation unit 56 generates image feature point data for each forward landscape photographed image from the image feature points, and outputs the image feature point data to the landscape matching unit 6 as matching data. Note that the shooting situation information used in the feature point importance determination unit 53 is generated by the shooting situation information generation unit 7 mounted on the vehicle and sent to the shot image processing unit 5. The shooting state information generation unit 7 is connected to the in-vehicle camera 14 to generate the travel lane data, and receives the same shot image sent to the shot image processing unit 5. Traveling lane data is created by image processing the received captured image using a known algorithm. The photographing situation information generation unit 7 is connected to an obstacle detection sensor group 18 in order to create the moving object data. Moving object data is created based on sensor information from the sensor group 18. Furthermore, the shooting situation information generation unit 7 is connected to the vehicle position determination unit 45 and the database 9 in order to create the area attribute data. The shooting situation information generation unit 7 searches the database 9 using the vehicle position coordinates from the vehicle position determination unit 45 and the search key, and acquires area attributes (mountainous part, city area, etc.) of the currently traveling place. Based on this, area attribute data is created.

  The landscape matching unit 6 performs matching between the reference data extracted from the reference data DB 92 and the image feature point data output from the image feature point data generation unit 56. In other words, the landscape matching unit 6 uses the reference data extracted from the reference data DB 92 based on the estimated vehicle position sent from the vehicle position determination unit 45 as a pattern, and the image sent from the captured image processing unit 5. Perform pattern matching processing on feature point data. At this time, the scenery matching unit 6 selects the reference data extracted based on the adjusted weighting coefficient adjusted by the adjustment coefficient, and uses it for pattern matching processing with image feature point data as matching data. Since the adjusted weighting factor is appropriately adjusted by the adjustment factor according to the distribution of the image feature point data in the captured image, the weighting factor after adjustment is suitable for pattern matching processing from the reference data. By selecting the image feature point data, more efficient matching processing is possible.

  When the pattern matching is successful, the photographing position associated with the reference data that is the matching pattern is read out. This photographing position is determined as the own vehicle position and transferred to the own vehicle position determination unit 45. The own vehicle position determination unit 45 corrects the own vehicle position by replacing the transferred own vehicle position with the estimated own vehicle position. As described above, the car navigation system according to the present embodiment includes the reference data DB 92, the data input unit 51, the feature point extraction unit 52, the image feature point data generation unit 56, and the landscape matching unit 6, and the image processing system described above. This is a position positioning system that determines the position of the vehicle through pattern matching processing using reference data created by. In this position positioning system, since the efficient matching process is possible as described above, the vehicle position can be determined satisfactorily.

  This car navigation system also includes an input device 11 such as a touch panel 11 or a switch as a peripheral device, and an operation input evaluation unit 21a that changes an operation input through the input device 11 into an appropriate operation signal and transfers the operation signal to the inside. An input operation module 21, a display module 22 for displaying image information necessary for car navigation on the monitor 12, a voice generation module 23 for sending voice information necessary for car navigation from the speaker 13 or a buzzer, braking, acceleration, steering, etc. The vehicle behavior detection module 24 that detects various behaviors of the vehicle based on behavior data sent through the in-vehicle LAN is provided.

In the above-described embodiment, the image feature point is an edge point obtained by edge detection processing, particularly a line segment edge constituting one line segment or an intersection point where such a line segment intersects, preferably approximately orthogonally. A corner edge is treated as an effective image feature point. However, the present invention is not limited to such edge points as image feature points. For example, typical edge points that form geometric shapes such as circles and quadrilaterals (three points on the circumference if circles) or the center of gravity of a geometric shape and the point as the center of gravity are also effective depending on the landscape. It is used because it becomes a typical image feature point. It is also preferable to adopt edge strength as a factor for calculating importance. For example, if a line segment consists of strong edges, the start point and end point of the line segment are more important than other edge points. Can be handled as high image feature points. In addition, a specific point in a characteristic geometric shape, for example, an end point of a bilaterally symmetric object, can be handled as an image feature point having higher importance than other edge points.
Furthermore, in addition to the edge points obtained by the edge detection process, it is also possible to regard a captured image as a change in hue or saturation and to employ a point with a large change as an image feature point. Similarly, an end point of an object having a high color temperature can be handled as a highly important image feature point based on color information.
That is, the image feature points handled in the present invention are all used if they are effective for similarity determination between the reference data and the image feature amount data generated from the actual captured image, for example, pattern matching. It becomes.

In the above-described embodiment, the weighting coefficient calculated according to the importance is assigned to each image feature point, but this weighting coefficient is a coefficient corresponding to the importance, and without introducing this weighting coefficient, The importance level may be used as it is as a weighting factor.
In the above-described embodiment, the reference data stored in the reference data DB 92 is associated with the shooting position and the shooting direction (camera optical axis direction). The date and time and the weather at the time of shooting may also be related.
The photographing position may be two-dimensional data such as latitude / longitude data at least, but may be three-dimensional data in addition to height data.
Further, it is not essential to relate the shooting direction to the reference data. For example, when creating reference data and when recognizing a landscape image using this reference data, the shooting direction is not required if it is guaranteed that the road is shot with substantially the same shooting direction. It becomes.
Conversely, in the case where reference data in which the shooting direction is appropriately shifted from the reference data in one basic shooting direction can be prepared, based on the traveling direction of the vehicle calculated from information such as a direction sensor, It is also possible to set only the reference data suitable for the traveling direction as the object of landscape image recognition.
The on-vehicle camera handled in the present invention is most suitable for photographing a landscape in front of the vehicle traveling direction. However, the camera may be a camera that takes a landscape oblique to the front, or may be a camera that captures a landscape of the side or rear. That is, the photographed image handled in the present invention is not limited to only a photograph of a front landscape in the vehicle traveling direction.

  The functional units shown in the functional block diagrams used in the description of the above-described embodiments are for easy understanding, and the present invention is not limited to the divisions shown here. It is possible to freely combine function parts or further divide one function part

  The image processing system of the present invention can be applied not only to car navigation but also to a technical field that measures the current position and orientation by landscape image recognition.

3: navigation control module 4: own vehicle position detection module 41: GPS processing unit 42: dead reckoning processing unit 43: own vehicle position coordinate calculation unit 44: map matching unit 45: own vehicle position determination unit 5: captured image processing unit 51 : Data input unit 52: feature point extraction unit 53: feature point importance determination unit 54: weighting unit 55: adjustment coefficient setting unit 56: image feature point data generation unit 57: reference data database creation unit 6: landscape matching unit

Claims (9)

A data input unit for inputting a photographed image of a landscape from a vehicle;
A feature point extraction unit that extracts image feature points from the input captured image;
A weighting unit that assigns weighting factors to the image feature points;
An image feature point data generation unit that generates image feature point data for each captured image from the image feature points based on the weighting factor;
A reference data database generating unit that associates the image feature point data with shooting attribute information including a shooting position of the shot image corresponding to the image feature point data, and forms a database as reference data used for landscape image recognition When,
An adjustment coefficient setting unit that sets an adjustment coefficient for adjusting a weight coefficient given to the image feature point according to a distribution state of the image feature point data in the captured image;
An image processing system comprising:   The image processing system according to claim 1, wherein a shooting area of the shot image is divided into a plurality of image sections, and a distribution state of the image feature point data is evaluated using the image section as a minimum unit.   The adjustment coefficient setting unit sets the adjustment coefficient so that the distribution of the image feature point data in which the weighting coefficient adjusted by the adjustment coefficient is equal to or greater than a predetermined value is uniform. The image processing system described.   When the image feature point extracted by the feature point extraction unit is an edge point, and the edge point is a straight line component edge point forming a straight line, the straight line component edge point is an edge other than the straight line component edge point. The image processing system according to any one of claims 1 to 3, wherein a weighting factor higher than a point is given.   The image processing system according to claim 4, wherein a higher weighting coefficient is given to an intersection edge point as an intersection of two linear components among the linear component edge points than to a linear component edge point other than the intersection edge point.   The image processing system according to claim 1, wherein the reference data database creating unit associates a weighting factor with each of the image feature point data constituting the reference data.   The image processing system according to claim 1, wherein the shooting attribute information includes a shooting direction in addition to a shooting position of the shot image. In the position measurement system which determines the own vehicle position through the matching process using the reference data created by the image processing system according to any one of claims 1 to 7,
A reference data database storing the reference data;
A data input unit for inputting a photographed image of a landscape from a vehicle;
A feature point extraction unit that extracts image feature points from the input captured image;
A photographed image processing unit that generates image feature point data for each photographed image from the image feature points and outputs the data as matching data;
A scenery matching unit that performs matching between the reference data extracted from the reference data database and the matching data, and determines a vehicle position based on a shooting position related to the reference data that has been successfully matched. Positioning system equipped. 9. The position positioning system according to claim 8, wherein the landscape matching unit selects the extracted reference data based on the weighting coefficient after adjustment by the adjustment coefficient and uses it for matching with the matching data.

Images