JP2012099010A - Image processing apparatus and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of properly executing image recognition, even when a movable object with a movable part is included in a landscape image.SOLUTION: The image processing apparatus includes a reference database 92 for storing reference data including positional information in a reference photographic image of a movable object characteristic point group GA, a movement range database 93 for storing movement range data R including information defining a range in which the movable object characteristic point group GA can move, an actual photographic image processing part for generating actual photographic data from an actual photographic image that is a landscape image photographed by an on-vehicle camera, and an image recognition part for recognizing an object characteristic point group GC included in the actual photographic data as a movable object 101 included in the reference data if the arrangement of the object characteristic point group GC is within a range defined in the movement range data R referencing the arrangement of the movable object characteristic point group GA included in the reference data.

Description

  The present invention relates to an image processing apparatus and an image processing program that perform image recognition using an image taken by an in-vehicle camera.

  As a conventional example of the image processing apparatus, for example, there is a technique described in Patent Document 1 below. In the apparatus disclosed in Patent Document 1, a road marking in an image taken by an in-vehicle camera is recognized, a feature point of the road marking is extracted, and calculated using a signal from a GPS satellite or the like. The coordinates of the feature point in the coordinate system (automobile coordinate system) based on the estimated own vehicle position are calculated. Then, the current position of the vehicle is corrected using the coordinates of the feature points in the calculated vehicle coordinate system and the coordinates of the feature points of the road marking in the world coordinate system stored in the road marking information database. Yes.

  However, in the configuration of Patent Document 1, the spatial coordinates of the feature points of the road marking on the road are obtained from the stereo image, and are obtained from the latitude / longitude of the road marking having the feature point stored in the road marking information database. Since the vehicle position is calculated using the coordinates, it cannot be used in places where there are no road markings. Further, since it is necessary to calculate the spatial coordinates of the feature points recognized by the image processing, the apparatus is required to have a high calculation capability, which causes a cost increase.

JP 2007-108043 A

  In view of the above problems, the use of landscape image recognition technology can be considered as a method that can be used on roads and roads without road markings and that does not have to calculate the spatial coordinates of each feature point for each position calculation. . In this case, the image processing apparatus is configured to perform image recognition processing on the landscape image captured by the in-vehicle camera. The landscape image is captured, for example, like a blocking bar of a circuit breaker installed at a crossing. There is a possibility that a movable part whose position in the image may be different depending on the timing of. Therefore, there is a possibility that image recognition cannot be performed properly by simply comparing data generated from a landscape image for reference for image recognition with data generated from a landscape image captured by an in-vehicle camera. There is.

  Therefore, it is desired to realize an image processing apparatus capable of appropriately performing image recognition even when a movable object including a movable part is included in a landscape image.

  The characteristic configuration of the image processing apparatus according to the present invention is such that, for a movable object having a movable part that moves according to a certain law while maintaining a certain shape, the movable object from a reference photographed image that is a photographed image of a landscape including the movable object. The feature point group constituting the movable object at an arbitrary time point generated by extracting the plurality of feature points is defined as a movable object feature point group, and position information of the movable object feature point group in the reference captured image is obtained. A reference database for storing reference data, a movement range database for storing movement range data including information defining a range in which the movable object feature point group may move, and a photographed image of a landscape by an in-vehicle camera. An actual captured image processing unit that extracts a plurality of feature points from the actual captured image and generates actual captured data including position information of the plurality of feature points in the actual captured image; Arrangement of the target feature point group consisting of at least a part of the plurality of feature points included in the actual photographing data is the movement range data based on the arrangement of the movable object feature point group included in the reference data. And an image recognition unit that recognizes the target feature point group as the movable object included in the reference data when it is within a prescribed range.

In the present application, the “positional information” for a feature point group does not necessarily mean the coordinates of the feature point group in the image, but is a concept that includes all information regarding the position.
Further, in the present application, “recognizing an object” for a target feature point group does not necessarily mean that the target feature point group recognizes even the type of the object, and the target feature point group is referred to It is a concept that includes recognizing that some object included in the data is formed.

According to this characteristic configuration, even when the position of the movable part included in the movable object is different between when the reference captured image is captured and when the actual captured image is captured, the movable image included in the actual captured image is included. When the object is a movable object including a movable part that moves according to a certain law while maintaining a certain shape, the movable object can be appropriately recognized.
In addition, since the movable object is recognized based on the movement range data including information that defines the range in which the movable object feature point group may move, it is possible to suppress erroneous recognition of different objects. Moreover, compared with the structure provided with a plurality of reference data according to the movement of the movable part, it is possible to reduce the labor and cost of collecting reference data and to reduce the amount of reference data provided in the reference database. . Therefore, the configuration of the image processing apparatus can be simplified.

  Here, the reference data further includes position information of the fixed object feature point group, which is a feature point group constituting a fixed object that does not move, in the reference captured image, and the image recognition unit includes the actual captured data. If the arrangement of the target feature point group consisting of at least a part of the plurality of feature points included in the image matches the arrangement of the fixed object feature point group included in the reference data, the target feature point group is It is preferable to recognize the fixed object included in the reference data.

  In the present application, “matching” with respect to the arrangement of feature points is not limited to the case where they completely match, but the two feature points to be compared are shifted within a range that can be regarded as constituting the same object. It is used as a concept that includes

  According to this configuration, since the fixed object can be recognized in addition to the recognition of the movable object, image recognition of the actual captured image can be performed more appropriately.

  Further, the reference database includes a plurality of the reference data, and stores each of the plurality of reference data in association with the shooting position of the standard shot image corresponding to each reference data. Based on a recognition result, a matching execution unit that performs matching between the actual photographing data and the reference data, and the reference data that has been successfully matched by the matching execution unit as identification reference data, and the association with the identification reference data It is preferable to further include a photographing position determining unit that determines the photographing position of the actual photographed image corresponding to the actual photographing data based on the photographing position.

  According to this configuration, since the shooting position of the actual shot image by the in-vehicle camera can be appropriately determined, the vehicle in which the in-vehicle camera is mounted with the same accuracy as the shooting position of the standard shot image associated with the reference data It becomes possible to determine the own vehicle position.

  The moving range database divides the movable object into a plurality of types and includes the moving range data for each type, and the image recognition unit includes the reference data and the movable object included in the reference data. It is preferable that the movable object is recognized using the moving range data corresponding to the type.

  According to this configuration, the amount of movement range data included in the movement range database can be reduced as compared with the case where movement range data is provided for each movable object, and the configuration of the image processing apparatus can be simplified. .

  The moving range data is obtained when the feature points at the ends of the feature point group arranged in a predetermined shape are fixed feature points and the feature point group is rotated around the fixed feature points by a predetermined angle. It is preferable that the movement range of the feature point group is defined.

  Alternatively, the movement range data may be preferably configured to define a movement range of the feature point group when the feature point group arranged in a predetermined shape is translated by a predetermined distance in a predetermined direction. is there.

  Alternatively, the movement range data includes a common feature point that is a common feature point at an end portion, and a feature point group including a first feature point group and a second feature point group that are arranged in a predetermined shape. The feature point at the end of the first feature point group opposite to the common feature point is a fixed feature point, the first feature point group is rotated around the fixed feature point by a predetermined angle, and the A configuration in which the movement range of the feature point group when the second feature point group is translated in accordance with the movement of the common feature point is also preferable.

  According to these configurations, for example, a breaker, signboard, etc. installed at level crossings, toll booths, parking lots, etc., are provided in a certain place and have a movable part that moves according to a certain law while maintaining a certain shape. With respect to various movable objects, it is possible to appropriately perform image recognition of the movable objects using movement range data corresponding to the movement rules of the movable part.

  The technical features of the image processing apparatus according to the present invention having the above-described configurations can also be applied to an image processing method and an image processing program. Therefore, the present invention also includes such a method and program. It can be.

  In this case, the characteristic configuration of the image processing program is that, for a movable object including a movable part that moves according to a certain law while maintaining a certain shape, the movable object is obtained from a reference photographed image that is a photographed image of a landscape including the movable object A feature point group that is generated by extracting a plurality of feature points and that constitutes the movable object at an arbitrary time point is set as a movable object feature point group, and a plurality of feature points are obtained from an actual captured image that is a captured image of a landscape by an in-vehicle camera Real captured image processing step for generating actual captured image data including positional information of the plurality of feature points in the actual captured image, and positional information of the movable object feature point group in the reference captured image A reference data extracting step for extracting the reference data from a reference database that stores reference data, and a range in which the movable object feature point group may move The moving range data extracting step for extracting the moving range data from the moving range database that stores the moving range data including the defined information, and the reference data extracted in the reference data extracting step as extraction reference data, Using the movement range data extracted in the movement range data extraction step as extracted movement range data, an arrangement of target feature points consisting of at least a part of the plurality of feature points included in the actual photographing data is as follows: The target feature point group included in the extraction reference data is included in the extraction reference data when it is within a range defined in the extracted movement range data based on the arrangement of the movable object feature point group included in the extraction reference data. An image recognition step for recognizing an object is performed by a computer.

  Naturally, this image processing program can also obtain the effects of the above-described image processing apparatus, and it is possible to incorporate some additional techniques mentioned as examples of suitable configurations thereof.

It is a schematic diagram which shows the concept of the own vehicle position determination system which concerns on embodiment of this invention. It is a mimetic diagram showing a schematic structure of a navigation device concerning an embodiment of the present invention. It is a schematic diagram which shows the flow of a process in the real picked-up image process part which concerns on embodiment of this invention. It is a schematic diagram which shows the concept of the pattern matching process which concerns on embodiment of this invention. It is a schematic diagram which shows the concept of the rotational movement range data which concerns on embodiment of this invention. It is a schematic diagram which shows the concept of the translation range data which concerns on embodiment of this invention. It is a schematic diagram which shows the concept of the rotation translation range data which concerns on embodiment of this invention. It is a schematic diagram which shows an example of the movable object which this invention makes object. It is a flowchart which shows the procedure of the imaging | photography position determination process which concerns on embodiment of this invention. It is a flowchart which shows the procedure of the pattern matching process which concerns on embodiment of this invention.

  Embodiments of the present invention will be described with reference to the drawings. Here, a case will be described as an example where the image processing device according to the present invention is provided in a navigation device and a vehicle position determination system for determining the vehicle position of a vehicle on which the navigation device is mounted is constructed. Hereinafter, with regard to the configuration of the vehicle position determination system according to the present embodiment, “schematic configuration of the vehicle position determination system”, “schematic configuration of the navigation device”, “configuration of the image processing device”, “procedure of operation processing” Details will be described in order.

1. First, the concept of the own vehicle position determination system according to the present embodiment will be described with reference to FIG. As shown in FIG. 1, a navigation device 1 including an image processing device 2 is mounted on a vehicle (hereinafter also referred to as “own vehicle”) 100. The vehicle 100 includes a camera 14 (see FIG. 2), and captures a landscape at a predetermined timing. Here, a photographed image of a landscape by the camera 14 included in the vehicle 100 is referred to as an “actual photographed image”. The predetermined timing can be set every predetermined time interval (for example, every second) or every time the vehicle 100 travels a predetermined distance (for example, 1 meter). Also, the predetermined timing can be set at a time point approaching a specific point (for example, an intersection or a pedestrian crossing). In the present embodiment, the camera 14 is a front camera that photographs the front of the vehicle 100. In the present embodiment, the camera 14 corresponds to the “vehicle camera” in the present invention.

  The image processing device 2 generates actual captured data from the actual captured image (step # 01). Here, the actual captured data is data (see FIG. 4B) generated by an actual captured image processing unit 5 described later, and specifically, a plurality of feature points (features) extracted from the actual captured image. This is data including position information of the point group G) in the actual photographed image.

  The navigation device 1 includes a reference database 92 that stores a plurality of reference data. Here, the reference data is data (see FIG. 4 (e)) that is a target of pattern matching (hereinafter sometimes simply referred to as “matching”) with actual captured data. , Data including position information in the reference photographed image of the feature point group G constituting the object (movable object 101, fixed object 102, etc.). Here, the “standard photographed image” is an image from which reference data is generated, and is a photographed image of a landscape photographed for the purpose of generating reference data. The reference data is stored in the reference database 92 in association with the shooting position of the standard shot image corresponding to the reference data.

  As will be described later, the vehicle 100 has a function of calculating the estimated position of the own vehicle (estimated own vehicle position). However, the vehicle 100 calculates the shooting position of the reference shot image associated with the reference data. It is determined with higher accuracy than the estimated vehicle position. Such a standard captured image is, for example, a captured image of a landscape by a data collection vehicle that travels for the purpose of generating reference data.

  The image processing apparatus 2 extracts, as matching candidate reference data, reference data in which the associated shooting position is located in the vicinity of the estimated host vehicle position from the reference database 92 based on the estimated host vehicle position at the time of shooting the actual shot image. (Step # 02).

  Then, pattern matching processing is executed between the actual image data and the reference data included in the matching candidate reference data (step # 03), and the reference data that has been successfully matched is used as identification reference data. The image processing device 2 reads the shooting position associated with the identification reference data (step # 04), and determines the vehicle position based on the shooting position (step # 05).

  Note that the image processing apparatus 2 according to the present embodiment is characterized by the pattern matching process in step # 03. Although the details will be described later, the image processing apparatus 2 does not succeed in pattern matching just by comparing the actual captured data and the reference data as they are. In other words, the actual captured data and the reference data do not match. Even so, when a specific condition is satisfied, it is determined that the pattern matching between the actual photographing data and the reference data is successful.

  Specifically, the arrangement of a feature point group (“target feature point group GC” to be described later) composed of at least a part of feature points included in actual photographing data is composed of at least a part of feature points included in the reference data. If it is within the range defined by the movement range data R based on the arrangement of the feature point group (“movable object feature point group GA” described later), the target feature point group GC is the movable object feature point group GA. It is determined that the same object as that formed by is configured. The moving range data R is stored in the moving range database 93. As a result, even when the movable object 101 (in the example shown in FIG. 1, the circuit breaker 10 having a blocking bar (movable part)) is included in the actual captured image, the movable object 101 is appropriately set. It is possible to recognize.

  In the example shown in FIG. 1, the extracted movement range data R defines a movement range related to rotational movement, and when the movable object feature point group GA is rotated, the arrangement of the movable object feature point group GA and the target feature point The arrangement of the group GC matches. Therefore, it can be determined that the target feature point group GC constitutes the same object as that formed by the movable object feature point group GA. In such a case, pattern matching between the actual shooting data and the reference data is performed. Is determined to be successful. As a result, even if the position of the movable object 101 (movable part) is different between the shooting of the reference shooting image and the shooting of the actual shooting image, the same shooting position as the shooting position of the actual shooting image is obtained. The associated reference data can be specified, and the vehicle position of the vehicle 100 can be corrected appropriately.

2. Schematic Configuration of Navigation Device Next, a schematic configuration of the navigation device 1 that constructs the vehicle position determination system described above will be described with reference to FIG. As shown in FIG. 2, the navigation device 1 includes a navigation control module 3, a vehicle position detection module 4, a road map database 91, a reference database 92, and a movement range database 93.

  As will be described later, each of the navigation control module 3, the vehicle position detection module 4, the actual captured image processing unit 5 provided in the vehicle position detection module 4, and the landscape matching unit 6 provided in the vehicle position detection module 4 includes a plurality of The functional part is provided. Each of these functional units includes hardware or software (programs) for performing various processes on the input data with a common processing unit such as a CPU as a core member. ) Or both. Each of these functional units is configured to be able to exchange information with each other via a communication line such as a digital transfer bus, and is configured to be able to extract data from each of the databases. Here, when each function part is comprised with software (program), the said software is memorize | stored in memory | storage means, such as RAM and ROM which an arithmetic processing unit can refer.

  The road map database 91, the reference database 92, and the movement range database 93 are configured by storage devices that are common to each other or independent of each other. This storage device includes a recording medium capable of storing and rewriting information, such as a hard disk drive and a flash memory, as a hardware configuration.

  The navigation control module 3 includes a route setting unit 31, a route search unit 32, and a route guide unit 33. The route setting unit 31 sets, for example, a departure point such as the vehicle position, an input destination, a passing point, and traveling conditions (whether or not an expressway is used). The route search unit 32 is a processing unit that performs calculation processing (route search processing) for searching for a guidance route from the departure point to the destination based on the conditions set by the route setting unit 31. The route guidance unit 33 operates according to the route from the departure place to the destination searched for by the route search unit 32 by guidance display on a display screen of a monitor (not shown) or voice guidance by a speaker (not shown). It is a process part which performs the arithmetic processing (route guidance process) for performing suitable route guidance with respect to a person. The road map database 91 stores map data that is referred to when the above route search processing and route guidance processing are executed, when map display processing on a monitor is executed, and the like.

  The own vehicle position detection module 4 has a function of correcting the estimated own vehicle position obtained by the conventional position calculation by GPS and the position calculation by dead reckoning according to the result of landscape image recognition. The own vehicle position detection module 4 includes an actual captured image processing unit 5, a landscape matching unit 6, a GPS processing unit 41, a dead reckoning processing unit 42, an own vehicle position coordinate calculation unit 43, a map matching unit 44, and an own vehicle position determination unit. 45. In the present embodiment, as shown in FIG. 2, the actual captured image processing unit 5, the scenery matching unit 6, the reference database 92, and the movement range database 93 constitute the image processing device 2. As shown in FIG. 2, the vehicle 100 includes a GPS measurement unit 15, a distance sensor 16, an orientation sensor 17, and a camera 14.

  A GPS measurement unit 15 that receives GPS signals from GPS satellites is connected to the GPS processing unit 41. The GPS processing unit 41 analyzes the signal from the GPS satellite received by the GPS measurement unit 15, calculates the current position (latitude and longitude) of the vehicle 100, and sends it to the own vehicle position coordinate calculation unit 43 as GPS position coordinate data. .

  The dead reckoning processing unit 42 is connected to the distance sensor 16 and the azimuth sensor 17. The distance sensor 16 is a sensor that detects the vehicle speed and the moving distance of the vehicle 100. The distance sensor 16 outputs information on the vehicle speed and moving distance as the detection result to the dead reckoning processing unit 42. The direction sensor 17 is composed of, for example, a gyro sensor, a geomagnetic sensor, and the like, and outputs direction information as a detection result to the dead reckoning processing unit 42. The dead reckoning processing unit 42 calculates dead reckoning position coordinates based on the moving distance information and the direction information sent every moment, and sends the dead reckoning position coordinate data to the own vehicle position coordinate calculating unit 43 as dead reckoning position coordinate data.

  The own vehicle position coordinate calculation unit 43 performs an operation for specifying the position of the vehicle from the GPS position coordinate data and the dead reckoning position coordinate data by a known method. The calculated vehicle position information is information including measurement errors and the like, and in some cases, the vehicle position information is off the road. Therefore, the map matching unit 44 determines the vehicle position on the road indicated on the road map. Correction is performed. The vehicle position coordinates are sent to the vehicle position determination unit 45 as the estimated vehicle position.

  Although details will be described later, the image processing apparatus 2 performs pattern matching (landscape matching) processing based on an actual captured image that is a captured image of a landscape by the camera 14 and an estimated vehicle position at the time of capturing the actual captured image. To determine the photographing position of the actual photographed image. Then, the image processing apparatus 2 outputs information on the shooting position to the vehicle position determination unit 45. The own vehicle position determination unit 45 replaces the photographing position transferred from the image processing apparatus 2 or the position calculated based on the photographing position with the estimated own vehicle position, and corrects the own vehicle position.

3. Configuration of Image Processing Device Next, the configuration of the image processing device 2 according to the present embodiment will be described. The image processing apparatus 2 has a function of performing pattern matching processing based on an actual captured image that is a captured image of a landscape by the camera 14 and determining a shooting position of the actual captured image. As described above, the image processing apparatus 2 includes the actual captured image processing unit 5, the reference database 92, the movement range database 93, and the landscape matching unit 6. These configurations will be described below in order.

3-1. Configuration of actual captured image processing unit The actual captured image processing unit 5 extracts a plurality of feature points from an actual captured image that is a captured image of a landscape by the camera 14, and positional information of the plurality of feature points in the actual captured image. The actual photographing data including is generated. In the present embodiment, the actual captured image processing unit 5 includes a feature point extraction unit 60, a feature object recognition unit 52, an attribute determination unit 53, and an actual captured data generation unit 57, as shown in FIG. ing. Hereinafter, with reference to FIG. 3, the flow of processing in the actual captured image processing unit 5 will be described.

  The feature point extraction unit 60 extracts a pixel or a pixel group indicating an image feature suitable for landscape image recognition as a feature point from an actual captured image that is a captured image of the landscape by the camera 14. Depending on the extraction method, there are various feature points indicating image features such as those represented by image feature points, those represented by lines, and those represented by areas. In the present embodiment, the feature point extraction unit 60 extracts only the outline from the captured image, and represents the extraction result as a collection of image feature points (a feature point group). Specifically, the feature point extraction unit 60 generates an edge detection image by performing edge detection processing on the actual captured image, and uses the edge point extracted based on the edge detection image as a feature point. That is, it can be said that the feature point extraction unit 60 generates a feature point image (edge point image) formed by a plurality of feature points (edge points) extracted from the actual captured image, as shown in FIG. .

  In this example, the actual captured image is an RGB image, and is configured to detect the contour based on a luminance difference (density difference). In addition, it can also be set as the structure which detects an outline by a saturation difference or a hue difference. Further, it is possible to set the extraction condition so that the feature points extracted by the feature point extraction unit 60 do not include feature points that are inappropriate for landscape matching. As such an extraction condition, it is possible to limit the extraction of feature points by area limitation, for example. Further, for example, it is possible to limit the extraction of feature points by the magnitude of the change (gradient) of the pixel value (luminance difference in this example), that is, the edge size (edge strength).

  Based on the feature points extracted by the feature point extraction unit 60, the feature object recognition unit 52 recognizes an identifiable thing in the actual captured image as a feature object. In this example, the feature object has a geometric shape, specifically, a basic geometric shape such as a straight line, a triangle, a quadrangle, or a circle. The feature object recognition unit 52 recognizes a geometric shape defined by two or more feature points. Such a geometric shape can be recognized, for example, by recognizing a straight line or a circle by Hough transform of an image, and by recognizing a plurality of straight lines or intersections of those straight lines, a triangle (three intersections) And squares (four intersections) can also be recognized. These recognition algorithms are well known and will not be described in detail. In the example illustrated in FIG. 3, the feature object recognition unit 52 causes the straight line corresponding to the blocking bar of the breaker 10 (see FIG. 4A) and the roof of the house (building) 12 (see FIG. 4A). Corresponding triangles are recognized.

  The attribute determination unit 53 determines an object attribute (geometric shape type and feature) for specifying the geometric shape of the geometric shape recognized by the feature object recognition unit 52. Note that the type of geometric shape is information on which of the above-mentioned geometric shapes such as a straight line, triangle, quadrangle, circle, and the like, and the feature of the geometric shape is information on length, area, angle, and the like. . In this example, when the type of geometric shape is a straight line, the feature of the geometric shape includes the length of the straight line. And the attribute determination part 53 produces | generates the information of the object attribute which specifies each geometric shape as attribute information for every geometric shape. The attribute information includes an identification code (ID code) for distinguishing from other geometric shapes of the same type.

  The actual photographing data generation unit 57 obtains actual photographing data including the position information (feature point image information) of the feature points extracted by the feature point extraction unit 60 and the attribute information generated by the attribute determination unit 53. Generate. Then, the actual shooting data generated by the actual shooting data generation unit 57 is transmitted to the landscape matching unit 6.

3-2. Reference Database The reference database 92 is a database that stores reference data. The reference data is matching target data used for pattern matching processing by the landscape matching unit 6. In each reference data, as shown in FIG. 4E, at least one of the movable object feature point group GA and the fixed object feature point group GB (both in the example shown in FIG. 4E). Location information at is included.

  Here, the “movable object feature point group GA” is a reference captured image that is a captured image of a landscape including the movable object 101 with respect to the movable object 101 having a movable part that moves according to a certain law while maintaining a certain shape. 3 is a feature point group constituting the movable object 101 at an arbitrary time point generated by extracting a plurality of feature points of the movable object 101. That is, the movable object feature point group GA is a feature point group constituting the movable object 101 at a certain moment, in other words, a feature constituting the movable object 101 when the movable part is fixed at an arbitrary position within the movable range. It is a point cloud. The “movable object” targeted in the present invention includes a movable part that moves according to a certain law while maintaining a certain shape, and is an artificially manufactured object. That is, the “movable object” in the present invention does not include an object (such as a tree or a leaf) that moves in response to a meteorological phenomenon (natural phenomenon) such as wind, rain, or snow. The movement is artificially created, that is, according to artificially defined laws.

  As such a movable object 101, for example, a breaker 10 (see FIGS. 1 and 4 (d)) that is installed at a railroad crossing and includes a blocking bar as a movable part, and a rotating plate as a movable part are provided. There is a signboard 11 (see FIG. 8). The circuit breaker is not limited to the one installed at the railroad crossing. For example, the circuit breaker used in an automatic toll collection system that automatically collects tolls for vehicles, or the circuit breaker installed in a parking lot or store. Is also included. The movable object 101 may include a plurality of movable parts.

  The “movable object 101” in the present embodiment includes a fixed portion (a portion fixed directly or indirectly on the ground) that supports the movable portion. That is, the movement or movements of one or a plurality of “movable parts” included in the movable object 101 are determined by the fixed part, and the base point or range of the movement is determined, and the “movable part” in the present embodiment is constant while maintaining a certain shape. In addition to moving according to the above-mentioned law, the fixed point defines the movement base point and range. In the present invention, an object including only a movable part without including a fixed part is also included in the movable object. In addition, the movable object targeted by the present invention does not include an object that does not always move according to a certain law, such as a running vehicle.

  Further, the “fixed object feature point group GB” is a feature point group constituting the fixed object 102 that does not move, and a plurality of features of the fixed object 102 from a reference photographed image that is a photographed image of a landscape including the fixed object 102. Generated by extracting points. Examples of such a fixed object 102 include a building such as a house 12 and a building 13 (see FIG. 4D), a road sign, and the like. That is, the fixed object 102 is an object that does not include a movable part.

  In the present embodiment, the reference database 92 includes a plurality of reference data, and stores each of the plurality of reference data in association with the shooting position of the standard shooting image corresponding to each reference data. Each of the reference data includes attribute information similar to the attribute information included in the actual photographing data. As described above, since both the reference data and the actual photographing data include the attribute information, the pattern matching process between the reference data and the actual photographing data can be appropriately performed.

  In the present embodiment, the reference data includes information on whether or not the movable object 101 is included in the reference data. Further, the reference data does not include information about which feature point group G constitutes the movable object 101. However, when the reference data includes the movable object 101, the type of the movable object 101 (to be described later). ) Information is included.

  Such a reference database 92 can be constructed using an apparatus having the same configuration as that of the actual captured image processing unit 5 described above, and thus detailed description thereof is omitted here. The reference data needs to be associated with a shooting position determined with higher accuracy than the estimated host vehicle position calculated by the vehicle 100. Therefore, when generating the reference data, a landscape shot image (standard shot image) is used. ) Is required to obtain the shooting position with high accuracy.

3-3. Moving Range Database The moving range database 93 is a database that stores moving range data R including information that defines a range in which the movable object feature point group GA may move. In the present embodiment, the movable object 101 is divided into a plurality of types based on the movement rule of the movable part. The moving range database 93 includes moving range data R for each type of the movable object 101. As a result, it is possible to simplify the moving range database as compared with the configuration including moving range data for each movable object 101 while suppressing the inappropriate moving range data from being associated with the movable object 101. It has become.

  In the present embodiment, the moving range database 93 includes three movable objects 101: a movable part that rotates and moves, a movable part that moves parallel, and two movable parts that are connected to each other rotate and translate. The movement range data R (rotation movement range data, translational movement range data, and rotation translational movement range data) corresponding to the three types is provided, divided into types.

  As shown in FIG. 5, the rotational movement range data includes feature points at the end of the feature point group G arranged in a predetermined shape as fixed feature points P1, and the feature point group G around the fixed feature points P1. The movement range of the feature point group G when the rotational movement is performed by a predetermined angle θ is defined. Here, the predetermined angle θ can be set to 45 degrees, 90 degrees, and the like, for example. Further, the predetermined angle θ may be either a configuration that distinguishes the clockwise direction side and the counterclockwise direction side or a configuration that does not distinguish. As shown in FIG. 5, when the movement range data R is rotational movement range data, a predetermined angle θ is the movement range data R. That is, the rotational movement range data can be said to be data that defines the rotational angle θ around the fixed feature point P1 of the feature point group G as the movement range.

  As shown in FIG. 6, the parallel movement range data defines the movement range of the feature point group G when the feature point group G arranged in a predetermined shape is translated by a predetermined distance D in a predetermined direction. is doing. Here, the predetermined distance D is determined based on, for example, the maximum range that can be moved in the reference photographed image, for example, as a half distance of the maximum range, or a direction orthogonal to the parallel movement direction of the feature point group G For example, the distance can be determined as a distance corresponding to the length. As shown in FIG. 6, when the movement range data R is parallel movement range data, the predetermined distance D is the movement range data R. That is, the parallel movement range data can be said to be data that defines the movement distance D along the predetermined direction of the feature point group G as the movement range.

  As shown in FIG. 7, the rotational translation range data has a common feature point P2 that is a common feature point at the end, and a first feature point group G1 and a second feature point group that are arranged in a predetermined shape. For the feature point group G composed of G2, the feature point at the end opposite to the common feature point P2 of the first feature point group G1 is set as the fixed feature point P1, and the first feature point group G1 is defined as the fixed feature point P1. A range of movement of the feature point group G when the second feature point group G2 is translated around the common feature point P2 while rotating around the predetermined angle θ is defined. Here, the predetermined angle θ can be set to 45 degrees, 90 degrees, and the like, for example. Further, the predetermined angle θ may be either a configuration that distinguishes the clockwise direction side and the counterclockwise direction side or a configuration that does not distinguish. As shown in FIG. 7, when the movement range data R is rotational translation range data, the predetermined angle θ is the first movement range data R1, and the arc of the arc formed by the trajectory of the common feature point P2 is formed. The length f (θ, L) is the second movement range data R2. Here, the arc length f (θ, L) is a function of the predetermined angle θ and the length L of the first feature point group G1. That is, the rotational translation range data includes the rotation angle θ around the fixed feature point P1 of the first feature point group G1, and the arc length f determined by the rotation angle θ and the length L of the first feature point group G1. It can be said that (θ, L) is data defining the movement range.

3-4. Configuration of Scene Matching Unit The landscape matching unit 6 performs a pattern matching process between the actual shooting data sent from the actual shooting data generation unit 57 and the reference data extracted from the reference database 92, and the actual shooting data It is a functional unit that determines the shooting position of the actual captured image that is the generation source. In the present embodiment, the scenery matching unit 6 includes an image recognition unit 71, a matching execution unit 72, and a shooting position determination unit 73.

3-4-1. Configuration of Image Recognizing Unit The image recognizing unit 71 is an actual photographing sent from the actual photographing data generating unit 57 based on the estimated own vehicle position at the time of photographing the actual photographing image sent from the own vehicle position determining unit 45. Reference data (matching candidate reference data) that is a candidate for pattern matching processing with data is extracted from the reference database 92. As described above, each reference data is associated with the photographing position of the standard photographing image corresponding to each reference data, and the image recognizing unit 71 captures the associated photographing position when photographing the actual photographing image. The reference data near the estimated vehicle position is extracted as matching candidate reference data. Then, the image recognition unit 71 compares the actual shooting data with the reference data, and generates information necessary for the matching execution unit 72 to execute a pattern matching process between the actual shooting data and the reference data.

  As described above, the actual shooting data includes position information of a plurality of feature points included in the actual shooting data in the actual shooting image from which the actual shooting data is generated. Further, the reference data includes position information of a plurality of feature points included in the reference data in the standard captured image from which the reference data is generated. Then, the image recognizing unit 71 compares the arrangement of the plurality of feature points included in the actual shooting data with the arrangement of the plurality of feature points included in the reference data, and determines the plurality of feature points included in the actual shooting data. To perform image recognition processing.

  Specifically, the image recognizing unit 71 sets a feature point group G composed of at least a part of a plurality of feature points included in actual captured data as a target feature point group GC (see FIG. 4B), and It is determined whether or not a feature point group G that matches the arrangement of the target feature point group GC is included in the reference data. As described above, the actual photographing data includes the attribute information generated by the attribute determination unit 53. Similarly, the reference data includes attribute information. Therefore, the image recognition unit 71 determines whether or not the reference data includes the feature point group G that matches the arrangement of the target feature point group GC based on the attribute information in addition to the position of each feature point in the image. Determine.

  As described above, the reference data includes the position information of the movable object feature point group GA, which is the feature point group G constituting the movable object 101, in the standard captured image, and the feature point group constituting the fixed object 102. The position information of at least one of the position information in the reference photographed image of the fixed object feature point group GB is included. Then, when the arrangement of the target feature point group GC matches the arrangement of the movable object feature point group GA, the image recognition unit 71 selects the target feature point group GC as the movable object that the movable object feature point group GA constitutes. 101 (movable object 101 included in the reference data). In addition, when the arrangement of the target feature point group GC matches the arrangement of the fixed object feature point group GB, the image recognition unit 71 selects the target feature point group GC as the fixed object that the fixed object feature point group GB configures. 102 (a fixed object 102 included in the reference data).

  By the way, when the movable object 101 is included in the actual captured image, the movable object 101 (more precisely, the movable object 101 includes the movable object 101) when the actual captured image is captured and when the reference captured image is captured. Part) may be different. In such a case, even if both the shooting position and the shooting direction are the same between the actual shot image and the reference shot image, for example, as shown in FIG. The arrangement of the target feature point group GC constituting the movable object 101 is different from the arrangement of the feature point group G (movable object feature point group GA) constituting the movable object 101 in the reference data. In this case, simply comparing the arrangement of the target feature point group GC included in the actual image data with the arrangement of the movable object feature point group GA included in the reference data, the target feature point group GC and the movable object feature point Although the group GA constitutes the same movable object 101, there is a possibility that it is erroneously determined that different objects are constituted.

  In this regard, the image recognition unit 71 has a configuration for recognizing the movable object 101 included in the actual shooting data using the moving range data R described above. Specifically, the image recognition unit 71 extracts movement range data R corresponding to the type of the movable object 101 included in the reference data to be matched from the movement range database 93. Then, the image recognizing unit 71 determines that the arrangement of the target feature point group GC included in the actual shooting data is within the range defined by the movement range data R based on the arrangement of the movable object feature point group GA included in the reference data. In this case, the target feature point group GC is recognized as the movable object 101 (the movable object 101 included in the reference data) formed by the movable object feature point group GA. As described above, in this embodiment, the image recognition unit 71 recognizes the movable object 101 using the reference data and the movement range data R corresponding to the type of the movable object 101 included in the reference data. In the following description, such recognition processing is referred to as “special recognition processing”.

  In the present embodiment, when the movable object 101 is included in the reference data, and the attribute information of the target feature point group GC matches the attribute information of the movable object feature point group GA, the image recognition unit 71 The special recognition process described above is executed. As described above, in the present embodiment, although the reference data includes information on whether or not the movable object 101 is included, which feature point group G constitutes the movable object 101. No information is included. Therefore, in the present embodiment, the feature point group G included in the reference data, and the feature point group G whose arrangement does not match any feature point group G included in the actual photographing data is defined as the movable object feature point group GA. The special recognition process is performed.

  In the present embodiment, as described above, when the reference data includes the movable object 101, which feature point group G is movable although the reference data includes information on the type of the movable object 101. Information about whether the object 101 is configured is not included. Therefore, when one reference data includes a plurality of different types of movable objects 101, the image recognition unit 71 performs a plurality of types corresponding to the plurality of types when executing the special recognition process. Using one movement range data R selected from the movement range data R, the arrangement of the target feature point group GC is within the range defined in the movement range data R based on the arrangement of the movable object feature point group GA. It is determined whether or not. If the arrangement of the target feature point group GC is outside the range defined in the movement range data R based on the arrangement of the movable object feature point group GA, another movement range data R is selected and the same. It is comprised so that determination may be performed.

3-4-2. Configuration of Matching Execution Unit The matching execution unit 72 performs matching (pattern matching, landscape matching) between actual captured data and reference data based on the recognition result of the image recognition unit 71. As described above, the image recognition unit 71 determines whether or not the feature point group G that matches the arrangement of the feature point group G included in the actual captured image is included in the reference data. Then, based on the determination result of the image recognition unit 71, the matching execution unit 72 determines whether or not the pattern matching between the actual captured data and the reference data is successful.

  Here, “the pattern matching between the actual shooting data and the reference data is successful” means that the shooting position of the actual shooting image from which the actual shooting data is generated is the shooting position of the standard shooting image from which the reference data is generated. Means that it can be considered the same. Specifically, when the image recognition unit 71 is able to recognize substantially all feature point groups G included in the actual shooting data as objects included in the reference data, the matching execution unit 72 Determines that the pattern matching between the actual image data and the reference data is successful. Then, the information on the determination result of the matching execution unit 72 is transmitted to the shooting position determination unit 73.

  Here, “substantially all” means that feature point groups unsuitable for landscape image recognition are excluded. Such an inappropriate feature point group is a feature point group that constitutes an object that does not correspond to any of the “movable object” and “fixed object” in the present invention. For example, a car, a bicycle, a person, or an animal Can be mentioned.

3-4-3. Configuration of Shooting Position Determination Unit The shooting position determination unit 73 receives the determination result of the matching execution unit 72, and when the pattern matching between the actual shooting data and the reference data is successful, the actual shooting data generation source Determine the shooting position of the shot image. Specifically, reference data that has succeeded in pattern matching is used as identification reference data (see FIG. 1), and the shooting position determination unit 73 acquires a shooting position associated with the identification reference data. Then, the shooting position determination unit 73 sets the shooting position as the shooting position of the actual shot image corresponding to the actual shooting data. In this example, the shooting position determination unit 73 is configured to determine the shooting position associated with the identification reference data as it is as the shooting position of the actual shot image. However, when there are a plurality of reference data having the same degree of matching (matching degree) by the pattern matching process, for example, taking an average value based on a plurality of shooting positions associated with the plurality of reference data, It can also be configured to determine the shooting position of the actual shot image.

3-4-4. Specific Example of Moving Object Recognition Processing Next, the concept of pattern matching processing by the landscape matching unit 6 will be described with reference to a specific example shown in FIG. FIG. 4A is an actual captured image of a landscape by the camera 14, and the actual captured image includes the house 12 and the building 13 as the fixed object 102 and the two circuit breakers 10 as the movable object 101. It is. FIG. 4B shows actual captured data generated by the actual captured image processing unit 5 from the actual captured image shown in FIG. The actual photographing data shown in FIG. 4B includes four feature point groups G corresponding to the house 12, the building 13, and the two circuit breakers 10 shown in FIG. Here, assuming that the actual image data shown in FIG. 4B is sent to the landscape matching unit 6, the pattern matching process in the landscape matching unit 6 will be described. In FIG. 4B, line segments not included in the actual shooting data are also displayed in order to make it easy to understand the position of each feature point in the actual shooting image (FIG. 4A). Yes. The same applies to FIG.

  When the image recognition unit 71 receives the actual image data shown in FIG. 4B, as described above, the reference data (matching candidate reference data) that is a candidate for pattern matching processing with the actual image data is obtained. Extract from the reference database 92. Then, one reference data is selected from the matching candidate reference data as reference data to be matched. Here, it is assumed that the reference data shown in FIG. FIG. 4D shows a standard captured image that is a source of the reference data shown in FIG. As apparent from a comparison between FIG. 4A and FIG. 4D, the actual captured image and the reference captured image are images captured in the same capturing direction at the same capturing position, but the movable object 101. The position in the image of the blocking bar (movable part) included in the circuit breaker 10 is different between the actual captured image and the reference captured image. Therefore, the arrangement of the feature point groups G constituting the circuit breaker 10 (blocking bar) is also different between the actual image data generated from these images and the reference data.

  As described above, the actual photographing data includes the four feature point groups G. Therefore, the image recognition unit 71 sets each of the four feature point groups G as the target feature point group GC, and determines whether or not the feature point group G that matches the arrangement of the target feature point group GC is included in the reference data. judge. As is clear from a comparison between FIG. 4B and FIG. 4E, the reference data includes a feature point group G (fixed object feature point group) that matches the arrangement of the target feature point group GC constituting the building 13. GB) and a feature point group G (fixed object feature point group GB) that coincides with the arrangement of the target feature point group GC constituting the house 12 is included. Therefore, the image recognizing unit 71 recognizes the two target feature point groups GC included in the actual photographing data as the house 12 and the building 13 that are constituted by the two fixed object feature point groups GB in the reference data, respectively.

  Here, for convenience of explanation, the expression that the image recognition unit 71 recognizes the house 12 or the building 13 is used. However, in the present invention, it is required that the image recognition unit 71 recognizes up to the type of the object. It is not. That is, in the present application, when the target feature point group GC included in the actual shooting data can be regarded as constituting some object included in the reference data, “the target feature point group GC is referred to as an object (movable object 101 or fixed object 102)”. Recognize. "

  On the other hand, as is apparent from a comparison between FIG. 4B and FIG. 4E, the reference data includes a feature point group G that matches the arrangement of the target feature point group GC that constitutes the circuit breaker 10. Not. Therefore, the actual photographing data shown in FIG. 4B and the reference data shown in FIG. Here, the target feature point group GC constituting the upper circuit breaker 10 (interrupt bar) in FIG. 4B, and the movable object feature point constituting the right circuit breaker 10 (interrupt bar) in FIG. 4E. The groups GA are both arranged in a straight line, and the lengths of the straight lines are the same. That is, the target feature point group GC and the movable object feature point group GA have the same attribute information (geometric shape type and feature) associated with each other. Similarly, the target feature point group GC constituting the lower circuit breaker 10 (interrupt bar) in FIG. 4B and the movable object feature constituting the left circuit breaker 10 (interrupt bar) in FIG. 4E. In the point group GA, attribute information (geometric shape type and feature) associated with each other matches each other. Therefore, the image recognizing unit 71 uses two target feature point groups GC (hereinafter referred to as the target feature point groups GC) constituting the two circuit breakers 10 (blocking bars) in FIG. , “Specific target feature point group GC1”).

  Here, the two movable object feature point groups GA included in the reference data constitute the movable object 101 of the same type, and the type includes the movement range data R as conceptually shown in FIG. Assume that rotational movement range data is associated. In the example shown in FIG. 4, when the left movable object feature point group GA in FIG. 4F is rotated around the fixed feature point P1 by the angle θ1 in the clockwise direction, the arrangement of the movable object feature point group GA is arranged. Corresponds to the arrangement of the lower specific target feature point group GC1 in FIG. Therefore, when the angle θ1 is equal to or smaller than the predetermined angle θ defined as the rotational movement range data (see FIG. 5), the arrangement of the lower specific target feature point group GC1 in FIG. In 4 (f), it is within the range defined by the movement range data R based on the arrangement of the left movable object feature point group GA.

  Further, when the right movable object feature point group GA in FIG. 4F is rotated around the fixed feature point P1 by an angle θ2 in the counterclockwise direction, the arrangement of the movable object feature point group GA is as shown in FIG. ) And the arrangement of the upper specific target feature point group GC1. Therefore, although illustration is omitted, when the angle θ2 is equal to or smaller than the predetermined angle θ defined as the rotational movement range data, the arrangement of the upper specific target feature point group GC1 in FIG. In 4 (f), it is within the range defined by the movement range data R based on the arrangement of the movable object feature point group GA on the right side.

  From the above, assuming that the larger one of θ1 and θ2 is θ3, when the angle θ3 is equal to or smaller than the predetermined angle θ defined as the rotational movement range data, the identification shown in FIG. The arrangement of the target feature point group GC1 is within a range defined in the movement range data R based on the arrangement of the movable object feature point group GA included in the reference data. In this case, the image recognizing unit 71 recognizes each of the two specific target feature point groups GC as the two circuit breakers 10 constituted by the two movable object feature point groups GA in the reference data. Then, the landscape matching unit 6 determines that the pattern matching between the actual shooting data shown in FIG. 4B and the reference data shown in FIG.

  As in this example, even if the movable object 101 is of the same type, the moving direction of the movable part (in this example, the rotation direction) may be different. In such a case, the movement range data R may be set individually for both movement directions, or the common movement range data R may be set for both movement directions.

  In addition, since the moving direction of the movable part also depends on the position of the movable part at the time of capturing the reference captured image, it is configured to distinguish whether the predetermined angle θ is the clockwise direction side or the counterclockwise direction side. In consideration of this, the movement range data R is set.

  Furthermore, here, the case where the reference data is generated is described as an example so that the movable object feature point group GA included in the reference data does not include the feature points constituting the fixed portion included in the movable object 101. However, when the movable object feature point group GA includes a feature point that constitutes the fixed portion, the feature point group G that constitutes the movable portion of the movable object feature point group GA is As for the feature point group G that constitutes the fixed part by applying the special recognition process, image recognition may be performed in the same manner as the fixed object feature point group GB.

  Although detailed explanation is omitted, depending on the type of the circuit breaker 10, the parallel movement range data shown in FIG. 6 and the rotational movement range data shown in FIG. 7 are associated with each other and the special recognition process is executed. be able to. For supplementary explanation, FIG. 6 shows parallel movement range data associated with the case where the blocking bar included in the breaker 10 moves in the vertical direction. In this example, the movable object feature point group GA is a specific target feature. It is located above the point group GC1. In this case, when the downward shift amount (distance) D1 of the specific target feature point group GC1 with respect to the movable object feature point group GA is equal to or less than a predetermined distance D that defines the translation range data, the specific target feature point The arrangement of the group GC1 is within the range defined in the movement range data R with reference to the arrangement of the movable object feature point group GA.

  FIG. 7 also shows that the breaker 10 includes two blocking rods, the two blocking rods are connected at one end, one blocking rod rotates, and the other blocking rod rotates to one blocking rod. In this example, the movable object feature point group GA is located on the upper left side with respect to the specific target feature point group GC1. In this case, the rotation angle θ1 of the part corresponding to the first feature point group G1 in the specific target feature point group GC1 with respect to the first feature point group G1 is equal to or less than the rotation angle θ as the first movement range data R1. The length f1 of the arc defining the boundary between the feature point corresponding to the common feature point P2 in the specific target feature point group GC1 and the common feature point P2 is the arc length as the second movement range data R2. When the length is less than or equal to the length f (θ, L), the arrangement of the specific target feature point group GC1 is within the range defined in the movement range data R based on the arrangement of the movable object feature point group GA.

  Moreover, the present invention can be applied to recognition of not only the circuit breaker 10 but also any other movable object 101 as the movable object 101 targeted by the present invention. For example, as shown in FIG. 8A, the present invention can be applied to image recognition of a signboard 11 having a rotating plate that rotates around a predetermined axis. In the example shown in FIG. 8, the left end of the sign 11 (rotary plate) in FIG. 8A translates in the left-right direction in the figure along with the rotation. As in FIG. 10, it can be recognized using parallel movement range data.

4). Procedure of Operation Process Next, with reference to FIG. 9 and FIG. 10, a procedure of an imaging position determination process (imaging position determination method) executed in the image processing apparatus 2 according to the present embodiment will be described. The procedure of the photographing position determination process described below is executed by hardware or software (program) or both constituting each functional unit of the navigation device 1 (image processing device 2). When each of the above function units is configured by a program, the arithmetic processing device included in the navigation device 1 operates as a computer that executes the program that configures each of the above function units. FIG. 10 is a flowchart showing the procedure of the pattern matching process in step # 14 of FIG.

4-1. Overall Procedure of Shooting Position Determination Processing As shown in FIG. 9, when an actual captured image captured by the camera 14 is input to the actual captured image processing unit 5 (step # 10: Yes), an actual captured data generation unit The actual photographing data is generated by 57 (step # 11). Then, the actual shooting data is transmitted to the landscape matching unit 6, and the image recognition unit 71 extracts reference data for performing pattern matching processing with the actual shooting data as matching candidate reference data from the reference database 92 (step #). 12). The extraction process of the matching candidate reference data in step # 12 may be configured to be performed simultaneously with the actual photographing data generation process in step # 11, or may be configured to be performed before the process in step # 11.

  Next, the image recognition unit 71 selects reference data from the matching candidate reference data (step # 13), and performs a pattern matching process between the selected reference data and actual captured data (step # 14). The pattern matching process will be described later in detail based on the flowchart of FIG.

  When the pattern matching between the reference data and the actual shooting data is successful (step # 15: Yes), the shooting position determination unit 73 is associated with the reference data (identification reference data) that has been successfully matched with the pattern. The shooting position is acquired, and the shooting position of the actual shot image is determined based on the shooting position.

  On the other hand, when the pattern matching between the reference data and the actual photographing data is not successful (step # 15: No), there is reference data that has not yet been subjected to the pattern matching process in the matching candidate reference data. Is determined (step # 17). If there is reference data that has not yet been subjected to pattern matching processing in the matching candidate reference data (step # 17: Yes), the processing is returned to step # 13, and the processing from step # 13 to step # 15 is performed. The process is executed again. On the other hand, when there is no reference data that has not yet been subjected to the pattern matching process in the matching candidate reference data (step # 17: No), that is, any reference in which the actual shooting data is included in the matching candidate reference data If pattern matching is not successful with the data, the process ends.

4-2. Pattern Matching Process Procedure Next, a pattern matching process procedure will be described with reference to FIG. In the pattern matching process, first, it is determined whether or not the actual image data matches the reference data (step # 20). Here, the fact that the actual shooting data and the reference data match each other means that the actual shooting data and the reference data are substantially included in the actual shooting data when the actual shooting data and the reference data are compared as they are (that is, without performing the special recognition process). This means that all feature point groups G can be recognized as objects included in the reference data. If the actual image data and the reference data match (step # 21: Yes), it is determined that the pattern matching is successful (pattern matching success) (step # 26), and the process ends.

  On the other hand, if the actual image data does not match the reference data (step # 21: No), it is determined whether or not the reference data includes the movable object 101 (step # 22). As described above, since the reference data includes information on whether or not the reference data includes the movable object 101, the determination in step # 22 is executed based on this information. If the reference data does not include the movable object 101 (step # 22: No), it is determined that the pattern matching has not been successful (pattern matching is unsuccessful) (step # 27), and the process ends.

  On the other hand, if the reference data includes the movable object 101 (step # 22: Yes), whether the attribute information of the specific target feature point group GC1 and the attribute information of the movable object feature point group GA are the same. Is determined (step # 23). The specific target feature point group GC1 is a feature point group G (target feature point group GC) included in the actual shooting data, and the target feature point whose arrangement does not match any feature point group G included in the reference data Group GC. Further, as described above, in this embodiment, the reference data includes information on whether or not the movable object 101 is included in the reference data, but which feature point group G identifies the movable object 101. It does not have information on how to configure. Therefore, in this example, the feature point group G included in the reference data, and the feature point group G whose arrangement does not match any of the feature point groups G included in the actual photographing data is set as the movable object feature point group GA.

  And when the attribute information of specific object feature point group GC1 and the attribute information of movable object feature point group GA are not the same (step # 23: No), pattern matching was not successful (pattern matching unsuccessful). A determination is made (step # 27), and the process ends. On the other hand, when the attribute information of the specific target feature point group GC1 and the attribute information of the movable object feature point group GA are the same (step # 23: Yes), the movable object feature point group GA is determined from the movement range database 93. The moving range data R corresponding to the type of the movable object 101 to be configured is extracted (step # 24). Based on the movement range data R extracted in step # 24, the arrangement of the specific target feature point group GC1 is within the range defined in the movement range data R based on the arrangement of the movable object feature point group GA. It is determined whether or not there is (step # 25).

  In step # 25, when it is determined that the arrangement of the specific target feature point group GC1 is within the range defined in the movement range data R based on the arrangement of the movable object feature point group GA (step # 25: Yes), it is determined that the pattern matching is successful (step # 26), and the process ends. On the other hand, if it is determined in step # 25 that the arrangement of the specific target feature point group GC1 is outside the range defined in the movement range data R based on the arrangement of the movable object feature point group GA (step # 25: No), it is determined that the pattern matching is unsuccessful (step # 27), and the process ends.

  By the way, there may be a case where a plurality of specific target feature point groups GC1 are included in the actual photographing data and a plurality of movable object feature point groups GA are included in the reference data. In such a case, in step # 23, the specific target feature point group GC1 and the movable object feature point group GA can be associated one-to-one and the specific target feature point groups associated with each other. Step # 23 when the attribute information of the specific target feature point group GC1 and the attribute information of the movable object feature point group GA constituting each set are the same for all the pairs of GC1 and the movable object feature point group GA. It determines with Yes.

  Similarly, for all of the pairs of the specific target feature point group GC1 and the movable object feature point group GA that are associated with each other, the arrangement of the specific target feature point group GC1 constitutes the same set as the specific target feature point group GC1. If it is within the range defined in the movement range data R based on the arrangement of the movable object feature point group GA to be determined, Yes is determined in step # 25.

5. Other Embodiments Finally, other embodiments according to the present invention will be described. Note that the features disclosed in each of the following embodiments can be used only in that embodiment, and can be applied to other embodiments as long as no contradiction arises.

(1) In the above embodiment, the moving range database 93 includes three examples of the moving range data R: rotational moving range data, parallel moving range data, and rotating parallel moving range data. As explained. However, the embodiment of the present invention is not limited to this, and may be configured to include only a part of these movement range data R, or may be configured to include movement range data R composed of a combination thereof. it can. Moreover, it can also be set as the structure further provided with the movement range data R other than the said three movement range data R. For example, the moving range data R may be configured to define an area of a predetermined size including the feature point group G in the image as the moving range of the feature point group G. Note that the predetermined size can be, for example, a rectangular region determined by the width and height, or a circular region determined by the diameter.

(2) In the above embodiment, the configuration in which the moving range database 93 divides the movable object 101 into a plurality of types and includes the moving range data R for each type has been described as an example. However, the embodiment of the present invention is not limited to this, and the moving range database 93 may include moving range data R for each movable object 101. In this case, unlike the above-described embodiment, the reference data is configured to include information for specifying the movable object 101 instead of the type of the movable object 101 included in the reference data. Further, the moving range database 93 may be configured to include moving range data R common to all the movable objects 101.

(3) In the above embodiment, the configuration in which the landscape matching unit 6 determines the shooting position of the actual shot image based on the recognition result of the image recognition unit 71 has been described as an example. However, the embodiment of the present invention is not limited to this, and the recognition result of the movable object 101 by the image recognition unit 71 can be used for other purposes. For example, when the image recognition unit 71 recognizes the movable object 101, the information may be provided to the driver of the vehicle (for example, provided as an alert). In such a case, the image processing apparatus 2 may be configured not to include the matching execution unit 72 and the shooting position determination unit 73.

(4) In the above-described embodiment, the configuration in which the vehicle position (estimated vehicle position) of the vehicle is corrected using the determination result of the shooting position of the actual captured image by the shooting position determination unit 73 has been described as an example. However, the embodiment of the present invention is not limited to this, and the determination result may be configured to be used for other purposes, such as running control of the vehicle 100 based on the determination result of the shooting position determination unit 73. Of course, it is possible.

(5) In the above embodiment, the image recognizing unit 71 moves the moving range data R corresponding to the type of the movable object 101 included in the reference data to be matched to the moving range database configured separately from the reference database 92. The configuration extracted from 93 has been described as an example. However, the embodiment of the present invention is not limited to this, and the reference database 92 and the movement range database 93 are integrally configured (hereinafter referred to as “integrated database”), and the movable object 101 is provided. It is also a preferred embodiment of the present invention that each of the included reference data is stored in association with the movement range data R. In such a configuration, the image recognizing unit 71 simultaneously extracts the reference data from the integrated database, and, when the reference data includes the movable object 101, the moving range data R corresponding to the movable object 101 from the integrated database. It becomes the structure to extract.

(6) In the above embodiment, the actual captured image processing unit 5 includes the feature object recognizing unit 52 and the attribute determining unit 53, and identification in the actual captured image based on the feature points extracted by the feature point extracting unit 60. An example has been described in which a possible object is recognized as a feature object and its attribute information is generated. However, the embodiment of the present invention is not limited to this, and the actual captured image processing unit 5 does not include the feature object recognition unit 52 and the attribute determination unit 53, and the actual captured data generation unit 57 includes the feature point extraction unit. The actual image data including only the position information (feature point image information) of the feature points extracted in 60 may be generated. In this case, the reference data can be configured not to include the attribute information. In such a configuration, the image recognizing unit 71 executes the special recognition process when the actual photographing data and the reference data do not match and the reference data includes the movable object 101. It can be.

(7) In the above embodiment, the reference data includes information on whether or not the movable object 101 is included in the reference data. Which feature point group G constitutes the movable object 101. A configuration that does not have the above information has been described as an example. However, the embodiment of the present invention is not limited to this, and the reference data may be configured to include information for specifying the feature point group G constituting the movable object 101. Further, the special recognition process may be executed when the reference data does not have information on whether or not the movable object 101 is included and the actual image data does not match the reference data.

(8) In the above embodiment, the configuration in which the camera 14 is a front camera that captures the front of the vehicle 100 has been described as an example. However, the embodiment of the present invention is not limited to this, and the in-vehicle camera may be a camera that captures a landscape in a diagonal direction in front of the vehicle, or a camera that captures a landscape on the side of the vehicle or behind the vehicle. That is, the actual captured image targeted by the present invention is not limited to the captured image of the landscape on the front side in the vehicle traveling direction.

(9) The assignment of the functional units described in the above embodiment is merely an example, and a plurality of functional units can be combined or one functional unit can be further divided.

(10) In the above-described embodiment, the configuration in which the image processing device 2 is provided in the navigation device 1 has been described as an example. However, the embodiment of the present invention is not limited to this, and the image processing device 2 may be provided in another device such as a vehicle travel control device.

(11) In the above embodiment, in addition to the camera 14, the actual captured image processing unit 5, the landscape matching unit 6, the reference database 92, and the moving range database 93 that constitute the image processing apparatus 2 are all mounted on the vehicle 100. The configuration to be described has been described as an example. However, the embodiment of the present invention is not limited to this, and at least one of the actual captured image processing unit 5, the landscape matching unit 6, the reference database 92, and the movement range database 93 is a communication network such as the Internet. It can also be set as the structure installed outside the vehicle 100 in the state connected via the. In such a configuration, the image processing apparatus 2 that processes an actual captured image captured by the camera 14 provided in the vehicle 100 by transmitting and receiving information and signals via the network is configured.

(12) Regarding other configurations as well, the embodiments disclosed herein are illustrative in all respects, and the embodiments of the present invention are not limited thereto. That is, as long as the configuration described in the claims of the present application and a configuration equivalent thereto are provided, a configuration obtained by appropriately modifying a part of the configuration not described in the claims is naturally also included in the present invention. Belongs to the technical scope.

  The present invention can be suitably used for an image processing apparatus and an image processing program that perform image recognition using an image taken by an in-vehicle camera.

2: Image processing device 5: Actual captured image processing unit 14: Camera (vehicle camera)
71: Image recognition unit 72: Matching execution unit 73: Shooting position determination unit 92: Reference database 93: Moving range database 101: Moving object 102: Fixed object D: Predetermined distance G: Feature point group G1: First feature point group G2: second feature point group GA: movable object feature point group GB: fixed object feature point group GC: target feature point group P1: fixed feature point P2: common feature point R: movement range data θ: predetermined angle

Claims (8)

A movable object having a movable part that moves according to a certain law while maintaining a certain shape is generated by extracting a plurality of feature points of the movable object from a reference photographed image that is a photographed image of a landscape including the movable object. A reference database that stores reference data including position information of the movable object feature point group in the reference captured image, the feature point group constituting the movable object at an arbitrary time point as a movable object feature point group;
A moving range database for storing moving range data including information defining a range in which the movable object feature point group may move;
An actual captured image processing unit that extracts a plurality of feature points from an actual captured image that is a captured image of a landscape by an in-vehicle camera, and generates actual captured data including position information of the plurality of feature points in the actual captured image; ,
The movement range data based on the arrangement of the target feature point group composed of at least a part of the plurality of feature points included in the actual photographing data based on the arrangement of the movable object feature point group included in the reference data An image recognition unit that recognizes the target feature point group as the movable object included in the reference data,
An image processing apparatus comprising: The reference data further includes position information in the standard captured image of a fixed object feature point group that is a feature point group constituting a fixed object that does not move,
The image recognizing unit matches an arrangement of a target feature point group including at least a part of the plurality of feature points included in the actual photographing data with an arrangement of the fixed object feature point group included in the reference data. The image processing apparatus according to claim 1, wherein the target feature point group is recognized as the fixed object included in the reference data. The reference database includes a plurality of the reference data, and stores each of the plurality of reference data in association with the shooting position of the standard shooting image corresponding to each reference data,
Based on the recognition result of the image recognition unit, a matching execution unit that performs matching between the actual photographing data and the reference data;
The reference data that has been successfully matched by the matching execution unit is used as identification reference data, and the shooting position of the actual shooting image corresponding to the actual shooting data is determined based on the shooting position associated with the identification reference data. A shooting position determination unit;
The image processing apparatus according to claim 1, further comprising: The moving range database classifies the movable object into a plurality of types, and includes the moving range data for each type,
The said image recognition part performs the recognition of the said movable object using the said reference data and the said movement range data corresponding to the classification of the said movable object contained in the said reference data. An image processing apparatus according to 1.   The moving range data is obtained when the feature points at the end of the feature point group arranged in a predetermined shape are fixed feature points, and the feature point group is rotated around the fixed feature points by a predetermined angle. The image processing apparatus according to claim 1, wherein a moving range of the feature point group is defined.   5. The movement range data defines a movement range of the feature point group when the feature point group arranged in a predetermined shape is translated in a predetermined direction by a predetermined distance. An image processing apparatus according to claim 1.   The moving range data includes the first feature point group and the second feature point group each having a common feature point that is a common feature point at an end portion and arranged in a predetermined shape. Using the feature point at the end of the feature point group opposite to the common feature point as a fixed feature point, the first feature point group is rotated around the fixed feature point by a predetermined angle, and the second feature point The image processing apparatus according to any one of claims 1 to 4, wherein a movement range of the feature point group when the feature point group is translated in accordance with the movement of the common feature point is defined. A movable object having a movable part that moves according to a certain law while maintaining a certain shape is generated by extracting a plurality of feature points of the movable object from a reference photographed image that is a photographed image of a landscape including the movable object. , A feature point group constituting the movable object at an arbitrary time point is a movable object feature point group,
An actual captured image processing step of extracting a plurality of feature points from an actual captured image that is a captured image of a landscape by an in-vehicle camera, and generating actual captured data including position information of the plurality of feature points in the actual captured image; ,
A reference data extraction step of extracting the reference data from a reference database that stores reference data including position information of the movable object feature point group in the standard captured image;
A moving range data extracting step of extracting the moving range data from a moving range database storing moving range data including information defining a range in which the movable object feature point group may move;
The reference data extracted in the reference data extracting step is used as extracted reference data, and the moving range data extracted in the moving range data extracting step is included in the actual photographing data as extracted moving range data. A range defined by the extracted movement range data based on the arrangement of the movable object feature point group included in the extraction reference data, the arrangement of the target feature point group including at least a part of the plurality of feature points An image recognition step for recognizing the target feature point group as the movable object included in the extraction reference data,
An image processing program for causing a computer to execute.

Images