JP4895126B2 - Image recognition device and navigation device - Google Patents

Description

  The present invention relates to an image recognition device that recognizes an image of another vehicle that exists in the vicinity of the host vehicle, and to a navigation device that executes a predetermined operation using information related to the other vehicle recognized by the image recognition device.

Navigation devices that acquire vehicle position information and provide guidance around the vehicle location and route guidance to the destination are well known, and today there are fewer vehicles that are not equipped with a navigation device. ing.
As one function of such a navigation device, information on the vehicle traveling ahead is obtained by an image recognition method, and it is determined whether the vehicle is in a running / stopped state or whether the vehicle is a large vehicle. Patent Document 1 discloses a technique for controlling the traveling state of the host vehicle appropriately. In the technique disclosed in Patent Document 1, it is determined whether or not the preceding vehicle is a large vehicle, and this information is used for traveling the vehicle.

JP 2006-251888 A

  Such a technique for identifying a preceding vehicle by image recognition basically assumes that a clear image of the preceding vehicle is obtained satisfactorily. Therefore, the image recognition is usually based on an image obtained in the daytime. However, the situation where the above driving control is required is not limited to daytime, and it is difficult to obtain a clear image at night (dark night). Is desirable. In particular, in a situation where the preceding vehicle performs a braking operation and the preceding vehicle is decelerating, it is desirable that this situation be accurately captured by the own vehicle.

  On the other hand, for example, while driving on a highway, the driver or his / her passenger is often interested in the type of vehicle traveling ahead. There are various ways of interest, but “What is the model name of the vehicle ahead?” “What is the manufacturer?” “What is the grade and price?” “The vehicle is about to be replaced soon. I want it. " Under such circumstances, conventionally, only information on the vehicle type can be obtained based on the knowledge of the driver or the passenger. In many cases, even the name of the car model was not known, and although the car model attracted interest, it did not lead to further movement.

  From the above situation, the purpose of the present application is, for example, a technique in which when a preceding vehicle performs a brake operation, the preceding vehicle can be accurately recognized by an image recognition method, and further, the vehicle type of the preceding vehicle can be accurately identified. Is to provide.

The first characteristic configuration of the image recognition apparatus for achieving the above object is as follows:
Image information acquisition means for acquiring image information around the host vehicle;
From the image information acquired by the image information acquisition unit, and the other vehicle image extracting means for extracting an image of another vehicle that exists around the own vehicle,
The other from the vehicle image extracting unit image extracted by addition to image recognition sequence of the other vehicle in the vehicle lamp Ru lighting state near provided in the light source as an array of bright spots, the image recognized vehicle the lamp Vehicle information acquisition means for obtaining vehicle information of other vehicles based on the arrangement of light sources in
The vehicle information obtained by the vehicle information acquisition means is output.

In the image recognition apparatus having this configuration, the image information acquisition unit acquires image information around the host vehicle, and the other vehicle image extraction unit uses the image information acquired by the image information acquisition unit to exist around the host vehicle. An image of the existing other vehicle is extracted.
Then, from the image extracted by the other vehicle image extraction means, the vehicle information acquisition means recognizes the arrangement of the light sources provided in the vehicle lamp of the other vehicle, and the light source arrangement in the vehicle lamp recognized by the image. Based on the above, vehicle information of other vehicles is obtained, and the vehicle information is output.

That is, in this image recognition apparatus, the arrangement of the light sources in the vehicle lamp is used to recognize other vehicles. For example, when there is a vehicle traveling ahead at night, the light source in the vehicle lamp is turned on when a braking operation is performed on the preceding vehicle. In recent years, LEDs and the like have been adopted as light sources for vehicle lamps, and these light sources are arranged in a unique arrangement for each vehicle type, as indicated by the alternate long and short dash line indicated by Ic in FIGS.
And at night, the front vehicle contour and the periphery and boundary other than the vehicle, the boundary of the outer shape of the vehicle lamp, etc. are not clearly captured, and the light recognition is still in a situation where the reliability of the image recognition is low. The light source can be easily identified as a clear luminescent spot, and even its arrangement can be easily identified.

Therefore, based on such an array of light sources in the vehicle lamp (on the image, an array of bright spots in the vehicle lamp), vehicle information such as the presence of the vehicle, the distance between the vehicle and the host vehicle, and the vehicle type of the vehicle is obtained. It can be obtained relatively easily.
The vehicle information obtained in this way can be used for traveling control of the own vehicle or for introduction of the vehicle type of the preceding vehicle, for example.

To realize the image recognition device,
An image information acquisition step of acquiring image information around the host vehicle;
From image information acquired by the image information acquisition step, and another vehicle image extraction step of extracting an image of another vehicle that exists around the own vehicle,
The other from a vehicle image extracting step with the extracted image while image recognition sequence of the other vehicle in the vehicle lamp Ru lighting state near provided in the light source as an array of bright spots, the image recognized vehicle the lamp Vehicle information acquisition step of obtaining vehicle information of other vehicles based on the arrangement of light sources in
What is necessary is just to store the image recognition program which stores and performs the output process which outputs the vehicle information obtained by the said vehicle information acquisition process in a computer.

In the image recognition apparatus, the vehicle information acquisition unit recognizes the vehicle lamp of the other vehicle from the image extracted by the other vehicle image extraction unit, and performs other based on the shape of the vehicle lamp recognized. It is preferable to obtain vehicle information of the vehicle.
When the vehicle lamp itself can be identified in the acquired image, the image recognition device having this configuration obtains useful vehicle information based on the shape of the vehicle lamp itself in addition to the arrangement of the light sources in the vehicle lamp. Can do.

In the configuration that also uses the shape of the vehicle lamp as described above, the vehicle information acquisition means can also acquire vehicle information of other vehicles based on the relationship between the arrangement of the light sources in the vehicle lamp recognized as the image and the shape of the vehicle lamp. It is preferable to obtain
The image recognition apparatus having this configuration can obtain useful vehicle information based on the relationship between the arrangement of the light sources in the vehicle lamp and the shape of the vehicle lamp itself when the vehicle lamp can be identified in the acquired image. . As this type of relationship, the position and size of the light source array with respect to the contour shape of the vehicle lamp, the range occupied in the contour, and the like can be increased.

It is also preferable that the vehicle information acquisition means recognizes the contour of the other vehicle from the image extracted by the other vehicle image extraction means and obtains the vehicle information of the other vehicle based on the recognized contour. .
When the contour of the vehicle can be identified in the acquired image, the image recognition device having this configuration obtains useful vehicle information based on the shape of the contour of the vehicle itself in addition to the arrangement of the light sources in the vehicle lamp. Can do.

Thus, in the configuration that also uses the contour of the vehicle, the vehicle information acquisition means obtains the vehicle information of the other vehicle based on the relationship between the arrangement of the light sources in the vehicle lamp recognized as the image and the contour of the other vehicle. It is preferable to do so.
The image recognition apparatus having this configuration can obtain useful vehicle information based on the relationship between the arrangement of the light sources in the vehicle lamp and the contour of the vehicle when the contour of the vehicle can be identified in the acquired image. . As this kind of relationship, the position and size of the light source array with respect to the contour shape of the vehicle, the range occupied in the contour, and the like can be increased.

In the image recognition apparatus adopting the above configuration, the vehicle information acquisition unit is configured as a vehicle type identification unit that identifies the vehicle type of the other vehicle, and is configured to output the identification result of the vehicle type identification unit. preferable.
As described above, since the arrangement of the light sources in the vehicle lamp is unique depending on the vehicle type, the vehicle type identification unit as the vehicle information acquisition unit identifies the vehicle type of the other vehicle using the arrangement. Then, the image recognition device outputs the identification result of the vehicle type obtained in this manner, so that the information related to the vehicle type of the other vehicle can be used for introduction of the vehicle type.
Naturally, in addition to the arrangement of the light sources in the vehicle lamp, elements such as the shape of the vehicle lamp, the outline of the vehicle can also be used for identifying the vehicle type, and the arrangement of the light sources in the vehicle lamp and the shape of the vehicle lamp, By using either one or both of the contours of the vehicle, it is possible to identify an effective vehicle type.

The image recognition apparatus described above is provided, and
Own vehicle position information acquisition means for acquiring own vehicle position information representing the current position of the own vehicle;
By configuring a navigation device that performs road guidance based on the vehicle position information acquired by the vehicle position information acquisition means, in the navigation device that can provide road guidance, vehicle information is acquired and the information is made effective. A navigation device to be used can be configured.
As this navigation device, the vehicle information obtained from the image recognition device generates information that can be used as a reference for traveling, the information that is used for traveling control, or the vehicle information is information on the vehicle type, the introduction of the vehicle type. Can do.

  That is, by providing the navigation device with introduction output means for introducing the vehicle type introduction information related to the identified vehicle type output by the image recognition device, the vehicle type introduction function can be achieved.

Further, the introduction output means is one or more selected from the name, concept, topic, grade, price, address information on the communication network that introduces the vehicle type, and information about the dealer that sells the vehicle type. It is preferable to be configured to be able to introduce. With such a configuration, each piece of information can be provided.
Here, the name, concept, topic, grade, price, etc. of the vehicle type are information that can be guided even when traveling, for example, at the time of actually looking at the vehicle of that vehicle type. It is preferable information to introduce.
On the other hand, the address information on the communication network introducing the vehicle type is information suitable for the driver to return home and access the Internet from each home. In this way, by introducing the address information, for example, it is possible to see vehicles of different colors, to confirm the appearance in different directions, or to compare with the same vehicle type for the same vehicle type. .
Further, when information on a dealer that sells the vehicle type is introduced, it is possible to stop by the nearest dealer and see the vehicle of the vehicle type or receive an explanation when traveling.

  On the other hand, if the navigation device is configured to include the image recognition device described above and generate and output the travel control information based on the vehicle information acquired by the image recognition device, Information on the light source identified in the obtained vehicle lamp is obtained, and appropriate travel guidance can be performed.

Next, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a functional block diagram showing a schematic configuration of a navigation device 1 according to the present embodiment.

Each functional unit of the navigation device 1 shown in FIG. 1, specifically, an image information acquisition unit 12, a vehicle position information acquisition unit 16, a feature information acquisition unit 17, an image recognition unit 18, and a vehicle position information correction unit 19. The navigation calculation unit 20 and the introduction calculation unit 30 have hardware or software (program) or functional units for performing various processes on input data with a calculation processing device such as a CPU as a core member. Both are implemented and configured. Each of these functional units is configured to exchange information with each other.
Also, a recording medium capable of storing information, such as a hard disk drive, a DVD drive equipped with a DVD-ROM, a CD drive equipped with a CD-ROM, and its driving means, such as the databases DB1 to DB3 of the navigation device 1 Is provided as a hardware configuration.

Hereinafter, the configuration of each part of the navigation device 1 according to the present embodiment will be described in detail. Since the navigation apparatus 1 includes the image recognition apparatus 100 according to the present application inside, the main configuration of the image recognition apparatus 100 is shown in FIG.
Further, in the following description, a configuration in which the vehicle information acquisition unit referred to in the present application identifies a vehicle type as vehicle information will be mainly described.

1. Map database The map database DB1 is a database in which map information M divided into predetermined sections is stored. FIG. 2 is a diagram illustrating an example of the configuration of the map information M stored in the map database DB1. As shown in this figure, the map information M is road information Ra representing a road network by a connection relationship between a number of nodes n corresponding to intersections and links k corresponding to roads connecting the intersections (see FIG. 1). have. Each node n has information on the position (coordinates) on the map expressed by latitude and longitude. Each link k is connected via a node n. Each link k has information such as a road type, a region type, a link length, a road width, and a shape interpolation point for expressing a link shape as attribute information. Here, the road type information is information on the road type when the road is divided into a plurality of types, such as an automobile-only road, a city road, a narrow street, a mountain road, and the like. In addition, the region type information is divided into a plurality of regions where roads corresponding to the link k are provided, such as regions such as the Kanto region and the Kansai region, and administrative divisions such as prefectures and municipalities. Information on the type of the area. The attribute information of these links k corresponds to the road attribute information Rb (see FIG. 1). In FIG. 2, only the road information Ra of one section is illustrated, and the road information Ra of other sections is omitted.

2. Feature database The feature database DB2 is a database in which information on various features provided on or around the road, that is, feature information F is stored. As shown in FIG. 1, in the present embodiment, feature information F is stored in the feature database DB2. Here, such feature information F is mainly maintained for some areas such as large cities and main roads in all areas where map information M including road information Ra is maintained. Has been. In the present embodiment, this feature database DB2 corresponds to the feature information storage means in the present invention.

  Features for which feature information F is stored in the feature database DB2 include road markings (paint markings) provided on the road surface. FIG. 3 is a diagram illustrating an example of the feature information F of the road marking stored in the feature database DB2. Features such as road markings include, for example, pedestrian crossings, stop lines, speed markings representing maximum speed, zebra zones, lane markings that divide lanes along the road (solid lines, broken lines, double lines, etc.) Including various lane markings), traffic direction markings by direction of travel that specify the direction of travel of each lane (including arrow markings such as straight arrows, right turn arrows, etc.), and the like. In addition to such road markings, the features in which the feature information F is stored can include various features such as traffic lights, signs, overpasses, and tunnels.

  The feature information F includes position information of each feature and feature attribute information associated therewith as its contents. Here, the position information includes information on the position (coordinates) on the map of the representative point of each feature associated with the link k or the node n constituting the road information Ra, and the direction of each feature. In this example, the representative point is set near the center in the length direction and width direction of each feature. The feature attribute information includes feature type information, feature form information such as the shape, size, and color of the feature. Here, the feature type specifically indicates the type of feature having basically the same form such as “pedestrian crossing”, “stop line”, “speed indication (30 km / hour)”, and the like. Information.

3. Vehicle Database The vehicle database DB3 is a database in which information on various vehicles traveling on the road, that is, vehicle information C is stored. The vehicle identification information stored in the vehicle database DB3 is used for identifying a vehicle to be recognized in the image recognition apparatus 100.
As shown in FIG. 1, in this embodiment, two types of information, vehicle basic information Ca and vehicle identification information Cb, are stored in the vehicle database DB3 for each vehicle type. In the figure, “A” and “B” are shown as vehicle types. As shown in FIG. 1, the information stored in the database DB3 can be updated to the latest information by appropriately fetching the latest information from the vehicle data update server CDS via the update terminal En and the Internet In. It has become.

  The vehicle basic information Ca is vehicle type information such as the name of the vehicle type, address information on a communication network that introduces the vehicle, information of a dealer that sells the vehicle type, and the like.

The vehicle identification information Cb is information capable of identifying a vehicle type when a specific vehicle is a vehicle type identification target. Specifically, the vehicle identification information Cb includes vehicle contour information Cb1, vehicle lamp information Cb2, and vehicle lamp bright spot array information Cb3.
The vehicle contour information Cb1 is information related to various vehicles and the contour shapes of the respective vehicle outer shapes, and is mainly information related to the contour shape when the vehicle is viewed from the rear. The shape may be a contour shape seen from the right side or the left side. FIG. 5 shows the contour shapes (a) and (b) of different vehicle types with thin solid lines. The vehicle contour information Cb1 is an identification element for identifying the vehicle type based on the contour shape and the size of the contour shape.

  The vehicle lamp information Cb2 is information on various types of vehicles and their respective lamps, and this information is also mainly information on the outer shape of the entire lamp that can be seen when the vehicle is viewed from the rear. Information on the outer shape of the entire lamp when viewed from the right side, and information when viewed from the right side and the left side may also be used. In the case of viewing from the rear, this information includes information on the contour shapes of the left and right individual lamps (including the contour itself and its size), as well as the positional relationship between the pair of left and right lamps, Furthermore, the information regarding the position, the magnitude | size of the lamp | ramp with respect to the outline shape of the vehicle external shape demonstrated previously, and the range which occupies in the outline is also contained. FIG. 5 shows the lamp shapes (a) and (b) of different vehicle types with thick solid lines. The vehicle lamp information Cb2 is an identification element for identifying the vehicle type from information such as the lamp itself, the contour shape of the vehicle outer shape and the position of the lamp.

  The vehicle lamp luminescent spot arrangement information Cb3 is information relating to the arrangement of various vehicles and light sources such as LEDs that are luminescent spots arranged in the respective lamps. This information is also mainly viewed from the rear. The information on the bright spot arrangement that can be seen when the vehicle is viewed may be information on the bright spot arrangement when the vehicle is viewed from the front, or information when the vehicle is viewed from the right side or the left side. The information when viewed from the back will be described. This information includes the bright spot arrangement in the left and right individual lamps (including the outer boundary shape surrounding all the bright spots, and the bright spots when the bright spots can be recognized individually). In addition, the information includes information on the position and size of the bright spot arrangement portion with respect to the outline shape of the vehicle outer shape and the outline shape of the lamp described above, and the range occupied in the outline. FIG. 5 shows the bright spot arrangements (a) and (b) of different vehicle types by a one-dot chain line. The vehicle lamp luminescent spot array information Cb3 serves as an identification element for identifying the vehicle type from the luminescent spot array itself or information such as the lamp shape or the contour shape of the vehicle outer shape and the position of the luminescent spot array.

  In the present embodiment, the vehicle database DB3 serves as vehicle information storage means.

4). Own vehicle position information acquisition unit The own vehicle position information acquisition unit 16 functions as own vehicle position information acquisition means for acquiring own vehicle position information P indicating the current position of the own vehicle 50. Here, the vehicle position information acquisition unit 16 is connected to the GPS receiver 13, the direction sensor 14, and the distance sensor 15.
Here, the GPS receiver 13 is a device that receives GPS signals from GPS (Global Positioning System) satellites. This GPS signal is normally received every second and is output to the vehicle position information acquisition unit 16. The own vehicle position information acquisition unit 16 analyzes signals from GPS satellites received by the GPS receiver 13 and acquires information such as the current position (latitude and longitude), traveling direction, and moving speed of the own vehicle 50. Can do.
The direction sensor 14 is a sensor that detects the traveling direction of the host vehicle 50 or a change in the traveling direction. The azimuth sensor 14 includes, for example, a gyro sensor, a geomagnetic sensor, an optical rotation sensor attached to the rotating part of the handle, a rotary resistance volume, an angle sensor attached to the wheel part, and the like. Then, the direction sensor 14 outputs the detection result to the vehicle position information acquisition unit 16.
The distance sensor 15 is a sensor that detects the vehicle speed and the moving distance of the host vehicle 50. The distance sensor 15 is, for example, a vehicle speed pulse sensor that outputs a pulse signal each time a drive shaft or wheel of the vehicle rotates by a certain amount, a yaw / G sensor that detects acceleration of the host vehicle 50, and integration of the detected acceleration. Configured by a circuit or the like. Then, the distance sensor 15 outputs information on the vehicle speed and movement distance as the detection result to the vehicle position information acquisition unit 16.

  And the own vehicle position information acquisition part 16 performs the calculation which pinpoints an own vehicle position by a well-known method based on the output from these GPS receiver 13, the direction sensor 14, and the distance sensor 15. FIG. In addition, the vehicle position information acquisition unit 16 acquires road information Ra around the vehicle position extracted from the map database DB1, and indicates the vehicle position in the road information Ra by performing known map matching based on the road information Ra. Also make corrections to fit on the road. In this way, the host vehicle position information acquisition unit 16 acquires host vehicle position information P including information on the current position of the host vehicle 50 represented by latitude and longitude and information on the traveling direction of the host vehicle 50.

5). Feature information acquisition unit The feature information acquisition unit 17 uses the vehicle location information P acquired by the vehicle location information acquisition unit 16 and the like to search for features existing around the vehicle 50 from the feature database DB2. The feature information F is extracted and acquired. In the present embodiment, as an example, the feature information acquisition unit 17 is based on the vehicle position information P, and the vehicle 50 is traveling from the current position of the vehicle 50 indicated by the vehicle position information P. The feature information F of the target feature existing before the end of the link k representing the road is extracted from the feature database DB2. Then, the feature information F acquired by the feature information acquisition unit 17 is output to the image recognition unit 18 and the vehicle position information correction unit 19. Here, the feature that is the target feature is a feature that is a target of image recognition processing by the image recognition unit 5. In the present embodiment, a feature of a feature type selected from various road markings provided on the road surface such as a pedestrian crossing, a stop line, and a speed marking is the target feature.

6). Image Recognizing Unit The image recognizing unit 18 functions as a vehicle position correcting image recognizing unit 18a that performs image recognizing processing of target features included in the image information G1 and G2 acquired by the image information acquiring unit 12, and the vehicle type. It functions as vehicle type identification image recognition means 18b for performing image recognition processing for identifying the vehicle. As will be described later, the vehicle type identification image recognition means 18b constitutes one functional part of the image recognition apparatus 100 according to the present application.
Self-vehicle position correction image recognition processing In the present embodiment, the image recognition unit 18 (specifically, the self-vehicle position correction image recognition means 18a) is a position correction image for use in correcting the own vehicle position information P. Perform recognition processing.
The image recognition unit 18 refers to the feature information F acquired from the feature database DB2 by the feature information acquisition unit 17 based on the vehicle position information P. Then, when it is determined that the target feature stored as the feature information F exists around the host vehicle 50, the position correction image recognition processing is performed. In the image recognition process for position correction, the image recognition unit 18 performs the back operation based on the feature information F of the target feature existing around the host vehicle 50 acquired from the feature database DB2 based on the host vehicle position information P. Image recognition processing of the target feature indicated by the feature information F included in the image information G1 captured by the camera 11a (see FIG. 4A) is performed. At this time, the image recognition unit 18 sets a predetermined recognition area E (see FIG. 7) for requesting recognition of the target feature indicated by the feature information F, and the image information G1 in the recognition area E is set. Perform image recognition processing of the target feature. This recognition area E is set as a position range on which the target feature is estimated to exist on the link k indicated by the road information Ra. Here, the actual road imaging area included in each image information G1 is the vehicle position and imaging area calculated in advance based on the mounting position, mounting angle, field angle, and the like of the imaging device 11 to the host vehicle 50. Can be obtained based on the vehicle position information P. Therefore, the image recognition unit 18 extracts the image information G1 corresponding to the recognition area E set for each target feature based on the information of the imaging area of each image information G1 thus obtained, and performs image recognition processing. I do. The image recognition result of the target feature by the position correction image recognition process is used for correcting the vehicle position information P by the vehicle position information correction unit 19.
In this processing, the image recognition unit 18 performs binarization processing, edge detection processing, and the like on the image information G1 when recognizing the target feature, and the ground included in the image information G1. The contour information of the object (road marking) is extracted. Thereafter, the image recognition unit 18 extracts an image of the target feature included in the image information G1 by performing pattern matching between the extracted contour information of the feature and the feature quantity of the target feature. Then, the image recognition unit 18 succeeds in image recognition of the target feature when the image of the target feature can be extracted from the image information G1 in the set recognition area E during the position correction image recognition processing. It is determined that On the other hand, as a result of performing the image recognition process on all the image information G1 in the recognition area E, the image recognition unit 18 fails to recognize the image of the target feature when the image of the target feature cannot be extracted. Is determined.

Vehicle Type Identification Image Recognition Processing In this embodiment, the image recognition unit 18 (specifically, vehicle type identification image recognition means 18b) performs vehicle type identification image recognition processing for use in vehicle type identification.
When the image recognition unit 18 recognizes the image of the target vehicle, for example, the image recognition unit 18 performs binarization processing on the image information G2 acquired by the front camera 11b (see FIG. 4B) serving as image information acquisition means in the present application. Edge detection processing and further expansion / contraction processing are performed to extract a characteristic image area included in the image information G2. That is, when there are image areas corresponding to the vehicle outline information Cb1, the vehicle lamp information Cb2, and the vehicle lamp bright spot array information Cb3 described above, each area is a closed area, extremely high brightness, etc. Therefore, corresponding image areas can be extracted for each of the vehicle contour, the vehicle lamp, and the vehicle lamp luminescent spot array (at this image processing stage, the extracted image areas are respectively the vehicle contours). It is not specified until it corresponds to the vehicle lamp and the vehicle lamp bright spot array).
For example, as shown in FIG. 5, the image region corresponding to the vehicle contour information Cb1 is a closed region identified by a specific color with respect to the image region around the vehicle, and thus is extracted according to such a condition. The The image area extracted in this way is set as a vehicle contour candidate image area Ia. Then, after finishing the above extraction process, the image recognition unit 18 uses the vehicle contour information Cb1 stored separately in the vehicle database DB3 for the extracted vehicle contour candidate image area Ia to perform pattern matching. I do. As a result, when a suitable one is found, the vehicle type identification information that the vehicle type of the vehicle traveling ahead is the corresponding vehicle type in the outline of the vehicle is output.

  The image area corresponding to the vehicle lamp information Cb2 is a closed area where a specific color (often in the red system) is identified with respect to the image area around the lamp and a bright spot may appear inside. , Extracted according to such conditions. The image area extracted in this way is set as a vehicle lamp candidate image area Ib. Then, after finishing the above extraction process, the image recognition unit 18 separately uses the vehicle lamp information Cb1 and the vehicle contour information Cb2 stored in the vehicle database DB3 for the extracted vehicle lamp candidate image area Ib. Use for pattern matching. When a suitable vehicle is found, vehicle type identification information indicating that the vehicle type of the vehicle traveling ahead is the vehicle type corresponding to the lamp and contour of the vehicle is output.

  Further, the vehicle lamp luminescent spot array information Cb3 is an area where the brightness difference from the lamp image area corresponding to the lamp is large (the brightness difference is large when the brake operation is performed in the preceding vehicle). The bright spot portion is separated and extracted by threshold processing in the image with the lamp turned on. It is extracted according to such conditions. The image area extracted in this way is the vehicle lamp bright spot array candidate image area Ic. Then, after finishing the above extraction process, the image recognition unit 18 separately supplies the vehicle lamp bright spot array information Cb3 stored in the vehicle database DB3 for the extracted vehicle lamp bright spot array candidate image area Ic. The pattern matching is performed using the vehicle lamp information Cb2 and the vehicle contour information Cb1. If a suitable vehicle is found, vehicle type identification information is output that the vehicle type of the vehicle traveling ahead is the vehicle type corresponding to the bright spot arrangement, lamp, and contour of the vehicle. Accordingly, the vehicle type identification image recognition unit 18b functions as a vehicle type extraction unit in the present application and also functions as a vehicle type identification unit which is a kind of vehicle information acquisition unit.

7). Self-vehicle position information correction unit The self-vehicle position information correction unit 19 is a result of image recognition processing of the target feature by the image recognition unit 18 and the position of the target feature included in the feature information F about the target feature. Based on the information, the vehicle position information P is corrected. In the present embodiment, the vehicle position information correction unit 19 includes the image recognition result of the target feature by the image recognition process for position correction based on the feature information F in the image recognition unit 18 and the feature information F included in the feature information F. The own vehicle position information P is corrected along the traveling direction of the own vehicle 50 using the position information of the target feature. Specifically, the own vehicle position information correction unit 19 first, based on the result of the position correction image recognition processing by the image recognition unit 18 and the mounting position, mounting angle, and angle of view of the imaging device 11, The positional relationship between the host vehicle 50 and the target feature at the time of acquiring the image information G1 including the image of the target feature is calculated. Next, the own vehicle position information correction unit 19 calculates the positional relationship between the own vehicle 50 and the target feature, and the position information of the target feature included in the feature information F about the target feature. Based on the above, the position information of the subject vehicle 50 in the traveling direction of the subject vehicle 50 (feature information F) is calculated and acquired with high accuracy. Then, the own vehicle position information correction unit 19 includes the own vehicle position information P acquired by the own vehicle position information acquisition unit 16 based on the highly accurate position information of the own vehicle 50 acquired in this way. Information on the current position in the traveling direction of the vehicle 50 is corrected. As a result, the host vehicle position information acquisition unit 16 acquires the corrected host vehicle position information P with high accuracy.

8). Navigation Calculation Unit The navigation calculation unit 20 operates in accordance with the application program 23 in order to execute navigation functions such as display of the vehicle position, route search from the departure place to the destination, route guidance to the destination, and destination search. Arithmetic processing means. For example, the navigation calculation unit 20 acquires map information M around the host vehicle 50 from the map database DB1 based on the host vehicle position information P, displays a map image on the display input device 21, and displays the map image. On top of this, based on the vehicle position information P, a process of displaying the vehicle position mark in an overlapping manner is performed. The navigation calculation unit 20 searches for a route from a predetermined departure point to a destination based on the map information M stored in the map database DB1. Further, the navigation calculation unit 20 uses one or both of the display input device 21 and the voice output device 22 based on the searched route from the departure point to the destination and the own vehicle position information P, and the driver. Provides route guidance for. In the present embodiment, the navigation calculation unit 20 is connected to the display input device 21 and the audio output device 22. The display input device 21 is a combination of a display device such as a liquid crystal display device and an input device such as a touch panel. The audio output device includes a speaker and the like. In the present embodiment, the navigation calculation unit 20, the display input device 21, and the audio output device 22 function as the guidance information output means 24 in the present invention. The display input device 21 and the audio output device 22 are also used for outputting vehicle type introduction information, which will be described later, and for inputting whether or not introduction is permitted. Therefore, it serves as an input / output device in the image recognition apparatus 100 according to the present application. Further, the process of handing over the obtained vehicle information from the image recognition apparatus 100 according to the present application to the navigation function unit provided in the navigation apparatus 1 also corresponds to the output process of the image recognition apparatus 100.

9. Introduction calculation unit The introduction calculation unit 30 is a part that functions as an introduction output unit that introduces the vehicle type identified by the image processing unit 18 for the target vehicle, and is stored and stored in the vehicle database DB3 for each vehicle type. At least the vehicle basic information Ca can be introduced. It functions as an introduction output means in the image identification apparatus 100 according to the present application.
As described above, the vehicle database DB3 stores and stores vehicle type information, address information on a communication network introducing the vehicle type, dealer information handling the vehicle type, and the like for each vehicle type. Yes.
The vehicle type information includes the name of the vehicle type, the concept of the vehicle type, the topic, the grade, the price, and the like. And when introducing the car model, “The car running in front is“ A ”. A supercharger appeared in this "A". In addition, minor changes have been made, and at the same time special edition cars are on sale. Want to consider buying once? The database has been constructed so that it can be introduced.

As shown in FIG. 1, the introduction calculation unit 30 includes a vehicle type introduction unit 30a and a dealer introduction unit 30b. These introduction sections 30a and 30b each have an introduction permission acquisition means 31, and can be introduced by the vehicle type introduction section 30a and the dealer introduction section 30b separately using the display input device 21 or the audio output device 22. The vehicle type of the target vehicle, the introduction of the dealer, and the like are configured only when the command is received (from the navigation device side, permission is given that the introduction may be performed).
The introduction permission acquisition means 31 is configured so that when each introduction is necessary, the user designates whether or not both introductions are permitted via the display input device 21 or the voice output device 22, or asks the user to inquire from the device side. By confirming the above, the operation of both the introduction units 30a and 30b is started. Therefore, when the user determines that the introduction is unnecessary, each introduction that is a feature of the present application is not performed.

The vehicle type introduction unit 30a introduces the vehicle type name, application, price, sales, and the like for the vehicle type identified by the image recognition unit 18 in the situation where the introduction is permitted as described above. The introduction is, for example, “The car running in front is“ A ”. This car was developed for the family and its price range is 3 to 4 million. It is often purchased as a second car for families with three generations. Etc.] is introduced.
Further, the vehicle type introduction unit 30a is also configured to introduce an address on the communication network that introduces the recognized vehicle in accordance with an input instruction from the user. Therefore, for example, when returning home, the display input device 21 receives an introduction of the Internet address of the vehicle that was running forward while traveling, and the user accesses the homepage of the vehicle type, so that more detailed information on the vehicle type can be obtained. Obtainable. Therefore, for example, the appearances of various colored vehicles from various directions can be confirmed.

  The vehicle type introduction unit 30a is provided with a unique storage unit 32. The storage unit 32 stores user information U that is information related to the user. A driving state information acquisition unit 33 that acquires the state information D is provided, and a configuration for introducing according to the user information U and the driving state information D is adopted.

  The user information U includes, for example, the model of the vehicle that the user owns or rides, the mileage of the vehicle, the family structure of the user, and the like. If you were asked, “Recently, specially-designed vehicles are on sale. The mileage has reached 80,000 km. How about buying a new one? 』Etc can be introduced. Furthermore, for information on the family structure, “How about“ B ”driving in front of you as your second car? Can be introduced.

  The traveling state information D includes, for example, the traveling speed, traveling acceleration, brake ON / OFF, accelerator ON / OFF, etc. for the host vehicle that is traveling. And, for example, when introducing a vehicle that is traveling ahead, the introduction that has been described so far is performed under the condition that the accelerator is depressed and the traveling speed and traveling acceleration are below a predetermined value. It is configured. The reason for adopting such a configuration is that the introduction in the present application does not cause any trouble even if the introduction is delayed, and it is not inevitable, and it is sufficient to perform the introduction at a time when there is room for the user.

The dealer introduction unit 30b introduces a dealer who handles the vehicle type identified by the image processing unit 18.
In the situation where introduction is permitted as described above, the dealer introduction unit 30b provides the name of the dealer handling the vehicle type, the location of the nearest dealer, etc., for the vehicle type identified by the image recognition unit 18. introduce. For example, “The car running in front is sold by“ X ”. The nearest dealer is “XX City XXX Town”. Etc.] is introduced.
Further, in the processing in the dealer introduction section 30b, as a question to the user, “Would you like to go to the nearest dealer? "Can you let the nearest dealer know that you were interested in" A "?" Ask questions such as. If the answer to the former question is “YES”, the guide route to the dealer is searched and displayed. On the other hand, if the answer to the latter question is “YES”, the dealer is informed that the user is interested in “A” ”. In this way, the user and the dealer can quickly and usefully transmit information.

10. Operation Process of Navigation Device Next, the vehicle position information correction process, the vehicle type identification process for the recognized vehicle, and the introduction process for the recognized vehicle, which are executed in the navigation apparatus 1 according to the present embodiment, will be described individually.

10-1 Own Vehicle Position Information Correction Process As shown in FIG. 6, in the navigation apparatus 1, first, the own vehicle position information acquisition unit 16 acquires own vehicle position information P (step # 01). Next, the image information acquisition unit 12 acquires image information G1 around the host vehicle 50 captured by the imaging device (specifically, the back camera 11a) (step # 02). Thereafter, the feature information acquisition unit 17 performs a period from the current position (vehicle position) of the host vehicle 50 indicated in the host vehicle position information P to the end of the link k representing the road on which the host vehicle 50 is traveling. The feature information F of the existing target feature is acquired from the feature database DB2 (step # 03). Then, the image recognition unit 18 sets the recognition area E and performs image recognition processing for each target feature indicated by the feature information F acquired in step # 03 (step # 04). When the target feature is image-recognized in the recognition area E in step # 04 (step # 05: Yes), the vehicle position information correction unit 19 uses the image recognition result of the target feature and the The own vehicle position information P is corrected based on the position information of the target feature included in the feature information F about the target feature (step # 06). On the other hand, when the target feature cannot be recognized in the recognition area E (step # 05: No), the navigation device 1 ends the vehicle position correction process in the navigation device 1 as it is.

10-2 Vehicle Type Identification Process As shown in FIG. 8, in this process, first, the image information acquisition unit 12 acquires image information G2 ahead of the host vehicle 50 imaged by the imaging device 11b (step #). 10).
Next, the image recognition unit 18 performs binarization processing, edge detection processing, expansion / contraction processing, and the like on the acquired image information G2, and extracts a vehicle contour candidate image region Ia (step # 11). . Then, after finishing the above extraction process, the image recognition unit 18 uses the vehicle contour information Cb1 stored separately in the vehicle database DB3 for the extracted vehicle contour candidate image area Ia to perform pattern matching. Is performed (step # 12). And when the suitable thing is found (step # 13: Yes), the vehicle type identification information (this information is that the vehicle type of the vehicle traveling ahead is the corresponding vehicle type in the outline of the vehicle. This is one of the vehicle type information, and is described as “vehicle type information” in FIG. 8 (step # 14).
If no headline can be obtained (step # 13: No), the following processing is sequentially performed.

  Subsequently, the image recognition unit 18 extracts a vehicle lamp candidate image region Ib in the vehicle contour candidate image region Ia (step # 15). Then, after completing the above extraction process, the image recognition unit 18 performs pattern matching on the extracted vehicle lamp candidate image area Ib using the vehicle lamp information Cb2 separately stored in the vehicle database DB3. Perform (Step # 16). At this time, the vehicle contour candidate image area Ia and the vehicle contour information Cb1 are also used. If a suitable vehicle is found (step # 17: Yes), vehicle type identification information is output that the vehicle type of the vehicle traveling ahead is the corresponding vehicle type in the vehicle lamp (step #). 14). If no headline can be obtained (step # 17: No), the following processing is subsequently performed.

Further, the image recognition unit 18 extracts a vehicle lamp bright spot array candidate image area Ic in the vehicle lamp candidate image area Ib (step # 18). Then, after finishing the above extraction process, the image recognition unit 18 separately uses the vehicle lamp luminescent spot array information Cb3 stored in the vehicle database DB3 for the extracted vehicle lamp luminescent spot array candidate image area Ic. Using this, pattern matching is performed (step # 19). At this time, the vehicle lamp candidate image area Ib and the vehicle lamp information Cb2 are also used. And when a suitable thing is found (step # 20: Yes), the vehicle type identification information that the vehicle type of the vehicle traveling ahead is the vehicle type corresponding to the array of bright spots in the vehicle lamp. Is output (step # 14). Even if the vehicle type cannot be identified in this way, vehicle type information cannot be output (step # 21).
In this way, the vehicle type identification process is completed.

10-3 Introduction Process As shown in FIG. 9, the introduction calculation unit 30 obtains permission for introduction of the recognized vehicle by operating the display input device 21 (step # 30). When permission is obtained (step # 30: Yes), the vehicle type of the recognized vehicle is introduced. On the other hand, when it is not obtained (step # 30: No), the process is terminated without introducing.

  Now, with the vehicle type identified for the recognized vehicle from the image information G2 captured by the front camera by the image recognition unit 18, the introduction calculation unit 30 acquires the vehicle type information (step # 31). On the other hand, in this state, the user information of the host vehicle 50 provided with the navigation device 1 is confirmed and the traveling state information is confirmed (step # 32). By this confirmation, the introduction content is determined based on the vehicle basic information and user information of the identified vehicle type (step # 33).

On the other hand, it is determined whether or not the running state is in a state where introduction is possible. If it is in a state where introduction is possible (step # 34: Yes), the introduction content including the vehicle type information determined in advance for the recognized vehicle is introduced. (Step # 35). If the referral is not possible (Step # 34: No),
Wait until the driving state is ready for introduction.

  Next, permission is given for whether or not the dealer can be introduced for the recognized vehicle by operating the display input device 21 (step # 36). If permission is obtained, the dealer is introduced for the identified vehicle (step # 37). Then, the driver is asked whether he / she wants to go to the nearest dealer. If the answer to the question is Yes (step # 38: Yes), route guidance to the dealer is executed (step # 39). If the answer is No (step # 38: No), the process is finished as it is.

  Further, regarding whether or not the user saw the identified vehicle, a question is asked as to whether or not the fact may be notified to the dealer, and if the answer to the question is Yes (step # 40: Yes). The dealer is notified (step # 41). When the answer is No (step # 40: No), the process is finished as it is.

[Another embodiment]
(1) In the above embodiment, the image recognition unit 18 includes the vehicle type identification image recognition unit 18b, and functions as the other vehicle image extraction unit and recognizes the vehicle type of the vehicle to be recognized. The case where the vehicle functions as the vehicle type identification means is described.
Here, in the present application, the vehicle type information is obtained as the vehicle information of the other vehicle from the inherent relationship between the arrangement of the light sources in the vehicle lamp and the vehicle type, which makes image recognition relatively easy. As described above, the arrangement of the light sources in the vehicle lamp (in the image, the arrangement of the bright spots in the vehicle lamp) is recognized with relatively high reliability, and the vehicle type can be identified. In the case where the vehicle is present, vehicle information such as the presence of the vehicle, the distance between the vehicle and the vehicle, and how the vehicle is moving is substantially captured. Therefore, such vehicle information may be used for generating guidance information in the navigation device.

(2) In the above embodiment, in identifying the vehicle type, the identification process is executed in the order of the contour of the vehicle, the shape of the vehicle lamp, and the arrangement of the light sources (bright spots) in the vehicle lamp. Naturally, the arrangement of the light sources (bright spots) in the vehicle lamp is used alone, and the light source (bright spots) arrangement in the vehicle lamp and / or the vehicle contour or the shape of the vehicle lamp are identified. It may be an element. Further, in the case of using the combination of identification elements as an identification element, if the relationship between two or three parties can be used, the identification can further increase the reliability. In addition, the order of the three parties in the identification process may be reversed with respect to the embodiment described above. (3) In the above embodiment, one of the two imaging devices provided as an identification target. However, the object to be imaged is not limited to the front, but may be the rear or even the side. That is, it is only necessary to capture image information of the vehicle in which the user is interested.

  In the case where the front vehicle performs a brake operation even during night driving, it is possible to provide a technology that can accurately recognize the front vehicle by an image recognition method and further accurately identify the vehicle type of the front vehicle.

The block diagram which shows schematic structure of the navigation apparatus which concerns on embodiment of this invention. The figure which shows the example of a structure of the map information memorize | stored in the map database The figure which shows the example of the feature information of the road marking memorize | stored in the feature database The figure which shows an example of the arrangement structure of the imaging device (back camera and front camera) to the own vehicle The figure which shows the example of the vehicle information memorize | stored in the vehicle database Flowchart of correction of own vehicle position information of navigation device according to embodiment of present invention Explanatory drawing of the own vehicle position information correction | amendment of the navigation apparatus which concerns on embodiment of this invention The flowchart of the vehicle type identification process of the navigation apparatus which concerns on embodiment of this invention The flowchart of the introduction process of the navigation device according to the embodiment of the present invention.

Explanation of symbols

1: Navigation device 12: Image information acquisition unit (image information acquisition means)
16: Own vehicle position information acquisition unit (own vehicle position information acquisition means)
17: Feature information acquisition unit 18: Image recognition unit (other vehicle image extraction means: vehicle information acquisition means (vehicle type identification means))
19: Own vehicle position information correction unit (own vehicle position information correction means)
23: Application program 24: Guidance information output means 30: Introduction calculation section (introduction output means)
50: Own vehicle DB1: Map database DB2: Feature database DB3: Vehicle database E: Recognition area F: Feature information G1: Image information G2: Image information k: Link n: Node

Claims (10)

Image information acquisition means for acquiring image information around the host vehicle;
From the image information acquired by the image information acquisition unit, and the other vehicle image extracting means for extracting an image of another vehicle that exists around the own vehicle,
The other from the vehicle image extracting unit image extracted by addition to image recognition sequence of the other vehicle in the vehicle lamp Ru lighting state near provided in the light source as an array of bright spots, the image recognized vehicle the lamp Vehicle information acquisition means for obtaining vehicle information of other vehicles based on the arrangement of light sources in
An image recognition apparatus for outputting vehicle information obtained by the vehicle information acquisition means.   The vehicle information acquisition means recognizes the vehicle lamp of the other vehicle from the image extracted by the other vehicle image extraction means, and obtains vehicle information of the other vehicle based on the shape of the vehicle lamp recognized as the image. The image recognition apparatus according to claim 1.   The image recognition apparatus according to claim 2, wherein the vehicle information acquisition unit obtains vehicle information of another vehicle based on a relationship between an array of light sources in the vehicle lamp recognized as the image and a shape of the vehicle lamp.   2. The vehicle information acquisition unit recognizes the contour of the other vehicle from the image extracted by the other vehicle image extraction unit, and obtains vehicle information of the other vehicle based on the image-recognized contour. Image recognition device.   The image recognition apparatus according to claim 4, wherein the vehicle information acquisition unit obtains vehicle information of another vehicle based on a relationship between an array of light sources in the vehicle lamp that has been image-recognized and a contour of the other vehicle.   The image recognition apparatus according to claim 1, wherein the vehicle information acquisition unit is configured as a vehicle type identification unit that identifies a vehicle type of the other vehicle, and outputs an identification result of the vehicle type identification unit. The image recognition apparatus according to any one of claims 1 to 6, comprising:
Own vehicle position information acquisition means for acquiring own vehicle position information representing the current position of the own vehicle;
A navigation device that provides road guidance based on the vehicle position information acquired by the vehicle position information acquisition means.   The navigation apparatus according to claim 7, further comprising an introduction output unit that introduces vehicle type introduction information related to the identified vehicle type output by the image recognition device.   The introduction output means introduces one or more types selected from the name, concept, topic, grade, price, address information on the communication network that introduces the vehicle type, and information on the dealer that sells the vehicle type. The navigation device according to claim 8, which is configured to be possible. An image information acquisition step of acquiring image information around the host vehicle;
From image information acquired by the image information acquisition step, and another vehicle image extraction step of extracting an image of another vehicle that exists around the own vehicle,
The other from a vehicle image extracting step with the extracted image while image recognition sequence of the other vehicle in the vehicle lamp Ru lighting state near provided in the light source as an array of bright spots, the image recognized vehicle the lamp Vehicle information acquisition step of obtaining vehicle information of other vehicles based on the arrangement of light sources in
An image recognition program for storing and executing an output step of outputting vehicle information obtained in the vehicle information acquisition step in a computer.

Images