JP2011198002A - Object image recognition apparatus for vehicle, object image recognition method for vehicle, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object image recognition apparatus, an object image recognition method and a computer program for vehicle that implement image recognition through simpler processing than before.SOLUTION: The apparatus is configured to detect a relative angle (object relative angle) of an object included in an image captured by a back camera with respect to an optical axis direction of the back camera (S5), to detect a relative angle (reference relative angle) of a reference object included in the image captured by the back camera with respect to the optical axis direction of the back camera (S7), to calculate a relative angle (comparison relative angle) of the object with respect to the reference object, on the basis of the detected reference relative angle and the object relative angle (S9), and to store the calculated comparison relative angle in association with the object into an object DB to form the object DB (S10).

Description

  The present invention relates to a vehicle object image recognition apparatus, a vehicle object image recognition method, and a computer program for recognizing a target object around a vehicle.

  2. Description of the Related Art Conventionally, there has been proposed a driving support device that acquires vehicle traveling information specified by various sensors such as map information and GPS, and notifies a driver or assists driving. Examples of such a driving support device include a navigation device, a pre-crash safety system, an auto cruise control system, a lane keeping assist system, and the like.

  In such a driving support device, an imaging device such as a camera is provided on the front surface of the vehicle, and an object (a person, an object, a sign, etc.) around the vehicle is imaged and an image of the captured image is displayed. Some perform recognition processing to recognize an object, and perform notification and vehicle control based on the recognition result. For example, there is an apparatus that recognizes an obstacle such as a person or a bicycle located around a vehicle from a captured image and warns the obstacle based on the type and position of the recognized obstacle. In addition, there is a vehicle that recognizes a forward vehicle located in front of the vehicle from a captured image and controls an accelerator and a brake based on the recognized distance from the forward vehicle. It also recognizes road signs such as maximum speed and lane markings formed on the road on which the vehicle travels from the captured image, and identifies the detailed current position of the vehicle based on the relative position of the recognized road sign to the vehicle. Or perform steering control.

  Here, there are the following problems when an object is imaged by an imaging device installed in a vehicle and the object is recognized based on the captured image. That is, since the positional relationship and angle of the imaging device with respect to the object change depending on the road shape and the running mode of the vehicle, the shape of the object in the captured image changes every time even when the same object is imaged. Therefore, for example, in Japanese Patent Application Laid-Open No. 8-320990, in a technique for recognizing a vehicle license plate from an image captured by a camera, a distortion amount of a vehicle number included in the license plate in the captured image is detected and detected. It describes correcting the distortion of the vehicle number based on the amount of distortion and recognizing the vehicle number based on the corrected image.

JP-A-8-320990 (pages 3 to 5)

  However, in the technique described in Patent Document 1, the image recognition process is performed after executing the process of detecting the distortion amount of the vehicle number and the process of correcting the distortion of the vehicle number based on the detected distortion amount. Therefore, there is a problem that the burden on the CPU related to the processing becomes large. In the image recognition process, there is a template matching method as another recognition method for specifying the type of captured object. In template matching, it is necessary to perform scanning by changing the size and angle of the template little by little to repeatedly determine whether an object in the captured image matches the template. Here, the size of the object on the road does not change so much, but the angle of the object when imaged changes greatly depending on the direction of the vehicle and the shape of the road. Therefore, in the image recognition processing by template matching, the processing time is prolonged and the processing load on the CPU is increased.

  The present invention has been made in order to solve the above-described problems in the related art. For a target object to be recognized, an image for detecting a target object after the relative angle with respect to a reference object is previously stored in a database. When executing recognition processing, it is possible to grasp in advance the relative angle of the target object with respect to the imaging device, and it is possible to execute image recognition processing by simpler processing than before. An object of the present invention is to provide a vehicle object image recognition method and a computer program.

In order to achieve the above object, a vehicle object image recognition device (1) according to claim 1 of the present application includes an imaging means (13) for imaging the periphery of the vehicle by an imaging device (19) installed in the vehicle, Reference relative angle detection means (13) for detecting, as a reference relative angle, a relative angle of the reference object (52) included in the captured image captured by the imaging means as a reference relative angle, and imaging captured by the imaging means Based on the target relative angle detection means (13) that detects the relative angle of the target object (51) included in the image as the target relative angle with respect to the imaging device, and based on the reference relative angle and the target relative angle, the reference A comparative relative angle calculating means (13) for calculating a relative angle of the target object with respect to the object as a comparative relative angle; The relative angle storing means for storing the serial in association with the object storage medium (33) (13), characterized in that it has a.
The “target object” and the “reference object” are objects whose positions and angles are fixed on or near the road, and include obstacles, buildings, road signs (including road markings), and the like.

  A vehicle object image recognition device (1) according to claim 2 is the vehicle object image recognition device according to claim 1, wherein the target object (51) is located around the vehicle. A target object determination unit (13) for determining whether or not the target object is positioned around the vehicle by the target object determination unit, and a captured image captured by the imaging unit (19) A first relative angle detecting means (13) for detecting a relative angle of the reference object (52) included in the imaging device as a first relative angle, and the comparison relative angle stored in the storage medium (33). And the comparison relative angle stored in association with the target object determined to be located around the vehicle by the target object determination means Based on the angle extracting means (13) for extracting, and the comparison relative angle and the first relative angle extracted by the angle extracting means, the target object determining means determines that the object is positioned around the vehicle. A second relative angle detection unit (13) that detects a relative angle of the target object with respect to the imaging device as a second relative angle, and based on the second relative angle, the target object determination unit is positioned around the vehicle. And target object recognition means (13) for performing image recognition processing on the captured image for detecting the determined target object.

  Further, the vehicle object image recognition device (1) according to claim 3 includes an imaging means (13) for imaging the periphery of the vehicle by an imaging device (19) installed in the vehicle, and a target for the reference object (52). Relative angle storage means (13) for storing the relative angle of the object (51) as a comparative relative angle in the storage medium (33) in association with the target object, and whether or not the target object is positioned around the vehicle. When the target object determining means (13) for determining and the target object determining means determine that the target object is positioned around the vehicle, the reference object included in the captured image captured by the imaging means First relative angle detection means (13) for detecting a relative angle with respect to the imaging device as a first relative angle; and the storage medium An angle extracting means (13) for extracting the stored relative relative angle stored in association with the target object determined to be located around the vehicle by the target object determining means among the stored comparative relative angles; Based on the comparative relative angle and the first relative angle extracted by the angle extraction means, the relative angle of the target object with respect to the imaging device determined by the target object determination means to be located around the vehicle And a second relative angle detecting means (13) for detecting the target object determined to be positioned around the vehicle by the target object determining means based on the second relative angle. Target object recognition means (13) for performing image recognition processing on the captured image for the purpose , Characterized by having a.

  The vehicle object image recognition method according to claim 4 is included in an imaging step of imaging the periphery of the vehicle by an imaging device (19) installed in the vehicle, and a captured image captured by the imaging step. A reference relative angle detection step of detecting a relative angle of the reference object (52) with respect to the imaging device as a reference relative angle, and a relative of the target object (51) included in the captured image captured by the imaging step with respect to the imaging device. A target relative angle detecting step for detecting an angle as a target relative angle; and a relative relative angle calculation for calculating a relative angle of the target object with respect to the reference object as a comparative relative angle based on the reference relative angle and the target relative angle A storage medium that associates the comparison relative angle with the target object And relative angle storage step of storing in 33), characterized by having a.

  Furthermore, the computer program according to claim 5 is included in an image pickup function that is mounted on a computer and picks up the periphery of the vehicle by an image pickup device (19) installed in the vehicle, and a picked-up image picked up by the image pickup function. A reference relative angle detection function for detecting a relative angle of the reference object (52) with respect to the imaging device as a reference relative angle, and a relative of the target object (51) included in the captured image captured by the imaging function with respect to the imaging device. Comparative relative angle calculation that calculates a relative angle of the target object relative to the reference object based on the reference relative angle and the target relative angle based on a target relative angle detection function that detects an angle as a target relative angle. The storage medium (the function and the comparison relative angle are associated with the target object) And the relative angle storage function of storing the 3), characterized in that for the execution.

  In the vehicle object image recognition device according to claim 1 having the above-described configuration, an image for detecting a target object thereafter by creating a database of relative angles with respect to a reference object for the target object to be recognized. When executing the recognition process, the relative angle of the target object with respect to the imaging device can be grasped in advance. Therefore, it is possible to execute an image recognition process for detecting the target object by a simpler process than before. As a result, the processing time required for the image recognition process can be shortened, and the processing load on the CPU can be reduced.

  The vehicle object image recognition apparatus according to claim 2 executes an image recognition process for detecting a target object to be recognized by using a relative angle of the target object with respect to a reference object stored in a database. It is possible to grasp in advance the relative angle of the target object with respect to the imaging device. As a result, it is possible to execute an image recognition process for detecting the target object by a simpler process than before.

  In the vehicle object image recognition device according to claim 3, an image recognition process for detecting a target object to be recognized is executed by providing a database storing a relative angle of the target object with respect to the reference object. It is possible to grasp in advance the relative angle of the target object with respect to the imaging device. As a result, it is possible to execute an image recognition process for detecting the target object by a simpler process than before.

  Further, in the vehicle object image recognition method according to claim 4, an image recognition process for detecting a target object after the relative angle with respect to the reference object is previously databased for the target object to be recognized. When executing the above, it is possible to grasp in advance the relative angle of the target object with respect to the imaging device. Therefore, it is possible to execute an image recognition process for detecting the target object by a simpler process than before. As a result, the processing time required for the image recognition process can be shortened, and the processing load on the CPU can be reduced.

  Furthermore, in the computer program according to claim 5, when a target object to be recognized is previously stored in a database as a relative angle with respect to a reference object, an image recognition process for detecting the target object is subsequently executed. The relative angle of the target object with respect to the imaging device can be grasped in advance. Therefore, it is possible to execute an image recognition process for detecting the target object by a simpler process than before. As a result, the processing time required for the image recognition process can be shortened, and the processing load on the CPU can be reduced.

It is the block diagram which showed the navigation apparatus which concerns on this embodiment. It is the figure which showed an example of the memory area of road marking DB. It is the figure which showed an example of the storage area of object DB. It is the figure which showed an example of the target object and the reference | standard object. It is a figure showing an example of a target object and a center line. It is a flowchart of the image recognition processing program which concerns on this embodiment. It is a flowchart of the image recognition processing program which concerns on this embodiment. It is the figure which showed the example of a detection of object relative angle and reference | standard relative angle.

  Hereinafter, a vehicle object image recognition device according to the present invention will be described in detail with reference to the drawings based on an embodiment embodied in a navigation device. First, a schematic configuration of the navigation device 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a navigation device 1 according to this embodiment.

  As shown in FIG. 1, the navigation apparatus 1 according to the present embodiment is based on a current position detection unit 11 that detects a current position of a vehicle, a data recording unit 12 that records various data, and input information. The navigation ECU 13 that performs various arithmetic processes, the operation unit 14 that receives operations from the user, the liquid crystal display 15 that displays a map and a guide route to the destination, and voice guidance related to route guidance are output to the user. A speaker 16, a DVD drive 17 that reads a DVD that is a storage medium storing a program, and a communication module 18 that communicates with an information center such as a traffic information center. In addition, a back camera 19 for detecting a reference object and a target object, which will be described later, is connected to the navigation device 1.

Below, each component which comprises the navigation apparatus 1 is demonstrated in order.
The current position detection unit 11 includes a GPS 21, a vehicle speed sensor 22, a steering sensor 23, a gyro sensor 24, an altimeter (not shown), and the like, and can detect the current vehicle position, direction, vehicle traveling speed, and the like. It has become. Here, in particular, the vehicle speed sensor 22 is a sensor for detecting a moving distance and a vehicle speed of the vehicle, generates a pulse according to the rotation of the wheel of the vehicle, and outputs a pulse signal to the navigation ECU 13. And navigation ECU13 calculates the rotational speed and moving distance of a wheel by counting the pulse which generate | occur | produces. Note that the navigation device 1 does not have to include all the four types of sensors, and the navigation device 1 may include only one or more types of sensors.

  The data recording unit 12 includes an external storage device and a hard disk (not shown) as a recording medium, a map information DB 31 recorded on the hard disk, a road marking DB 32, an object DB 33, an object identification information 34, a predetermined program, and the like. A recording head (not shown), which is a driver for reading and writing predetermined data to the hard disk, is provided.

Here, the map information DB 31 stores various map data necessary for route guidance, traffic information guidance, and map display.
The map data specifically includes link data relating to road (link) shape, node data relating to node points, POI data which is information relating to points such as facilities, intersection data relating to each intersection, and search for searching for a route. Data, search data for searching for points, maps, roads, traffic information, and other images are composed of image drawing data for drawing on the liquid crystal display 15 and the like.

  The road marking DB 32 is a DB in which road surface display information related to road markings, which are objects formed on the road surface, is stored. Here, FIG. 2 is a diagram showing an example of a storage area of the road marking DB 32. As shown in FIG. 2, the road marking DB 32 includes coordinate data for specifying the position of the road marking on the map, and the type of road marking formed on the road surface (for example, stop line, pedestrian crossing, character string, maximum speed). ) Identification information, control objects associated with road markings (stop lines, corners, intersections, etc.), and road distances to control objects associated with road markings The

  For example, in the example shown in FIG. 2, a road marking “with a pedestrian crossing” is formed at the coordinates (x1, y1), and the road surface of the “stop line” is 60 m ahead of the road marking as a control object. Indicates that a sign is associated. Then, when the navigation ECU 13 recognizes any road marking recorded in the road marking DB 32 from the captured image captured by the back camera 19, it is based on the road distance to the control object associated with the recognized road marking. The detailed distance from the vehicle to the control object is indirectly calculated. And guidance and vehicle control are performed based on the calculated distance.

The object DB 33 stores a relative angle of a target object that is an object to be recognized in the image recognition processing of the present embodiment with respect to a reference object that is a comparison reference object in association with the target object. It is. Here, in the present embodiment, the target object is a road marking (for example, stop line, pedestrian crossing, character string, maximum speed, etc.) formed on the road surface. The basic object is a lane line (a lane center line, a lane boundary line, a lane outside line, etc.), but an object other than the lane line may be a basic object. Here, FIG. 3 is a diagram showing an example of the storage area of the object DB 33. As shown in FIG. 3, the object DB 33 identifies the coordinate data for specifying the position of the target object on the map and the type of the target object (for example, stop line, pedestrian crossing, character string, maximum speed). It is composed of information (identifier) and a relative angle with respect to the reference object (hereinafter referred to as a comparative relative angle).
The reference object is an object that can always reliably detect the relative angle to the back camera 19. In the present embodiment, as described above, lane markings (roadway center line, lane boundary line, roadway outer line, etc.) are used. Here, the lane marking is a white line that is greatly different in color and brightness from the road surface that is the background, and has many straight lines. As a result, there is an advantage that the amount of deviation of the edge processing vector is small and the angle can be easily detected. In addition, since the lane markings are arranged on almost all roads, there is an advantage that they can be detected regardless of the road on which the vehicle is traveling.

For example, in the example shown in FIG. 3, it is indicated that the road marking “with crosswalk” exists as the target object at the coordinates (x1, y1), and the target object has a comparative relative angle of 5 degrees with respect to the reference object. Show. The comparative relative angle is defined by an angle formed by the center line of the target object and the center line of the reference object. Here, FIG. 4 is an example in which the target object is a road marking “with a pedestrian crossing” and the reference object is a roadway outside line. As shown in FIG. 4, the angle θ of the center line 53 of the target object 51 with respect to the center line 54 of the reference object 52 is a comparative relative angle. Note that the center line 53 of the target object 51 is a center line passing through the center of the target object 51 in the left-right direction as shown in FIG. For example, the target object 51 in which two numbers are formed in parallel like the maximum speed is a line passing between the numbers. The target object 51 formed from one character or symbol is a line passing through the center of the character or symbol. Note that an object formed from a character string such as “stop” may be stored in the object DB 33 for each character constituting the character string instead of the character string.
And navigation ECU13 detects previously the relative angle with respect to the optical axis of the back camera 19 of the target object located in the vehicle periphery based on the information regarding the target object memorize | stored in object DB33. And the simplified image recognition process of the target object contained in the picked-up image imaged with the back camera 19 based on the relative angle detected beforehand is performed. Specifically, image recognition processing by pattern matching is performed, and scanning is started with the relative angle calculated as the initial angle of the template angle in template matching at that time being set.
The road marking DB 32 and the object DB 33 may be configured by the same DB.

  The object identification information 34 is information used for image recognition processing of a road marking that is an object formed on the road surface. Specifically, a template for each type of road marking (for example, stop line, pedestrian crossing, character string, maximum speed) is stored.

  On the other hand, the navigation ECU (Electronic Control Unit) 13 performs a guidance route setting process for setting a guidance route from the current position to the destination when the destination is selected, and a back camera 19 installed in the vehicle. Included in the imaging process for imaging the periphery, the process for detecting the relative angle of the reference object included in the captured image with respect to the optical axis of the back camera 19 (hereinafter referred to as the reference relative angle), and the captured image Processing for detecting a relative angle of the target object with respect to the optical axis of the back camera 19 (hereinafter referred to as target relative angle), processing for calculating a comparative relative angle of the target object with respect to the reference object based on the reference relative angle and the target relative angle The object D is obtained by associating the calculated comparative relative angle with the target object. 33, processing for detecting the relative angle of the target object located around the vehicle based on the object DB 33 with respect to the optical axis of the back camera 19, and target object located around the vehicle based on the detected relative angle This is an electronic control unit that performs overall control of the navigation device 1 such as a process for performing a simplified recognition process on the camera, a process for performing guidance and vehicle control based on the recognized target object, and the like. The CPU 41 as an arithmetic device and a control device, the RAM 41 used as a working memory when the CPU 41 performs various arithmetic processes, a RAM 42 for storing route data when a route is searched, and a control program In addition, an internal storage device such as a ROM 43 in which an image recognition processing program (see FIGS. 6 and 7) described later is recorded and a program read from the ROM 43 is stored.

  The operation unit 14 is operated when inputting a departure point as a travel start point and a destination as a travel end point, and includes a plurality of operation switches (not shown) such as various keys and buttons. Then, the navigation ECU 13 performs control to execute various corresponding operations based on switch signals output by pressing the switches. In addition, it can also be comprised by the touchscreen provided in the front surface of the liquid crystal display 15.

  The liquid crystal display 15 also includes a map image including a road, traffic information, operation guidance, operation menu, key guidance, a planned travel route from the departure point to the destination, guidance information along the planned travel route, news, weather Forecast, time, mail, TV program, etc. are displayed.

  Further, the speaker 16 outputs voice guidance for guiding traveling along the planned traveling route and traffic information guidance based on an instruction from the navigation ECU 13.

  The DVD drive 17 is a drive that can read data recorded on a recording medium such as a DVD or a CD. Based on the read data, the map information DB 31 and the road marking DB 32 are updated.

  In addition, the communication module 18 is traffic information composed of information such as traffic jam information, regulation information, and traffic accident information transmitted from a traffic information center such as a VICS (registered trademark: Vehicle Information and Communication System) center or a probe center. For example, a mobile phone or DCM.

  The back camera 19 uses a solid-state image sensor such as a CCD, for example, and is installed near the upper center of the license plate mounted on the rear side of the vehicle, and installed with the line-of-sight direction downward from the horizontal by a predetermined angle. Is done. Then, an image is taken of the rear of the vehicle that is opposite to the traveling direction of the vehicle during traveling. Then, the type and position of the target object and the reference object around the vehicle are detected by performing image recognition processing of the captured image. Then, the object DB 33 is created based on the detected target object and the reference object, and the detailed road distance from the vehicle to the control target (stop line, corner, intersection, etc.) is indirectly calculated.

  Next, an image recognition processing program executed in the navigation device 1 having the above configuration will be described with reference to FIGS. 6 and 7 are flowcharts of the image recognition processing program according to the present embodiment. Here, the image recognition processing program is executed after the ACC of the vehicle is turned on, creates the object DB 33 based on the captured image captured by the back camera 19, and simplifies the target object based on the created object DB 33. This is a program for performing the image recognition processing. 6 and 7 are stored in the RAM 42, ROM 43, etc. provided in the navigation ECU 13 and executed by the CPU 41.

  In the image recognition processing program, first, in step (hereinafter abbreviated as S) 1, the CPU 41 acquires vehicle information. The vehicle information acquired in S <b> 1 includes information on the current position of the vehicle, the direction of the vehicle, and the optical axis direction of the back camera 19. The current position and direction of the vehicle are acquired based on the detection results of the GPS 21 and the gyro sensor 24. Further, for the current position of the vehicle, map matching processing for specifying the current position of the vehicle on the map using the map information stored in the map information DB 31 is also performed. On the other hand, the optical axis direction of the back camera 19 is calculated based on the direction of the vehicle and the design value of the back camera 19. The design value of the back camera 19 is stored in the ROM 43 or the like.

  Next, in S <b> 2, the CPU 41 acquires information on target objects around the vehicle based on the current position and direction of the vehicle acquired in S <b> 1. In the present embodiment, the target object is a road marking formed on the road surface. Therefore, in S2, the CPU 41 determines information on road markings located in the vicinity of the vehicle (for example, 2000 m forward to 500 m behind the vehicle) based on the road marking position information recorded in the road marking DB 32 (see FIG. 2). (Position coordinates of road marking, identifier, associated control object, road distance to the control object) are read from the road marking DB 32.

  Subsequently, in S3, the CPU 41, based on the current position and direction of the vehicle acquired in S1 and the information on the target object acquired in S2, among the target objects acquired in S2, particularly around the vehicle ( It is determined whether or not there is a target object located 30 m forward to 20 m behind the vehicle. When it is determined that there is a target object around the vehicle (S3: YES), the process proceeds to S4. On the other hand, when it is determined that there is no target object around the vehicle (S3: NO), the image recognition processing program is terminated.

  In S4, the CPU 41 determines whether or not the target object located around the vehicle is a target object whose information is already stored in the object DB 33 (FIG. 3). And when it determines with the target object located in the circumference | surroundings of a vehicle being the target object which has already memorize | stored information in object DB33 (S4: YES), it transfers to below-mentioned S11. On the other hand, when it is determined that the target object located around the vehicle is a target object for which information is not yet stored in the object DB 33 (S4: NO), the process proceeds to S5. In S5 and later, processing relating to creation of the object DB 33 is executed, and in S11 and later, image recognition processing of the target object using the object DB 33 is executed.

  In S <b> 5, the CPU 41 performs image recognition processing by normal pattern matching on the captured image captured by the back camera 19, and detects a target object included in the captured image. Note that image recognition processing by normal pattern matching is already known, and thus detailed description thereof is omitted.

  Next, in S <b> 6, the CPU 41 can detect the target object included in the captured image by the image recognition processing by the normal pattern matching performed in S <b> 5, and the target object is relative to the optical axis direction of the back camera 19. It is determined whether or not the angle (target relative angle) α has been detected. Here, the target relative angle α is defined by an angle formed by the optical axis direction of the back camera 19 and the center line of the target object. FIG. 8 shows an example in which the vehicle 61 detects a road marking 62 “with crosswalk” as a target object. As shown in FIG. 8, the angle of the center line 63 of the road marking 62 with respect to the optical axis L of the back camera 19 is the target relative angle α.

  When it is determined that the target relative angle α of the target object with respect to the optical axis direction of the back camera 19 has been detected (S6: YES), the process proceeds to S7. On the other hand, when it is determined that the target relative angle α of the target object with respect to the optical axis direction of the back camera 19 cannot be detected (S6: NO), the image recognition processing program is terminated.

  In S <b> 7, the CPU 41 performs image recognition processing based on normal feature point matching on the captured image captured by the back camera 19, and detects a reference object included in the captured image. In the present embodiment, the reference objects are lane markings (roadway center line, lane boundary line, roadway outer line, etc.) formed on the road surface. Note that image recognition processing by normal feature point matching is already known, and thus detailed description thereof is omitted.

  Next, in S <b> 8, the CPU 41 can detect the reference object included in the captured image by the image recognition process based on the normal feature point matching performed in S <b> 7, and the reference object in the optical axis direction of the back camera 19 can be detected. It is determined whether or not the relative angle (reference relative angle) β has been detected. Here, the reference relative angle β is defined by an angle formed by the optical axis direction of the back camera 19 and the center line of the reference object. FIG. 8 shows an example when the vehicle 61 detects a roadway outer line 64 as a reference object. As shown in FIG. 8, the angle of the center line 65 of the roadway outer line 64 with respect to the optical axis L of the back camera 19 becomes the reference relative angle β.

  If it is determined that the reference relative angle β of the reference object with respect to the optical axis direction of the back camera 19 has been detected (S8: YES), the process proceeds to S9. On the other hand, if it is determined that the reference relative angle β of the reference object with respect to the optical axis direction of the back camera 19 cannot be detected (S8: NO), the image recognition processing program is terminated.

In S9, the CPU 41 calculates a relative angle (comparison relative angle) θ of the target object with respect to the reference object based on the target relative angle α and the reference relative angle β detected by the image recognition processing in S5 and S7. Specifically, it is calculated by the following equation (1).
θ = β−α (1)
(Α: target relative angle, β: reference relative angle, θ: comparative relative angle)

  Subsequently, in S10, the CPU 41 stores the comparison relative angle θ calculated in S9 in the object DB 33 (FIG. 3) in association with the detected target object. Thereby, the object DB 33 is created.

  On the other hand, when it is determined in S4 that the target object located around the vehicle is a target object whose information is already stored in the object DB 33 (S4: YES), in S11, the CPU 41 Information (position coordinates, identifier, comparison relative angle θ) of the target object located around the vehicle is acquired from the object DB 33.

  Next, in S <b> 12, the CPU 41 performs image recognition processing by normal feature point matching on the captured image captured by the back camera 19, and detects a reference object included in the captured image. In the present embodiment, the reference objects are lane markings (roadway center line, lane boundary line, roadway outer line, etc.) formed on the road surface. Note that image recognition processing by normal feature point matching is already known, and thus detailed description thereof is omitted.

  Subsequently, in S13, the CPU 41 can detect the reference object included in the captured image by the image recognition process based on the normal feature point matching performed in S12, and the reference object in the optical axis direction of the back camera 19 can be detected. It is determined whether or not the relative angle (first relative angle) γ has been detected. Here, the first relative angle γ is defined by the angle formed by the optical axis direction of the back camera 19 and the center line of the reference object, similarly to the reference relative angle β.

  If it is determined that the first relative angle γ of the reference object with respect to the optical axis direction of the back camera 19 can be detected (S13: YES), the process proceeds to S14. On the other hand, when it is determined that the first relative angle γ of the reference object with respect to the optical axis direction of the back camera 19 cannot be detected (S13: NO), the process proceeds to S16.

In S14, the CPU 41 determines a target located around the vehicle based on the information (position coordinates, identifier, comparison relative angle θ) acquired from the object DB 33 in S11 and the first relative angle γ detected by the image recognition process in S12. A relative angle (second relative angle) φ of the object with respect to the optical axis direction of the back camera 19 is calculated. Specifically, it is calculated by the following equation (2).
φ = γ−θ (2)
(Φ: second relative angle, γ: first relative angle, θ: comparative relative angle)

  Subsequently, in S <b> 15, the CPU 41 uses the second relative angle φ of the target object with respect to the optical axis direction of the back camera 19 calculated in S <b> 14 to simplify the captured image captured by the back camera 19. Image recognition processing by matching is executed, and target objects included in the captured image are detected. Note that in the image recognition processing by simplified pattern matching, scanning is started by setting the second relative angle φ calculated as the initial angle of the template angle in template matching. Then, template matching is performed by changing the angle of the template around the second relative angle. As a result, the processing time can be simplified as compared with the image recognition processing by normal pattern matching in which the initial angle is started from a fixed angle (for example, 0 degree). In addition, the processing load on the CPU 41 can be reduced. Thereafter, the process proceeds to S17.

  In S <b> 16, the CPU 41 performs image recognition processing by normal pattern matching on the captured image captured by the back camera 19, and detects a target object included in the captured image. Note that image recognition processing by normal pattern matching is already known, and thus detailed description thereof is omitted. Thereafter, the process proceeds to S17.

  In S17, the CPU 41 can detect the target object included in the captured image by the image recognition process based on the simplified pattern matching performed in S15 or the image recognition process based on the normal pattern matching performed in S16. To determine.

  And when it determines with the target object having been detected (S17: YES), it transfers to S18. On the other hand, if it is determined that the target object could not be detected (S17: NO), the image recognition processing program is terminated.

  In S18, the CPU 41 executes guidance and vehicle control processing based on the detected target object. For example, the detailed distance from the vehicle to the control object is indirectly calculated based on the road distance to the control object (stop line, corner, intersection, etc.) associated with the detected target object. Guidance and vehicle control are performed based on the distance.

As described above in detail, in the navigation apparatus 1 according to the present embodiment, the vehicle object image recognition method using the navigation apparatus 1, and the computer program executed by the navigation apparatus 1, the captured image captured by the back camera 19 is included in the captured image. The relative angle (target relative angle) of the included target object with respect to the optical axis direction of the back camera 19 is detected (S5). Similarly, the light of the back camera 19 of the reference object included in the captured image captured by the back camera 19 is detected. A relative angle (reference relative angle) with respect to the axial direction is detected (S7), and a relative angle (comparison relative angle) of the target object with respect to the reference object is calculated based on the detected reference relative angle and target relative angle (S9). ), The calculated comparative relative angle is associated with the target object Stored in-object DB 33, to create an object DB 33 (S10). Thereafter, when executing an image recognition process for detecting the target object stored in the object DB 33, the relative angle of the reference object included in the captured image with respect to the optical axis direction of the back camera 19 (first relative angle). Is detected (S12), and a relative angle (second relative angle) of the target object with respect to the optical axis direction of the back camera 19 is calculated based on the detected first relative angle and the comparative relative angle stored in the object DB 33. (S14) Based on the calculated second relative angle, an image recognition process is performed on the captured image for detecting the target object (S15), so an image recognition process (S15) for detecting the target object is executed. In this case, the relative angle of the target object with respect to the optical axis direction of the back camera 19 is grasped in advance. Theft is possible. Therefore, it is possible to execute an image recognition process for detecting the target object by a simpler process than before. As a result, the processing time required for the image recognition process can be shortened, and the processing load on the CPU can be reduced.
In the image recognition process (S15) based on simplified pattern matching, scanning is started after setting the second relative angle calculated as the initial angle of the template angle in template matching. Then, template matching is performed by changing the angle of the template around the second relative angle. As a result, the processing time can be simplified as compared with the image recognition processing by normal pattern matching in which the initial angle is started from a fixed angle (for example, 0 degree).

Note that the present invention is not limited to the above-described embodiment, and various improvements and modifications can be made without departing from the scope of the present invention.
For example, in this embodiment, a road marking (stop line, pedestrian crossing, character string, maximum speed, etc.) is detected as a target object. However, the target object is not limited to a road marking, but on or near a road. As long as it has a fixed position and angle, it may be an obstacle, a building, another road sign, or the like.

  Similarly, in the present embodiment, the lane marking is used as the reference object, but other objects may be used as long as the position and angle are fixed on or near the road. Further, the reference object is not limited to one type, and a different reference object may be used depending on the surrounding road shape and environment. In that case, it is necessary to store the information specifying the reference object in the object DB 33 in association with the comparative relative angle.

  In this embodiment, after the target object is detected by the image recognition process, the control target is detected from the vehicle based on the road distance to the control target object (stop line, corner, intersection, etc.) associated with the detected target object. Although a detailed distance to an object is indirectly calculated and guidance and vehicle control are performed based on the calculated distance, other guidance and vehicle control based on the target object may be performed.

  In this embodiment, when calculating the relative angle with respect to the imaging device, the relative angle with respect to the optical axis of the back camera 19 is calculated. However, the relative angle with respect to another reference direction based on the back camera 19 is calculated. Also good.

1 Navigation device 13 Navigation ECU
41 CPU
42 ROM
43 RAM
19 Back camera 33 Object DB
51 Target object 52 Reference object

Claims (5)

Imaging means for imaging the periphery of the vehicle by an imaging device installed in the vehicle;
Reference relative angle detection means for detecting, as a reference relative angle, a relative angle of a reference object included in a captured image captured by the imaging means with respect to the imaging device;
Target relative angle detection means for detecting, as a target relative angle, a relative angle of a target object included in a captured image captured by the imaging means with respect to the imaging device;
Comparison relative angle calculation means for calculating a relative angle of the target object with respect to the reference object as a comparison relative angle based on the reference relative angle and the target relative angle;
Relative angle storage means for storing the comparison relative angle in a storage medium in association with the target object. Target object determination means for determining whether or not the target object is located around the vehicle;
When the target object determining unit determines that the target object is located around the vehicle, the relative angle of the reference object included in the captured image captured by the imaging unit with respect to the imaging device is a first relative angle. First relative angle detecting means for detecting
Angle extracting means for extracting the comparative relative angle stored in association with the target object determined to be located around the vehicle by the target object determining means among the comparative relative angles stored in the storage medium When,
Based on the comparison relative angle and the first relative angle extracted by the angle extraction means, a relative angle of the target object determined to be positioned around the vehicle by the target object determination means is determined with respect to the imaging device. Second relative angle detection means for detecting as a second relative angle;
Target object recognition means for performing image recognition processing on the captured image for detecting the target object determined to be located around the vehicle by the target object determination means based on the second relative angle. The vehicle object image recognition apparatus according to claim 1. Imaging means for imaging the periphery of the vehicle with an imaging device installed in the vehicle;
A relative angle storage means for storing a relative angle of the target object with respect to a reference object as a comparison relative angle in a storage medium in association with the target object;
Target object determination means for determining whether or not the target object is located around the vehicle;
When the target object determining unit determines that the target object is located around the vehicle, the relative angle of the reference object included in the captured image captured by the imaging unit with respect to the imaging device is a first relative angle. First relative angle detecting means for detecting
Angle extracting means for extracting the comparative relative angle stored in association with the target object determined to be located around the vehicle by the target object determining means among the comparative relative angles stored in the storage medium When,
Based on the comparison relative angle and the first relative angle extracted by the angle extraction means, a relative angle of the target object determined to be positioned around the vehicle by the target object determination means is determined with respect to the imaging device. Second relative angle detection means for detecting as a second relative angle;
Target object recognition means for performing image recognition processing on the captured image for detecting the target object determined to be located around the vehicle by the target object determination means based on the second relative angle. A vehicle object image recognition device characterized by the above. An imaging step of imaging the periphery of the vehicle with an imaging device installed in the vehicle;
A reference relative angle detection step of detecting, as a reference relative angle, a relative angle of a reference object included in the captured image captured by the imaging step with respect to the imaging device;
A target relative angle detection step of detecting a relative angle of a target object included in the captured image captured by the imaging step with respect to the imaging device as a target relative angle;
A comparative relative angle calculation step of calculating a relative angle of the target object with respect to the reference object as a comparative relative angle based on the reference relative angle and the target relative angle;
And a relative angle storage step of storing the comparative relative angle in a storage medium in association with the target object. On the computer,
An imaging function for imaging the periphery of the vehicle with an imaging device installed in the vehicle;
A reference relative angle detection function for detecting, as a reference relative angle, a relative angle of a reference object included in a captured image captured by the imaging function with respect to the imaging device;
A target relative angle detection function for detecting, as a target relative angle, a relative angle of a target object included in a captured image captured by the imaging function with respect to the imaging device;
A comparative relative angle calculation function for calculating a relative angle of the target object with respect to the reference object as a comparative relative angle based on the reference relative angle and the target relative angle;
A relative angle storage function for storing the comparison relative angle in a storage medium in association with the target object;
A computer program for executing

Images