JP2010176645A - Image recognition method and image recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To properly recognize a static object in an image captured by a camera mounted on a running vehicle. <P>SOLUTION: An image recognition method includes: a step (S5002) for recognizing the position of a road on image data according to the image data created when a camera images the outside of a vehicle; a step (S5009) for comparing the recognized position of the road with the position of the road on map data obtained by seeing the vehicle from a predetermined point of view, the map data being stored in a navigation device for specifying the position of the vehicle, to calculate an amount of distortion between the image data and the map data; steps (S5011, S5012) for converting, based on the amount of distortion, either the image data or the map data into data as seen from the point of view of the other data; and a static object recognition step (S5014) for recognizing a static object by determining whether or not the static object exists in the same positions in the image data and the map data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image recognition method and an image recognition apparatus for recognizing a stationary object from an image taken from the outside of a vehicle.

  In recent years, with the advancement of digital technology and the spread of automobiles, the technology of navigation devices has rapidly advanced. The navigation device obtains an optimum guidance route to a preset destination based on the map data held in the navigation device. In addition, as the vehicle travels, guidance for turning left or right is displayed on the display at important points such as intersections in the route.

  In addition, the navigation device displays a signboard, road traffic sign, guide sign, and the like that can identify various facilities on a map displayed on the display.

  However, in reality, after the navigation device is shipped, a new signboard or sign is installed on the road, and the map data stored in the memory of the navigation device is not always the latest.

On the other hand, car navigation technology is known in which map data is automatically updated to always keep the map data up-to-date (see, for example, Patent Document 1). In this method, a landmark is detected from an image taken by a camera attached to a traveling vehicle, a position of the detected landmark is estimated, and is detected within a predetermined range including the estimated position. It is determined whether or not there is a landmark on the map corresponding to the landmark. If it is determined that there is no landmark on the map, the detected landmark is added to the map.
Japanese Patent Laid-Open No. 2004-45051

  However, the viewpoint differs between the image photographed by the camera and the map. For this reason, there exists a subject that it cannot judge correctly whether stationary objects, such as a landmark detected from the picture photoed with the camera, exist on a map.

  Along with such a problem, there is a problem that a stationary object such as a detected landmark is not always added to the map as a corresponding position.

  The present invention has been made in view of the above-described problems, and provides an image recognition method capable of accurately recognizing a stationary object from images captured by a camera attached to a traveling vehicle. Objective.

  In addition, a stationary object such as a landmark detected from an image captured by a camera attached to a traveling vehicle is added to an accurate position on map data provided in the navigation device, or a stationary object that does not exist is added. It is another object of the present invention to provide an image recognition method that can be deleted from map data.

  In order to achieve the above object, an image recognition method according to the present invention is an image recognition method for recognizing a stationary object included in an image captured by a camera attached to a vehicle, wherein the camera The road recognition step for recognizing the position of the road on the image data, the position of the road recognized in the road recognition step, and the position of the vehicle are identified from image data generated by photographing the outside. A distortion amount for calculating a distortion amount between the image data and the map data by comparing the position of the road on the map data when the vehicle stored in the navigation device is viewed from a predetermined viewpoint Based on the calculation step and the amount of distortion, one of the image data and the map data is converted into data viewed from the viewpoint of the other data. Determining whether there is a stationary object at a position corresponding to each other based on the image data and the map data, one of which is converted in the step and the converting step, A stationary object recognition step for recognizing the stationary object in the data.

  According to this configuration, the distortion amount of both data is obtained by comparing the road position of the image data with the road position of the map data. The road position itself is less likely to change than a stationary object, and the coordinates of interest are easily extracted at a position where the road intersects. For this reason, it becomes possible to calculate the amount of distortion accurately. Therefore, it is possible to accurately recognize a stationary object by performing image conversion based on this distortion amount and determining whether or not a stationary object exists in both data.

  Preferably, in the stationary object recognition step, the image corresponding to the position of the stationary object on the map data based on the image data and the map data, one of which is converted in the conversion step. A search area determining step for determining a predetermined area including a position on the data as a search area for the stationary object on the image data; and whether the stationary object exists in the search area of the image data And recognizing the stationary object in the image data by determining whether or not.

  By such an image recognition method, it is possible to preferentially search an area where a stationary object can exist from a camera image.

  More preferably, the stationary object recognition step excludes the position of the road recognized in the road recognition step based on the image data and the map data, one of which is converted in the conversion step. A search area determining step for determining an area of data as a search area for the stationary object on the image data; and determining whether or not the stationary object exists in the search area of the image data. And recognizing the stationary object in the image data.

  By such an image recognition method, it is possible to search for a stationary object by giving priority to an area that is excluded from a part of the area (road area) from the camera image.

  More preferably, the stationary object recognition step includes a road contour extraction step for extracting a road contour from the image data after the conversion in the conversion step, and a plurality of the image data temporally continuous, and Extracting the stationary object from the plurality of image data after conversion in the conversion step, and determining whether the stationary object is moving along the contour of the road, Recognizing the stationary object.

  By using the condition that the stationary object exists along the road contour, the stationary object can be accurately recognized. Further, motion compensation can be performed on the detected stationary object.

  More preferably, the stationary object recognition step further includes the stationary object of the map data when it is determined that the stationary object exists only in either one of the image data and the map data. Update data about.

  By such an image recognition method, a stationary object such as a landmark can be added to an accurate position on the map data provided in the navigation device, or a non-existing stationary object can be deleted from the map data. . Therefore, the map data used in the navigation device can be updated to the latest one.

  More preferably, in the stationary object recognition step, a condition that the stationary object is detected from the image data and a stationary object corresponding to the detected stationary object is not detected from the map data is satisfied. If it is determined that the condition is satisfied a specified number of times or more, information on the stationary object is added to the map data.

  By determining whether or not the above condition is satisfied a specified number of times or more, recognition of a stationary object can be prevented, and information regarding the stationary object can be accurately added to the map data.

  More preferably, in the stationary object recognition step, a condition that the stationary object is detected from the map data and a stationary object corresponding to the detected stationary object is not detected from the image data is satisfied. If it is determined that the condition is satisfied a specified number of times or more, information on the stationary object is deleted from the map data.

  By determining whether or not the above condition is satisfied a specified number of times or more, recognition of a stationary object can be prevented, and information regarding the stationary object can be accurately deleted from the map data.

  More preferably, the stationary object recognition step further includes the stationary object of the map data when it is determined that the stationary object exists only in either one of the image data and the map data. A data update request is transmitted to the server that manages the map data.

  When the map data is managed by the server, stationary objects such as landmarks are added to the exact location on the map data provided by the server, or stationary objects that do not exist are deleted from the map data. can do. Therefore, the map data managed by the server can be updated to the latest one.

  More preferably, in the stationary object recognition step, a condition that the stationary object is detected from the image data and a stationary object corresponding to the detected stationary object is not detected from the map data is satisfied. If it is determined that the condition is satisfied a specified number of times or more, a request for adding information on the stationary object to the map data is transmitted to the server.

  By determining whether or not the above condition is satisfied a specified number of times or more, recognition of a stationary object can be prevented, and information regarding the stationary object can be accurately added to the map data managed by the server. .

  More preferably, in the stationary object recognition step, a condition that the stationary object is detected from the map data and a stationary object corresponding to the detected stationary object is not detected from the image data is satisfied. And when it is determined that the condition is satisfied a specified number of times or more, a request to delete the information related to the stationary object from the map data is transmitted to the server.

  By determining whether or not the above condition is satisfied a specified number of times or more, recognition of a stationary object can be prevented, and information regarding the stationary object can be accurately deleted from the map data managed by the server. .

  More preferably, the above-described image recognition method further includes a receiving step of receiving updated map data from the server.

  As a result, the navigation device can present the latest map data to the driver of the vehicle.

  The present invention can be realized not only as an image recognition method including such characteristic steps, but also as an image recognition apparatus using the characteristic steps included in the image recognition method as a processing unit. it can. It can also be realized as a program that causes a computer to execute characteristic steps included in the image recognition method. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM (Compact Disc-Read Only Memory) or a communication network such as the Internet.

  According to the present invention, a stationary object can be accurately recognized from an image taken by a camera attached to a traveling vehicle.

  In addition, a stationary object such as a landmark detected from an image captured by a camera attached to a traveling vehicle is added to an accurate position on map data provided in the navigation device, or a stationary object that does not exist is added. It can be deleted from the map data.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  In the following embodiments, various modifications can be made to the hardware and software implementations. Therefore, first, the configuration of the navigation device common to each embodiment will be described, and then each embodiment will be described. The following embodiments do not limit the patented invention, and all the features described in the embodiments are not necessarily essential constituent elements of the invention.

  FIG. 1 is a functional block diagram showing the configuration of the navigation device.

  This navigation device is attached to a vehicle such as an automobile, searches for and sets information on stationary objects such as landmarks based on map data prepared in advance, and displays map data indicating the corresponding position. A communication control unit 101, a self-contained navigation control unit 102, a GPS (Global Positioning System) control unit 103, a road traffic information receiving unit 104, an audio output unit 105, a navigation control unit 106, and a map data storage Unit 107, update data storage unit 108, imaging unit 109, image processing unit 110, image composition processing unit 111, and image display processing unit 112.

  The communication control unit 101 controls wireless or wired data communication. The communication function may be built in the navigation device or may be provided outside. For example, a mobile communication terminal such as a mobile phone having a communication function and the communication control unit 101 may be connected to perform data communication via the mobile communication terminal. With such a configuration, the user can access the external server via the communication control unit 101.

  The self-contained navigation control unit 102 includes a vehicle speed sensor that detects a moving speed of the host vehicle, an angle sensor that detects a rotation angle of the host vehicle, and the like.

  The GPS control unit 103 receives GPS radio waves transmitted from a plurality of artificial satellites (GPS satellites) arranged in a predetermined orbit at an altitude of about 20,000 km on the earth by a GPS receiver, and information included in the GPS signal Measure the current position and direction of the vehicle on the earth.

  The road traffic information receiving unit 104 sequentially receives current road traffic information outside the host vehicle via an external antenna.

  The navigation control unit 106 serves as a control circuit that controls each processing unit of the navigation device.

  The map data storage unit 107 is a memory that stores various types of data necessary for the operation of the apparatus, and holds various types of data related to stationary objects such as map data and landmarks, facility data, and the like. Necessary map data is read from the navigation control unit 106. The memory in the map data storage unit 107 here may be a CD-ROM, a DVD-ROM, or a hard disk drive (HDD).

  The update data storage unit 108 is a memory that stores update data indicating the difference between the map data stored in the map data storage unit 107 and the map data updated from the map data. The update data is set by the navigation control unit 106.

  The voice output unit 105 includes a speaker, and outputs a voice such as an intersection guide during route guidance, for example.

  The imaging unit 109 is a camera provided with an imaging element such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor installed in front of the host vehicle.

  The image processing unit 110 converts the image data output from the imaging unit 109 into image data of another format. The image processing unit 110 receives map data from the navigation control unit 106 as input, and performs image processing on the map data. Further, the image processing unit 110 receives stationary object information (information including position information of the stationary object and image information of the stationary object) from the navigation control unit 106, performs image processing on the stationary object information, The stationary object information obtained as a result of the image processing is output to the navigation control unit 106.

  The image composition processing unit 111 receives map data including the current position of the host vehicle as an input from the navigation control unit 106 and composes it with a camera image output from the image processing unit 110 (an image captured by the image capturing unit 109). .

  Then, the image display processing unit 112 displays the image data combined by the image combining processing unit 111 on a display or the like of the navigation device.

(Embodiment 1)
A navigation apparatus according to Embodiment 1 of the present invention will be described with reference to the drawings.

  The overall configuration of the navigation device is as described with reference to FIG.

  FIG. 2 is a block diagram illustrating a detailed internal configuration of the image processing unit 110.

  The image processing unit 110 includes a luminance signal color difference signal separation processing unit 202, a luminance signal processing unit 203, a color difference signal processing unit 204, a road recognition processing unit 205, a target coordinate extraction unit 206, a selector 208, and coordinate conversion. A processing unit 209, a selector 210, and a stationary object recognition processing unit 213 are included.

  Hereinafter, each processing unit constituting the image processing unit 110 will be described.

<Luminance Signal Color Difference Signal Separation Processing Unit 202>
The luminance signal color difference signal separation processing unit 202 receives image data of a camera image from the imaging unit 109 as an input. When the luminance signal chrominance signal separation processing unit 202 receives image data consisting of three colors (three primary colors of light) of red (R), green (G), and blue (B) from the imaging unit 109 as input, The image data is converted into YUV color format image data using a general color space conversion formula (the following formulas (1) to (3)).

Y = 0.29891 × R + 0.58661 × G + 1.14848 × B (1)
U = −0.16874 × R−0.33126 × G + 0.50000 × B (2)
V = 0.50,000 × R−0.41869 × G−0.0811 × B (3)

  Also, using the following equations (4) to (6), ITU-R BT. The image data may be converted into image data in the YCbCr color format defined in 601.

Y = 0.257R + 0.504G + 0.098B + 16 (4)
Cb = −0.148R−0.291G + 0.439B + 128 (5)
Cr = 0.439R−0.368G−0.071B + 128 (6)

  Here, Y indicates a luminance signal (brightness), Cb (U) indicates a blue differential signal, and Cr (V) indicates a red differential signal. Further, when image data including three colors of cyan (C), magenta (M), and yellow (Y) (the three primary colors) is received as an input from the imaging unit 109, the luminance signal color difference signal separation processing unit 202 is received. Is converted into RGB format image data using a general conversion formula (following formulas (7) to (9)), and then converted into formula (1) to (3) or formulas (4) to (6). Is used to convert the image data into YUV format or YCbCr format image data. The luminance signal color difference signal separation processing unit 202 outputs the converted image data.

R = 1.0-C (7)
G = 1.0-M (8)
B = 1.0−Y (9)

  In addition, when YUV format image data is input from the imaging unit 109, the luminance signal color difference signal separation processing unit 202 does not particularly convert image data, but only performs YUV signal separation.

<Luminance signal processing unit 203>
The luminance signal processing unit 203 receives the image data of the luminance signal (Y) output from the luminance signal color difference signal separation processing unit 202 as input, performs signal processing according to the luminance, and outputs it. Furthermore, the luminance signal processing unit 203 performs a contour pixel determination process. For example, it may be determined whether the pixel of interest is a contour pixel by simply using the eight pixels around the pixel of interest. That is, as shown in FIG. 3, a 3 × 3 target pixel D35 and its surrounding eight pixels D31 to D34 and D36 to D39 are considered. The luminance of the pixel of interest D35 is compared with the luminance of each of the surrounding pixels D31 to D34 and D36 to D39, and if the sum of absolute values of luminance differences is greater than a preset value, the pixel of interest D35 is determined as a contour pixel. . By scanning a 3 × 3 window as shown in FIG. 3 on the image data output from the luminance signal color difference signal separation processing unit 202, a contour pixel determination process is performed for each pixel of the image data. For example, when image data of a camera image as shown in FIG. 4 is input, if contour pixels are extracted based on luminance information, the image data is converted into image data as shown in FIG. In the image data shown in FIG. 5, black pixels are contour pixels.

<Color difference signal processing unit 204>
The color difference signal processing unit 204 receives the color difference signal output from the luminance signal color difference signal separation processing unit 202 as input, performs signal processing according to the color difference, and outputs the signal. Further, the color difference signal processing unit 204 performs a determination process for pixels having a color difference equivalent to that of a predetermined specific area. For example, the color difference signal processing unit 204 recognizes the color difference of a road on which the host vehicle is traveling. That is, when the imaging unit 109 is installed at the front center of the host vehicle, the host vehicle is always present on the road, so the lower center of the camera image is the road. Therefore, the specific area A601 is set at the lower center with respect to the camera image shown in FIG. The color difference signal processing unit 204 calculates a color difference representing the specific area A601. For example, a color difference that represents an average value of color differences may be used. The color difference signal processing unit 204 identifies a pixel having a color difference equivalent to the calculated color difference as a pixel having a color difference equivalent to that of the specific area A601 from the image data of the camera image. As a result, image data consisting only of road color difference signals as shown in FIG. 7 is obtained. An area composed of road color difference signals is referred to as a road color difference signal area A701.

<Road recognition processing unit 205>
The road recognition processing unit 205 receives the contour component image data from the luminance signal processing unit 203 and the color difference component image data from the color difference signal processing unit 204 as inputs. For example, the road recognition processing unit 205 receives the image data of the contour component shown in FIG. 5 and the image data of the color difference signal shown in FIG. 7 as inputs. In this case, the road recognition processing unit 205 outputs an image signal of a contour component adjacent to the road color difference signal area A701, or an image signal of a contour component that is not adjacent to the road color difference signal area A701 but is adjacent to the road color difference signal area A701 (road color difference signal). (Image signal of a contour component within a predetermined distance from the area A701) is recognized, and only the corresponding contour pixel is extracted. Then, by synthesizing the contour pixel and the road color difference signal area A701 and recognizing the synthesized result as road image data, road image data as shown in FIG. 8 is output. Thereby, a road can be recognized from an image of a camera that captures an external image from the host vehicle.

<Attention coordinate extraction unit 206>
The attention coordinate extraction unit 206 receives the road image data (FIG. 8) recognized by the road recognition processing unit 205 and the map data (FIG. 9) from the navigation control unit 106 as inputs. First, the attention coordinate extraction unit 206 calculates a portion where the road contour is refracted in the road image data, and detects the detected portion as an attention point as an intersection. For example, as shown in FIG. 10, the coordinates P1001 to P1004 are detected as points of interest.

  Next, a method for calculating the refracted portion of the road contour on the road image data will be described. Referring to FIG. 10, road image data is divided into left and right based on a base line L1005. A road contour vector (road contour vector) V1006 is obtained from the road image data on the left side. This is based on the perspective method and limited to the direction vector in the first quadrant as the vector V1102 in FIG. Also, a road contour vector V1007 is obtained from the right road image data. This is based on the perspective method and limited to the direction vector in the second quadrant as the vector V1101 in FIG. The vector can be obtained by calculating a linear approximation straight line with respect to the pixels of the road outline. The refracting contour coordinates of the road contour that follow the vector V1006 and the vector V1007 (continuous to the vector V1006 and the vector V1007, respectively) are calculated as the attention points.

  Similarly, the attention coordinate extraction unit 206 calculates a portion where the road outline is refracted in the map data shown in FIG. 12, and detects the corresponding coordinates P1201 to P1204 as attention points as intersections. Next, a method for calculating the refracted portion of the road contour on the map data will be described. Referring to FIG. 12, map data is divided into left and right based on a base line L1205. A road contour vector V1206 is obtained from the map data on the left side. This is limited to the direction vector of the first quadrant as the vector V1102 in FIG. Also, a road contour vector V1207 is obtained from the map data on the right side. This is limited to the direction vector in the second quadrant as the vector V1101 in FIG. The contour coordinates (coordinates P1201 to P1204) that refract the road contour that follow the vectors V1206 and V1207 (continue to the vectors V1206 and V1207, respectively) are calculated as the coordinates of the attention point. Then, the attention coordinate extraction unit 206 outputs the coordinates of the attention point calculated for each of the camera image and the map data.

  As described above, the two-dimensional map data is taken as an example, but the attention point can be calculated by the same processing with the three-dimensional map data.

  However, when another vehicle, an obstacle, or the like exists at the point of interest to be calculated, the point-of-interest extraction unit 206 cannot detect the point of interest as an intersection in the road image data. For example, as shown in FIG. 13, in the road image data, only two coordinates P1002 and P1003 can be detected among the coordinates of the four points of interest. In such a case, the intersection coordinates may be calculated by the following method.

  That is, as shown in FIG. 14, based on road contour vectors V1006, V1007, V1407, and V1408 and intersection coordinates P1002 and P1003, intersection coordinates P1403 are calculated using direction vectors V1409 and V1410. Here, the direction vector V1409 is in the same direction as the vector V1006, and the start point is a vector having the coordinates P1003. The direction vector V1410 is an inverse vector with respect to the road contour vector V1407, and the start point is a vector of coordinates P1002. The coordinates of the intersection of the vector V1409 and the vector V1410 are obtained as the intersection coordinate P1403.

  Similarly, based on the road contour vectors V1006, V1007, V1407, V1408 and the intersection coordinates P1002, P1003, the intersection coordinates P1404 are calculated using the direction vectors V1411, V1412. Here, the direction vector V1411 is an inverse vector with respect to the road contour vector V1007, and the start point is a vector of coordinates P1002. In the direction vector V1412, the start point is the coordinate P1003 and the direction is the same vector as the vector V1408. The coordinates of the intersection of the vector V1411 and the vector V1412 are obtained as the intersection coordinate P1404. And the coordinate of the attention point calculated about each of a camera image and map data is output.

  Further, the attention coordinate extraction unit 206 may calculate the attention point as an intersection from the road contour vector. That is, as shown in FIG. 15, the attention coordinate extraction unit 206 calculates intersections of road contour vectors in the road image data, and detects intersecting coordinates P1505 to coordinates P1508 as attention points as intersections. In the vector intersection calculation method, road image data is first divided into left and right based on a base line L1005, and a road contour vector V1501 is obtained from the left road image data. This is based on the perspective method and is limited to the direction vector in the first quadrant as shown in V1102 in FIG. Further, a road contour vector V1502 is obtained from the road image data on the right side. This is based on the perspective method and is limited to the direction vector in the second quadrant as in V1101 in FIG. Separately from these vectors V1501 and V1502, road contour vectors V1503 and V1504 are obtained. Then, coordinates intersecting with each of the road contour vectors V1501 to V1504 can be detected as attention points as intersections. Then, the attention coordinate extraction unit 206 outputs the coordinates of the attention point and the road contour vector calculated for each of the camera image and the map data.

  With this configuration and method, in a camera image, a portion where the road contour is refracted can be detected as a point of interest as an intersection. Further, in the map data stored in advance in the navigation device, a portion where the road contour is refracted can be detected as a point of interest as an intersection. Furthermore, when an attention point as an intersection is not detected by another vehicle, an obstacle, or the like, the attention point can be changed around the detected attention point of the camera image.

<Coordinate conversion processing unit 209>
The coordinate conversion processing unit 209 outputs the coordinates of the attention points of the camera image and the map data output from the attention coordinate extraction unit 206, the camera images from the luminance signal processing unit 203 and the color difference signal processing unit 204, and navigation control. The map data from the unit 106 is received as input. Switching between the camera image and the map data is performed by the selector 208.

  First, the coordinate conversion processing unit 209 receives, as inputs, coordinates P1601 to P1604 as shown in FIG. In addition, the coordinate conversion processing unit 209 receives coordinates P1605 to P1608 as input as the coordinates of the target point of the road image captured by the camera. Then, the coordinate conversion processing unit 209 calculates the distortion amount so that the coordinate P1601 moves to the coordinate P1605, the coordinate P1602 moves to the coordinate P1606, the coordinate P1603 moves to the coordinate P1607, and the coordinate P1604 moves to the coordinate P1608. Then, the coordinate conversion processing unit 209 performs image deformation according to the distortion amount on the input map data. For this image deformation, a bilinear method (linear interpolation) or a bicubic method often used for enlarging / reducing an image may be used. Also, a technique for deforming an arbitrary quadrangle may be used. For example, in FIG. 16, a rectangle is represented by connecting coordinates P1601 to P1604 and coordinates P1605 to P1608 with dotted lines, and a rectangle composed of coordinates P1601 to P1604 is represented as a rectangle composed of coordinates P1605 to P1608. Transforms into This figure shows the image deformation by a square, and the square is not necessary to calculate the distortion amount.

  With this configuration and method, it is possible to calculate the distortion amount so that the attention point coordinates of the map data match the attention point coordinates of the camera image, and to transform the map data by coordinate conversion according to the distortion amount. For example, when the map data as shown in FIG. 9 is deformed based on the amount of distortion obtained according to FIG. 16, image data as shown in FIG. 17 is obtained.

  In addition, the coordinate conversion processing unit 209 may perform image deformation by obtaining a distortion amount from a road contour vector. That is, the coordinate conversion processing unit 209 outputs the road contour vectors of the camera image and map data output from the target coordinate extraction unit 206, the camera images from the luminance signal processing unit 203 and the color difference signal processing unit 204, and The map data from the navigation control unit 106 is received as an input. Switching between the camera image and the map data is performed by the selector 208.

  First, the coordinate conversion processing unit 209 receives, as inputs, vectors V1801 to V1804 as shown in FIG. 18 as road contour vectors of map data. Also, the coordinate conversion processing unit 209 receives as input a vector V1805 to a vector V1808 as a vector of the road contour of the road image taken by the camera. The coordinate conversion processing unit 209 calculates the distortion amount so that the vector V1801 moves to the vector V1805, the vector V1802 moves to the vector V1806, the vector V1803 moves to the vector V1807, and the vector V1804 moves to the vector V1808. Then, the coordinate conversion processing unit 209 performs image deformation according to the distortion amount on the input map data. For this image deformation, a bilinear method (linear interpolation) or a bicubic method often used for enlarging / reducing an image may be used. Also, a technique for deforming an arbitrary quadrangle may be used.

  With this configuration and method, it is possible to calculate the distortion amount so that the attention point coordinates of the map data coincide with the attention point coordinates of the camera image, and to deform the map data by coordinate conversion according to the distortion amount. For example, when the image deformation is performed on the map data as shown in FIG. 9, the distortion amount in FIG. 16 is as shown in FIG.

  Similarly, the coordinate transformation processing unit 209 can also transform the camera image by coordinate transformation corresponding to the distortion amount. That is, the coordinate conversion processing unit 209 performs image deformation corresponding to the amount of distortion on the camera image input via the selector 208. For example, the image deformation is performed based on the distortion amount calculated in FIG. 16 in the direction opposite to the four vectors illustrated in FIG. Thus, image data as shown in FIG. 19 is obtained by performing image deformation on the camera image as shown in FIG.

  Further, the coordinate conversion processing unit 209 receives, from the navigation control unit 106, information on a stationary object M2001 such as a landmark as shown in FIG. In this example, information on a stationary object representing a company mark “6” is received as an input. The input information is the mark information of the stationary object held in the map data storage unit 107 and the coordinate data indicating the coordinates on the map. As shown in FIG. 21, the coordinate conversion processing unit 209 converts the coordinates of the stationary object M2101 on the map into coordinates A2201 as shown in FIG. The coordinate conversion is performed in the same manner as the image data conversion described above. The coordinate conversion processing unit 209 outputs information on the coordinates A2201 to the stationary object recognition processing unit 213.

<Still Object Recognition Processing Unit 213>
The stationary object recognition processing unit 213 receives the coordinates of the stationary object (coordinate A2201 in FIG. 22) obtained from the map data output from the coordinate conversion processing unit 209 as input. The stationary object recognition processing unit 213 receives the contour component image data from the luminance signal processing unit 203 and the color difference component image data from the color difference signal processing unit 204 as inputs. Then, the stationary object recognition processing unit 213 receives information about the stationary object M2001 such as a landmark from the navigation control unit 106 as an input. As shown in FIG. 23, the stationary object recognition processing unit 213 includes a stationary object in a search area A2301 within a predetermined distance from the coordinates A2201 of the image data received from the luminance signal processing unit 203 and the color difference signal processing unit 204. The image information corresponding to the object M2001 is searched. Any method can be employed regardless of a method for searching, detecting, and recognizing a stationary object such as a landmark, a signboard, a traffic sign, and a road guide sign. For example, if there is a stationary object at a position corresponding to each other in the image data and the map data, it can be recognized that the stationary object exists. If a stationary object exists only in the image data, it can be determined that a new stationary object has been created. Further, if there is a stationary object only in the map data, it can be determined that the stationary object has disappeared.

  Note that the stationary object recognition processing unit 213 may search for image information corresponding to the stationary object M2001 in an area A2402 excluding the road image area A2401, as illustrated in FIG.

  The stationary object recognition processing unit 213 may search for a stationary object using two camera images that are temporally continuous. That is, the stationary object recognition processing unit 213 receives the coordinates of the attention point calculated for each of the camera image and the map data from the coordinate conversion processing unit 209 as input. The stationary object recognition processing unit 213 receives a road contour vector from the coordinate conversion processing unit 209.

  As shown in FIG. 25, it is assumed that road contour vectors V2501 and V2502 are obtained from a camera image at a certain time. These road contour vectors V2501 and V2502 are the same as vector V1501 and vector V1502, respectively. The stationary object recognition processing unit 213 obtains an intersection M2503 between the road contour vectors V2501 and V2502. This intersection M2503 is the center coordinate in perspective.

  Next, the stationary object recognition processing unit 213 holds the coordinate information of the stationary objects M2504 and M2505 detected by the stationary object recognition processing unit 213 with respect to two temporally continuous image data. That is, if the automobile is moving forward, the stationary object at the current time is the stationary object M2505, and the stationary object at the past time is the stationary object M2504.

  Then, the stationary object recognition processing unit 213 calculates a vector V2506 that connects the coordinates of the stationary objects M2504 and M2505. The stationary object recognition processing unit 213 calculates a distance between the vector V2506 and a vector (vector V2501 or V2502) based on the intersection M2503. The stationary object recognition processing unit 213 determines whether or not the calculated distance is within an error specified in advance, and determines that the stationary objects M2504 and M2505 are surely stationary objects if within the error. That is, it is determined that the stationary object moving along the contour of the road is a desired stationary object. In this example, the coordinate information of the stationary object obtained from the two pieces of image data is used to determine whether or not the object is a stationary object. However, the number of image data is not limited to this.

  The selector 210 receives the output from the navigation control unit 106 and the outputs from the luminance signal processing unit 203 and the color difference signal processing unit 204 as inputs, and outputs one of them.

  Next, an image recognition method using the navigation device described above will be described using the flowchart shown in FIG.

  The imaging unit 109 captures an external image (camera image) from the host vehicle (step S5001).

  The road in the camera image is recognized by the luminance signal color difference signal separation processing unit 202, the luminance signal processing unit 203, the color difference signal processing unit 204, and the road recognition processing unit 205 (step S5002).

  The attention coordinate extraction unit 206 acquires map data (step S5003).

  If there is an instruction to calculate the direction vector (YES in S5004), the attention coordinate extraction unit 206 calculates a direction vector of the road contour (step S5005).

  The attention coordinate extraction unit 206 detects an attention point as an intersection (step S5006).

  If the attention point cannot be detected (NO in step S5007), the attention coordinate is changed (step S5008). That is, it tries to extract other attention points of the intersection.

  If an attention point has been detected (YES in step S5007), the coordinate conversion processing unit 209 calculates a distortion amount (S5009).

  Next, the coordinate conversion processing unit 209 determines whether the object of image deformation is a camera image or map data (step S5010). It is assumed that an image transformation target is designated in advance.

  If the object of image deformation is map data (YES in step S5010), the coordinate conversion processing unit 209 deforms the map data and generates image data as shown in FIG. 17 (step S5011).

  When the image deformation target is a camera image (NO in step S5010), the coordinate conversion processing unit 209 deforms the camera image and generates an image as shown in FIG. 19 (step S5012).

  The coordinate conversion processing unit 209 determines whether or not it is designated to determine a stationary object search region based on the map data (S5013a). This designation is assumed to have been made in advance.

  When it is specified that the search area for the stationary object is determined based on the map data (YES in S5013a), as described with reference to FIGS. 21 and 22, the coordinate conversion processing unit 209 performs the map conversion process. The coordinates of the stationary object M2101 obtained from the data are converted into coordinates A2201 as shown in FIG. 22 (S5013). That is, the coordinates are converted into coordinates A2201 on the camera image.

  Next, as described with reference to FIG. 23, the stationary object recognition processing unit 213 determines a search area A2301 in the camera image (S5014). After that, the stationary object recognition processing unit 213 recognizes the stationary object.

  The stationary object recognition processing unit 213 determines whether it is designated to determine a search area for a stationary object based on the road image area (S5015a). This designation is assumed to have been made in advance.

  If it is specified that the search area for the stationary object is determined based on the road image area (YES in S5015a), the stationary object recognition processing unit 213 uses the camera image as shown in FIG. A region A2402 excluding the road image region A2401 is determined as a stationary object search region (step S5015). After that, the stationary object recognition processing unit 213 recognizes the stationary object.

  The stationary object recognition processing unit 213 determines whether a stationary object has been detected in the processes so far (S5016). When a stationary object is detected (YES in step S5016), the stationary object recognition processing unit 213 holds information on the detected stationary object (S5017). If the host vehicle has not reached the destination (NO in step S5018), the processing from S5001 is repeatedly executed.

  In step S5017, information on the stationary object is accumulated from moment to moment. The stationary object recognition processing unit 213 determines whether or not the accumulated stationary object is moving along the contour of the road by the method described with reference to FIG. The stationary object recognition processing unit 213 determines that the stationary object moving along the road outline is a desired stationary object.

  As described above, according to the present embodiment, it is possible to read stationary object information from the navigation device and recognize the stationary object from the deformed map data and the camera image. Further, the search area for the stationary object on the camera image can be determined from the position of the stationary object on the map and the amount of distortion.

  In addition, an area excluding the road area can be determined as a stationary object search area from the camera image.

  Furthermore, it is possible to detect stationary objects detected along the road contour at a plurality of times. For this reason, motion compensation can be performed from the reference point searched from the road contour vector.

(Embodiment 2)
A navigation device according to Embodiment 2 of the present invention will be described with reference to the drawings. In the first embodiment, the stationary object is recognized. In the present embodiment, processing for adding a stationary object to map data or deleting a stationary object from map data is performed based on the recognition result of the stationary object.

  The configuration of the navigation device is the same as that described with reference to FIGS. Therefore, detailed description thereof will not be repeated here. Hereinafter, a description will be given focusing on differences from the first embodiment. In the present embodiment, as in the first embodiment, a road is recognized from a camera image, a point of interest as an intersection is detected, and the image is deformed based on the amount of distortion.

  In addition, the following processing is executed. That is, the coordinate conversion processing unit 209 receives the information on the stationary object detected by the stationary object recognition processing unit 213 and the image data from the luminance signal processing unit 203 and the color difference signal processing unit 204 as inputs.

  As shown in FIG. 27, the coordinate conversion processing unit 209 performs coordinate conversion of the stationary object M2704 detected on the camera image into image data viewed from the same starting point as the map data as shown in FIG. Image data as shown in FIG. 28 is obtained. At this time, the stationary object M2704 is converted into a stationary object M2801. At this time, if there is no stationary object on the desired coordinates of the map data as shown in FIG. 9 (on the coordinates of the stationary object M2801), the coordinate conversion processing unit 209 displays the map as shown in FIG. In order to newly add a stationary object M2901 to the coordinates of the stationary object M2801 in the data, information on the stationary object M2901 is output to the navigation control unit 106. The navigation control unit 106 stores information on the stationary object in the update data storage unit 108 as update data.

  In addition, when information on a stationary object is input from the navigation control unit 106 to the stationary object recognition processing unit 213 assuming that there is a stationary object, the stationary object is not detected. The object recognition processing unit 213 outputs an instruction to delete the stationary object to the navigation control unit 106. The navigation control unit 106 stores information indicating that the stationary object has been deleted in the update data storage unit 108 as update data.

  Next, an image recognition method using the navigation device described above will be described using the flowchart shown in FIG.

  The imaging unit 109 captures an external image (camera image) from the host vehicle (step S5101).

  The stationary object recognition processing unit 213 detects a stationary object from the camera image (step S5102).

  If a stationary object is detected (YES in step S5103), the coordinate conversion processing unit 209 converts the coordinates of the stationary object into coordinates on map data (step S5104). The stationary object recognition processing unit 213 determines whether or not there is a stationary object on the converted coordinates on the map data, and whether or not the determination that there is a stationary object has been made a specified number of times or more. It is determined (whether or not it is determined that there is a stationary object in a specified number of camera images continuous in time) (step S5112). That is, in order to prevent erroneous detection of a stationary object, a specified number of times is provided in step S5112, and if the determination that there is a stationary object is made more than the specified number of times, it is considered that the detected stationary object actually exists. .

  If it is determined that there is a stationary object more than the specified number of times (YES in step S5112), the coordinate conversion processing unit 209 sends information indicating the position of the stationary object on the map data to the navigation control unit 106. Output. The navigation control unit 106 stores information indicating the position of the stationary object to be added as update data in the update data storage unit 108 (step S5105). As a result, the stationary object is added to the map data.

  If no stationary object is detected (NO in step S5103), the coordinate conversion processing unit 209 changes the search range of the stationary object in order to prevent detection of the stationary object (step S5106). If the search range is within the range of the target specified in advance (YES in step S5107), the coordinate conversion processing unit 209 performs redetection of the stationary target object within the search range (S5102).

  If there is no more search range for the stationary object (NO in step S5107), the stationary object recognition processing unit 213 considers that there is no stationary object (step S5108). The stationary object recognition processing unit 213 determines whether or not the determination that the stationary object does not exist has been made a specified number of times (whether or not it has been determined that the stationary object does not exist in the specified number of camera images that are temporally continuous). Is determined (step S5109). In other words, in order to prevent erroneous detection of a stationary object, the designated number of times is set in step S5109, and if it is determined that there is no stationary object, the detected stationary object does not exist. to decide.

  If the determination that the stationary object does not exist is made the designated number of times (YES in step S5109), the coordinate conversion processing unit 209 outputs data that instructs the navigation control unit 106 to delete the stationary object. The navigation control unit 106 deletes the stationary object from the map data by storing the data instructing the deletion of the stationary object as update data in the update data storage unit 108 (step S5110).

  The navigation control unit 106 updates the map information displayed on the image display processing unit 112 based on the map data stored in the map data storage unit 107 and the update data stored in the update data storage unit 108 (S5111). The processing of S5101 to S5112 is repeatedly executed until the destination is reached (step S5113).

  As described above, according to the present embodiment, the position of the stationary object detected from the camera image is converted into the position on the map data, and the position information of the stationary object is displayed at the corresponding position on the map data. If a stationary object is detected in the camera image and the number of detections exceeds the specified number, information on the stationary object can be added to the map data.

  In addition, even if there is position information of a stationary object on the map data, if the stationary object is not detected at the corresponding position in the camera image and the number of times that it was not detected exceeds the specified number of times, Information on stationary objects can be deleted from the data.

(Embodiment 3)
A navigation device according to Embodiment 3 of the present invention will be described with reference to the drawings. In the second embodiment, update data is stored in the update data storage unit 108 in order to add a stationary object to map data or delete a stationary object from map data. In the present embodiment, the update data is transmitted to the server, so that the stationary object can be added or deleted by the server that manages the map data.

  In the following description, differences from the first and second embodiments will be mainly described.

  The configuration of the navigation device according to the present embodiment is the same as that described with reference to FIGS. Therefore, detailed description thereof will not be repeated here.

  In Embodiment 2, when a stationary object is added to map data or when a stationary object is deleted from map data, the coordinate conversion processing unit 209 or the stationary object recognition processing unit 213 performs navigation control on the update data. To the part 106. In the present embodiment, the communication control unit 101 transmits the update data notified to the navigation control unit 106 to a base station server or the like. The base station server manages map data. For this reason, the base station server can update the map data based on the update data.

  Next, an image recognition method using the navigation device described above will be described using the flowchart shown in FIG.

  Here, as in the second embodiment, when a new stationary object is detected, a request for adding a stationary object (update data) is transmitted to the navigation control unit 106, and the stationary object is deleted. When it is detected that a deletion has been detected, a stationary object deletion request (update data) is transmitted to the navigation control unit 106.

  When the navigation control unit 106 receives a request for adding a stationary object (YES in step S5201), the navigation control unit 106 determines whether or not an additional request for the same stationary object has been received a specified number of times or more in order to prevent erroneous detection. (Step S5202). If it is determined that the addition request has been received more than the specified number of times (YES in step S5202), the navigation control unit 106 transmits a request for adding a stationary object to the base station server via the communication control unit 101 (S5203). . The base station server that has received the addition request updates the map data by adding a stationary object to the map data managed by the base station server based on the addition request. The updated map data is transmitted from the base station server to the navigation device. The navigation control unit 106 receives the updated map data via the communication control unit 101 and stores it in the map data storage unit 107.

  On the other hand, when the navigation control unit 106 receives a request for deleting a stationary object (YES in step S5204), in order to prevent erroneous detection, the navigation control unit 106 determines whether or not a deletion request for the same stationary object has been received a specified number of times or more. Judgment is made (step S5205). If it is determined that the deletion request has been received more than the specified number of times (YES in step S5205), the navigation control unit 106 transmits a deletion request for the stationary object to the base station server via the communication control unit 101 (S5206). . The base station server that has received the deletion request updates the map data by deleting the stationary object from the map data managed by the base station server based on the deletion request.

  As described above, according to the present embodiment, the position of the stationary object detected from the camera image is converted into the position on the map data, and the position information of the stationary object is displayed at the corresponding position on the map data. If a stationary object is detected in the camera image and the number of detections exceeds the specified number, a request for adding the stationary object can be transmitted to the base station server. For this reason, the base station server can add the information of the stationary object to the map data managed by itself. The updated map data is transmitted from the base station server to the navigation device. The navigation control unit 106 receives the updated map data via the communication control unit 101 and stores it in the map data storage unit 107.

  In addition, even if there is position information of a stationary object on the map data, if the stationary object is not detected at the corresponding position in the camera image and the number of times that it was not detected exceeds the specified number of times, A deletion request for deleting the information of the stationary object from the data can be transmitted to the base station server. For this reason, the base station server can delete the stationary object information from the map data managed by itself.

  Although the navigation apparatus according to the embodiments of the present invention has been described above, the present invention is not limited to these embodiments.

  For example, in the above-described embodiment, the stationary object is exemplified using the mark of the specific facility. Of course, other stationary objects such as marks, traffic signs, road signs, and traffic lights are used. There may be.

  In addition, although an intersection such as a crossroad is used as an example, it goes without saying that other intersections such as a T-shaped road, a three-way road, and a plurality of branches may be used. The road type is not limited to the intersection between the priority road and the non-priority road, and may be an intersection where a traffic light is installed or an intersection of a road having a plurality of lanes.

  Furthermore, in the above-described embodiment, the navigation apparatus has been described as using two-dimensional map data. However, the present invention can also be realized using three-dimensional map data such as a bird's-eye view.

  Furthermore, in the above-described embodiment, the communication control unit transmits / receives data to / from the base station server, but without using a dedicated wireless communication device, it can be connected to the Internet such as a general mobile phone or a portable terminal device. Data can also be transmitted and received via a connectable device.

  In the navigation device described with reference to FIG. 1, the imaging unit 109, the image processing unit 110, and the navigation control unit 106 may be configured by a single chip LSI (Large Scale Integration). . The same applies to other combinations in other blocks.

  Further, each of the above devices may be specifically configured as a computer system including a microprocessor, ROM, RAM, hard disk drive, display unit, keyboard, mouse, and the like. A computer program is stored in the RAM or hard disk drive. Each device achieves its functions by the microprocessor operating according to the computer program. Here, the computer program is configured by combining a plurality of instruction codes indicating instructions for the computer in order to achieve a predetermined function.

  Further, some or all of the constituent elements constituting each of the above-described devices may be configured by one system LSI. The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a computer system including a microprocessor, ROM, RAM, and the like. . A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating according to the computer program.

  Furthermore, some or all of the constituent elements constituting each of the above-described devices may be configured from an IC card that can be attached to and detached from each device or a single module. The IC card or module is a computer system that includes a microprocessor, ROM, RAM, and the like. The IC card or the module may include the super multifunctional LSI described above. The IC card or the module achieves its function by the microprocessor operating according to the computer program. This IC card or this module may have tamper resistance.

  Further, the present invention may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the computer program.

  Furthermore, the present invention provides a computer-readable recording medium such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray Disc). (Registered trademark)), or recorded in a semiconductor memory or the like. Further, the digital signal may be recorded on these recording media.

  In the present invention, the computer program or the digital signal may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.

  The present invention may be a computer system including a microprocessor and a memory, wherein the memory stores the computer program, and the microprocessor operates according to the computer program.

  In addition, the program or the digital signal is recorded on the recording medium and transferred, or the program or the digital signal is transferred via the network or the like, and is executed by another independent computer system. It is also good.

  Furthermore, the above embodiment and the above modification examples may be combined.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  The image recognition method and the image recognition apparatus according to the present invention can be used in a computer apparatus or the like having a navigation function. Further, in addition to the navigation function, an audio function or a video function may be included. Further, it can be used not only for vehicles having a navigation function but also for all moving objects.

It is a block diagram which shows the structure of a navigation apparatus. It is a block diagram which shows the internal structure of an image process part. It is a figure which shows the structure of the pixel used when judging a contour pixel. It is a figure which shows an example of the image image | photographed with the camera. It is a figure which shows an example of the camera image which extracted the outline component. It is a figure which shows an example of the camera image which showed the specific area | region. It is a figure which shows an example of the image which consists only of a color difference signal of a road. It is a figure which shows an example of the recognized road image. It is a figure which shows an example of map data. It is a figure which shows an example of the location where the road outline in the camera image is refracting. It is a figure which shows an example of a road outline vector. It is a figure which shows an example of the location where the road outline is refracting in map data. It is a figure which shows an example of the location where the road outline in the camera image is refracting. It is a figure which shows an example of the road outline vector in a camera image. It is a figure which shows an example of the road outline vector in a camera image. It is a figure for demonstrating a coordinate transformation process. It is a figure which shows an example of the map data after image deformation. It is a figure which shows an example of a road outline vector. It is a figure which shows an example of the camera image after an image deformation | transformation. It is a figure which shows an example of a stationary object and map data. It is a figure which shows an example of a stationary object coordinate and map data. It is a figure which shows an example of the map data containing a stationary target object coordinate after image deformation. It is a figure which shows an example of the search range of the stationary target object in a camera image. It is a figure which shows an example of the search range of the stationary target object in a camera image. It is a figure which shows an example of the motion vector of a stationary target object. It is a flowchart of the image deformation | transformation method which concerns on Embodiment 1 of this invention. It is a figure which shows an example of a stationary target object and a camera image. It is a figure which shows an example of the camera image containing a stationary target object after image deformation. It is a figure which shows an example of the map data to which the stationary target object was provided. It is a flowchart of the image display method which concerns on Embodiment 2 of this invention. It is a flowchart of the image display method which concerns on Embodiment 3 of this invention.

DESCRIPTION OF SYMBOLS 101 Communication control part 102 Autonomous navigation control part 103 GPS control part 104 Road traffic information receiving part 105 Audio | voice output part 106 Navigation control part 107 Map data memory | storage part 108 Update data memory | storage part 109 Imaging part 110 Image processing part 111 Image composition process part 112 Image display processing unit 202 Luminance signal color difference signal separation processing unit 203 Luminance signal processing unit 204 Color difference signal processing unit 205 Road recognition processing unit 206 Attention coordinate extraction unit 208, 210 Selector 209 Coordinate conversion processing unit 213 Stationary object recognition processing unit

Claims (13)

An image recognition method for recognizing a stationary object included in an image taken by a camera attached to a vehicle,
A road recognition step for recognizing a position of a road on the image data from image data generated by the camera photographing the outside of the vehicle;
Comparing the position of the road recognized in the road recognition step with the position of the road on map data when the vehicle stored in a navigation device for specifying the position of the vehicle is viewed from a predetermined viewpoint. A distortion amount calculating step of calculating a distortion amount between the image data and the map data;
A conversion step of converting either one of the image data and the map data into data viewed from the viewpoint of the other data based on the amount of distortion;
Based on the image data and the map data, one of which has been converted in the conversion step, by determining whether or not a stationary object exists at a position corresponding to each other in the data, A stationary object recognition step of recognizing the stationary object. The stationary object recognition step includes:
A predetermined area including a position on the image data corresponding to a position of the stationary object on the map data, based on the image data and the map data, one of which is converted in the conversion step, A search area determination step for determining a search area for the stationary object on the image data;
The image recognition method according to claim 1, further comprising a recognition step of recognizing the stationary object in the image data by determining whether or not the stationary object exists in the search area of the image data. . The stationary object recognition step includes:
Based on the image data and the map data, one of which is converted in the conversion step, the area of the image data excluding the road position recognized in the road recognition step is the image data on the image data. A search area determination step for determining a search area for a stationary object; and
The image recognition method according to claim 1, further comprising a recognition step of recognizing the stationary object in the image data by determining whether or not the stationary object exists in the search area of the image data. . The stationary object recognition step includes:
A road contour extracting step for extracting a road contour from the image data after the conversion in the converting step;
The stationary object is extracted from a plurality of the image data that are temporally continuous and are converted in the conversion step, and the stationary object moves along the contour of the road. The image recognition method according to claim 1, further comprising: a recognition step of recognizing the stationary object in the image data by determining whether or not the image is present. The stationary object recognition step further updates data related to the stationary object in the map data when it is determined that the stationary object exists in only one of the image data and the map data. The image recognition method according to claim 1. In the stationary object recognition step, whether or not a condition that the stationary object is detected from the image data and a stationary object corresponding to the detected stationary object is not detected from the map data is satisfied. The image recognition method according to claim 5, wherein when the condition is determined to be satisfied more than a specified number of times, information on the stationary object is added to the map data. In the stationary object recognition step, whether or not a condition that the stationary object is detected from the map data and a stationary object corresponding to the detected stationary object is not detected from the image data is satisfied. The image recognition method according to claim 5, wherein when the condition is determined to be satisfied a specified number of times or more, information on the stationary object is deleted from the map data. In the stationary object recognition step, when it is determined that the stationary object exists in only one of the image data and the map data, the map data is updated with respect to the stationary object. The image recognition method according to claim 1, wherein the request is transmitted to a server that manages the map data. In the stationary object recognition step, whether or not a condition that the stationary object is detected from the image data and a stationary object corresponding to the detected stationary object is not detected from the map data is satisfied. The image recognition method according to claim 8, wherein a request for adding information related to the stationary object to the map data is transmitted to the server when it is determined that the condition is satisfied a specified number of times or more. In the stationary object recognition step, whether or not a condition that the stationary object is detected from the map data and a stationary object corresponding to the detected stationary object is not detected from the image data is satisfied. The image recognition method according to claim 8, wherein a request for deleting information related to the stationary object from the map data is transmitted to the server when it is determined that the condition is satisfied a specified number of times or more. The image recognition method according to claim 8, further comprising a receiving step of receiving updated map data from the server. An image recognition device for recognizing a stationary object included in an image taken by a camera attached to a vehicle,
A road recognition unit that recognizes a position of a road on the image data from image data generated by the camera photographing the outside of the vehicle;
Comparing the position of the road recognized by the road recognition unit with the position of the road on map data when the vehicle stored in a navigation device for specifying the position of the vehicle is viewed from a predetermined viewpoint. A distortion amount calculation unit that calculates a distortion amount between the image data and the map data,
Based on the distortion amount, a conversion unit that converts any one of the image data and the map data into data viewed from the viewpoint of the other data;
Based on the image data and the map data, one of which is converted by the conversion unit, by determining whether or not a stationary object exists at a position corresponding to each other, An image recognition apparatus comprising: a stationary object recognition unit that recognizes the stationary object. A program for recognizing a stationary object included in an image taken by a camera attached to a vehicle,
A road recognition step for recognizing a position of a road on the image data from image data generated by the camera photographing the outside of the vehicle;
Comparing the position of the road recognized in the road recognition step with the position of the road on map data when the vehicle stored in a navigation device for specifying the position of the vehicle is viewed from a predetermined viewpoint. A distortion amount calculating step of calculating a distortion amount between the image data and the map data;
A conversion step of converting either one of the image data and the map data into data viewed from the viewpoint of the other data based on the amount of distortion;
Based on the image data and the map data, one of which has been converted in the conversion step, by determining whether or not a stationary object exists at a position corresponding to each other in the data, A program for causing a computer to execute the stationary object recognition step for recognizing the stationary object.

Images