JP2009085651A - Image processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that in a conventional technique when an object to be recognized has moved between an overlapped area and a non-overlapped area of photographed images photographed by a plurality of photographing apparatuses, in other words, when it has moved from the overlapped area to the non-overlapped area, accurate image processing cannot be performed. <P>SOLUTION: A vehicle detection system includes: an input section for inputting an overlapped area and a non-overlapped area of a first area photographed by a first single-lens camera and a second area photographed by a second single-lens camera; a distance calculation section for recognizing another vehicle from an image within the overlapped area and calculating the distance between one's own vehicle and the other vehicle; and a tracking section which, when the other vehicle has moved from the overlapped area to the non-overlapped area, estimates the relative position of the other vehicle with respect to one's own vehicle on the basis of the distance calculated by the distance calculation section. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a system for processing an image of an approaching other vehicle.

  The demand for automobile safety is increasing. Along with the progress of an aging society, there is a need for a function that assists the driver in driving safely and comfortably.

  A technique using a stereo camera is known as a technique for monitoring the rear of the vehicle. However, using a stereo camera for rear monitoring is more expensive than using a monocular camera for rear monitoring. A stereo camera requires at least two imaging devices for one stereo camera, the data transfer amount of the stereo camera is at least double that of a monocular camera, and CPU processing performance is required to process the data. This is because periodic calibration is necessary in order for the relative position of the two cameras to change over time. Therefore, in recent years, the mounting rate of monocular rear view cameras has increased. This rear view camera is mainly used for rearward confirmation when the host vehicle moves backward. A camera for monitoring the rear side is also being mounted on the door mirror. Here, there is a system that performs image processing on imaging data captured by a plurality of imaging devices attached to a vehicle, recognizes an obstacle, and warns (see Patent Document 1).

JP 2006-50451 A

  Patent Document 1 does not describe a method for dealing with a case where a recognition target moves between an overlapping region and a non-overlapping region of captured images captured by a plurality of imaging devices. That is, there is a problem that accurate image processing cannot be performed when the recognition target moves from the overlapping region to the non-overlapping region.

  Therefore, an object of the present invention is to provide a system that performs accurate image processing even when a recognition target moves from an overlapping region to a non-overlapping region.

  In order to solve the above problems, one of the desirable embodiments of the present invention is as follows.

  The vehicle detection system includes: an input unit that inputs an overlapping area and a non-overlapping area between a first area captured by a first monocular camera and a second area captured by a second monocular camera; and an image in the overlapping area A distance calculation unit for recognizing the other vehicle and calculating a distance between the host vehicle and the other vehicle, and when the other vehicle moves from the overlapping region to the non-overlapping region, the distance calculation unit calculates the distance And a tracking unit that estimates a relative position of the other vehicle with respect to the host vehicle.

  According to the present invention, it is an object of the present invention to provide a system that performs accurate image processing even when a recognition target moves from an overlapping region to a non-overlapping region.

  Hereinafter, it will be described in detail with reference to the drawings.

  FIG. 1 is a configuration example of the entire system of the present invention.

  In FIG. 1, 101 is an overall system, 102 and 103 are on-vehicle cameras, 104 is a video input interface (input unit), 105 is a microcomputer, 106 is a random access memory, 107 is a read-only memory, 108 is a distortion correction unit, 109 is An image processing unit, 110 is a display unit, 111 is a display interface, 112 is an audio output unit, 113 is an audio output interface, and 114 is a bus.

  The in-vehicle cameras 102 and 103 are installed at predetermined locations outside the vehicle and transmit images outside the vehicle to the video input interface 104. In this embodiment, the in-vehicle camera 102 images the side and the in-vehicle camera 103 images the rear, and there is an overlapping area between these two imaging areas. The video input interface 104 is an interface for connecting the in-vehicle cameras 102 and 103 to the system of this embodiment. The microcomputer 105 is a circuit that can perform general-purpose arithmetic processing. The microcomputer 105 reads and activates the activation program from the read-only memory (ROM) 106, changes the data on the random access memory 106, and performs the entire system of this embodiment. Control the behavior. The random access memory 106 stores image data, mid-calculation results, vehicle detection results, and the like. The read only memory 107 stores the system program of this embodiment, constant values used for image correction, and the like. The distortion correction unit 108 stores the images acquired by the in-vehicle cameras 102 and 103 and stored in the random access memory 124 in advance in the ROM based on the parameters of the in-vehicle cameras 102 and 103. The image is corrected to a non-existing image and stored in the random access memory 107 again.

  The image processing unit 109 is a dedicated circuit for image processing, performs processing on the image data stored on the random access memory 106, and writes the processing result back on the random access memory 106 again. Image processing can be performed only by the software of the microcomputer 105, but can also be performed at high speed using the image processing unit 109 as an accelerator. In that case, the microcomputer 105 makes settings necessary for the processing to the image processing unit 109, and the microcomputer 105 sends status information (such as information on the completion of the operation and information on the image properties such as a histogram) as necessary. Received from 109.

  The display unit 111 is connected to the bus 114 through the display interface 130 and is a device for displaying the vehicle detection result of the present embodiment. The display unit 111 is attached to the driver's seat side of the own vehicle. When notifying the user of the vehicle detection result of the system of the present embodiment, the audio output unit 112 notifies the user with a sound, an alarm sound, or the like (for example, a speaker). The bus 114 is connected to the main parts of the figure such as the microcomputer 105 and the random access memory 106, and information is exchanged between the connected components.

  In this paper, the case where the in-vehicle camera 103 is behind is described, but it goes without saying that the present invention can also be applied to a front camera.

  FIG. 2 is a diagram showing a range captured by the in-vehicle cameras 102 and 103 from above the own vehicle.

  In FIG. 2, 201 and 203 are lanes of the roadway, 202 is a center line of the roadway, 204 is an optical axis direction of the in-vehicle camera 103, 205 and 209 are lines written to indicate a photographing range of the in-vehicle camera 102, and 206 208 is a line drawn to indicate the shooting range of the in-vehicle camera 103, 207 is the optical axis direction of the in-vehicle camera 102, 210 is the own vehicle on which the embodiment of the present invention is mounted, and 211 is the direction in which the own vehicle is traveling. It is an arrow which shows.

  In FIG. 2, alphabets such as A, B, C, D, and E are shown. Each of these indicates a certain area, area A is a place where only the in-vehicle camera 103 is imaged, area B is a place where the in-vehicle cameras 102 and 103 are imaged, and area C is a place where only the in-vehicle camera 102 is imaged Is shown. Regions D and E indicate places where neither the in-vehicle camera 102 nor 103 is picked up.

  FIG. 3 is a flowchart for explaining the overall operation. FIG. 3A is a flowchart for explaining the overall operation during normal operation, and FIG. 3B is a flowchart for explaining the overall operation of calibration.

  In FIG. 3A, 302 is an input image confirmation process, 303 is a detection process, and 304 is a display process.

  The input image confirmation process 302 is a process for confirming whether there is a defect in the input image. The detection process 303 is a process for detecting and tracking a rear vehicle. The display process 304 is a process for displaying the detected vehicle in an easy-to-understand manner for the user or for announcing it by voice.

  In FIG. 3B, reference numeral 307 denotes an input image confirmation process, and reference numeral 308 denotes a calibration process.

  The input image confirmation process 307 is the same process as the input image confirmation process 302. The calibration process 308 is a process for adjusting parameters so that a three-dimensional position can be grasped more accurately with respect to the overlapping area by the in-vehicle cameras 102 and 103. The calibration process is a process performed in the manufacturing process before product shipment. In this process, the parameters are determined, and are written in the ROM before shipment.

  FIG. 4 is a diagram for explaining the input image confirmation processing 302. FIG. 4A is a flowchart of the input image confirmation process 302.

  In FIG. 4A, 402 is a histogram calculation process, and 403 is a smear confirmation process.

  The histogram calculation process 402 is a process for calculating the sum of gray images in the vertical direction in the input image as shown in the figure. By this histogram calculation process 402, one value is calculated for each X coordinate of the screen, and each sum is stored in an array having the same size as the horizontal size of the screen.

  The smear confirmation processing 403 uses the array in which the previously calculated result is stored to check whether smear has occurred at the X coordinate based on whether or not the array element exceeds a certain threshold. . Check all X coordinates on the screen to make sure that no smear has occurred. If smear is generated, the smear is transmitted to the subsequent processing. If smear has occurred, image processing is not performed for the frame in this embodiment. As other coping methods, a method of using the value of the adjacent pixel as it is without using the pixel of the column, or a method of using the average of surrounding pixels can be considered. Note that smear can occur when a CCD camera is used, and does not occur when a CMOS camera is used. In this embodiment, it is assumed that a CCD camera is used.

  FIG. 5 is a flowchart of the detection process 303.

  In FIG. 5, 501 is a lateral input image, 502 is a backward input image, 503 and 506 are distortion correction coefficients, 504 and 505 are distortion correction processes, 507 is an image preparation process, 509 and 510 are projective transformation processes, 508 and 511 are coefficients for projective transformation, 512 is a road detection process, 513 is a block matching process, 514 is a priority process candidate instruction process, 515 is a rear vehicle determination process, 516 is a tracking process, and 517 is a detection result. is there.

  The distortion correction process 504 is a process for obtaining an image in which the distortion of the camera is corrected based on a predetermined coefficient value using the side input image 501 as an input. The distortion correction processing 505 is the same except that the input image is the rear input image 502.

  The image preparation process 507 is a preparation process for detecting a rear vehicle from an image whose distortion has been corrected. Here, it is confirmed whether or not there is a defect in the input image, and the difference in brightness between the two in-vehicle cameras is corrected.

  The projective transformation process 509 is a process of artificially combining two camera images having different attachment positions and imaging directions (optical axes) (allowing parallax for the attachment positions). The same applies to the projective transformation processing 510. In the present embodiment, processing is performed on an image having a higher resolution of the two input images, thereby aiming to suppress a decrease in detection accuracy due to loss of image information.

  The traveling road detection process 512 is a process using an image, and is based on a technique generally called lane recognition. Furthermore, the shape of the rear road on the image is grasped based on the image information, the past travel history (location history), and the traveling direction of the host vehicle. Thereby, it is possible to recognize the rear vehicle approaching on the traveling road and the overtaking vehicle separately from the guardrail, the roadside tree, and the opposing vehicle. Although not described in this embodiment, road shape data can be obtained from a car navigation system and used.

  The block matching processing 513 performs block matching processing based on the data of the priority processing candidate instruction 514. This instruction data is obtained by the tracking process 516. Although block matching should be performed for all regions where vehicles can be detected, the data of the priority processing candidate instruction 514 is used as to which part of the region to process. In general, the block matching process takes time because of a large amount of calculation. Therefore, priorities are set in the processing areas on the image. If the processing does not end within a predetermined time, the low priority is not processed.

  The rear vehicle determination process 515 is a process of detecting a rear vehicle based on the results of the travel path detection process 512 and the block matching process 513.

  The tracking process 516 outputs the detection result 517 and the data of the priority process candidate instruction 514.

  Here, the operation of the distortion correction processing 504 and 505 will be described with reference to FIG.

  In FIG. 6, reference numeral 601 denotes an image before distortion correction, 602 denotes an image after distortion correction, and 603 to 606 and 607 to 610 each have four coordinates, one of a data group for instructing a distortion correction method. Is a set. Reference numerals 611 to 614 denote four pixels, and the intersection of two dotted lines 615 and 616 is a point (a point in sub-pixel units) where it is desired to obtain color information of the pixel after distortion correction.

  In order to create an image after distortion correction from an image having distortion by the distortion correction unit 108, a distortion correction table (a table indicating coordinate correspondence) is used based on the camera parameters of the in-vehicle cameras 102 to 104. It is created in advance by design data and calibration. In this embodiment, since there are two on-vehicle cameras, there are also two tables. The table is made up of a plurality of sets of coordinate data shown in FIG. For example, the position of the pixel that becomes the coordinates in the range indicated by coordinates 607 to 610 after distortion correction is obtained by performing linear interpolation calculation based on the coordinates 603 to 606 before distortion correction. At this time, the value of the coordinates before distortion correction may be coordinates after the decimal point, that is, in units smaller than the pixels (coordinates indicated by 615 and 616). In this case, the pixels 611 to 614 are used. Based on the color information of the four pixels, interpolation calculation is performed to obtain color information of coordinates represented by 615 and 616, which is used as the color of the pixel after distortion correction. Such processing is performed over the entire area of the image 602 after distortion correction, and the image 602 after distortion correction is obtained. The processing by the distortion correction unit is performed on images captured by all on-vehicle cameras in the present embodiment.

  FIG. 7 is a flowchart of the image preparation process 507.

  In FIG. 7, reference numeral 702 denotes a reduction process, and reference numeral 703 denotes a brightness correction process.

  The reduction process 702 is a process for reducing an image input from the camera. This is a process of creating a reduced image for an input image by taking the average of adjacent pixels. In image processing, generally, even if an input image has a high resolution, when it is actually processed, a reduced image is created, and image recognition processing is often performed on the reduced image.

  The brightness correction processing 703 is a process for correcting the brightness when the input images of the two cameras have different brightness. The aim of this processing is to reduce the influence of the difference in image brightness between the two cameras. What is used here is a road surface image. It is known that the road surface is generally irregularly reflected and often appears to have uniform brightness when viewed from any direction. In the present embodiment, this is used to adjust the apertures of the two cameras so that the same point on the road surface can be seen from the two cameras with the same brightness with respect to images taken by the two in-vehicle cameras. In addition, since it is possible to include an error if only one pair (coordinates) is compared, a plurality of comparison candidates are prepared in advance, and the coordinates are used as a list, and brightness comparison is performed for the plurality of candidates. After performing the processing, the expected brightness is obtained with little error.

  As described above, when the mounting positions and directions of the plurality of imaging devices are different, it is possible to reduce the effects of the brightness adjustment and the illumination / reflection on those captured images.

  The projective transformation processes 509 and 510 will be described with reference to FIGS. Projective transformation refers to arithmetic processing such as image enlargement, reduction, movement, and trapezoid transformation.

  FIG. 8 is a diagram showing that the camera is artificially rotated by the angle θ, and FIG. 9 is a diagram showing that the camera image is enlarged by L and the camera position is changed pseudo. At this time, with respect to the two cameras, the main process (projection conversion process) is performed on the camera that can obtain a higher resolution image in the overlapping region. This is to reduce loss of image information due to projective transformation.

  The two in-vehicle cameras have different imaging directions (optical axis directions) and their mounting positions. Therefore, projective transformation corresponding to the angle indicated by reference numeral 801 in FIG. 8 is performed on the image of the side camera to generate an image in which the optical axes are pseudo-parallel. Further, the image is converted (enlarged), whereby the camera image is converted into an image in the case where the attachment position 902 in FIG. As for the projective transformation performed here, it is known in advance what kind of transformation should be performed by calibration processing. The stereo matching processing in which the parallax between the two camera images is d can be applied by the above-described projective transformation twice.

  Although not described in the present embodiment, two images, that is, a photographed image at the camera position 102 in FIG. 2 and an image actually photographed by installing the camera at the position 502 in FIG. It is also conceivable to grasp the contents of image conversion to be performed by comparing the two points and grasping the corresponding points.

  FIG. 10 is an explanatory diagram of the block matching processing 513 in the present embodiment.

  In FIG. 10, reference numeral 1001 denotes a region cut out from the comparative image, 1002 denotes a reference image, 1003 denotes a line indicating a search direction, and 1004 denotes a distance image.

  The block matching process is a process performed by the CPU 105 on the overlapping area of the image. This processing is necessary for the stereo method calculation described in FIG.

  A region 1001 is a 5 × 5 image cut out from the comparison image, and is an image from one of the two in-vehicle cameras. The reference image 1002 is a vehicle-mounted camera image to which the region 1001 does not belong. Using the region 1001 and the reference image 1002, the difference in the horizontal direction (1003) between the two images is calculated in sub-pixel units. In this embodiment, it is obtained by normalized correlation. When used here, the distance in the depth direction is obtained, and a distance image 1004 is obtained. The distance image is a density image, and the density value is a distance value in the depth direction.

  A method for grasping a three-dimensional distance by the block matching processing 513 in FIG. 5 will be described with reference to FIG. In this embodiment, image measurement is performed by a stereo method.

  In FIG. 11, P0 is a target point, Pl is a point on the left eye image, Pr is a point on the right eye image, Po_l is the origin of the left eye coordinate system, Po_r is the origin of the right eye coordinate system, and O is xyz. The origin of the coordinate system, O_l is the center of projection of the left-eye camera, and O_r is the center of projection of the right-eye camera. Further, x, y, and z are the coordinate system of the entire space with O as the origin, v_l and v_r are the axes of the left eye coordinate system, and u_l and u_r are the axes of the right eye coordinate system. Further, f is the distance between the projection center and the image plane, and L is the distance between the left eye camera and the right eye camera.

  Here, it is known that the following relational expression holds because two straight lines of O_l and Pl and a straight line of O_r and Pr intersect at a point P0.

x0 = (L (u_l + u_r)) / (2 (u_l-u_r))
y0 = Lv (u_l-u_r)
z0 = (Lf) / (u_l-u_r)
Here, v_l and v_r have the same value and are represented by v.

  The coordinate of Pl and the coordinate of Pr can be obtained by the block matching process, and the coordinate of P0 can be obtained by the three relational expressions shown above, and the distance in the z direction can be obtained from the two-dimensional image. This arithmetic processing is performed by the CPU 105.

  The rear vehicle detection process 512 in FIG. 5 will be described with reference to FIGS. 12 and 13.

  In FIG. 12, 1201 is a left eye image (image by a side camera), 1202 is a right eye image (image by a rear camera), 1203 is a left eye image in an overlapping area, and 1204 is a left eye image 1203 in the overlapping area. It is an image divided into

  In FIG. 13, FIG. 13A shows an image 1204 in which the left-eye image is divided into minute regions, and FIG. 13B shows a voting result when the distance is taken on the horizontal axis for a certain minute region.

  In FIG. 13B, a portion 1301 is a region having a significant height with four minute regions. The presence of this region indicates the presence of an obstacle, and the width of the obstacle can be understood from the widths of the four minute regions. The rear vehicle is detected from this and the relative speed of the tracking target object. The relative speed can be calculated from the difference between the three-dimensional position stored in the tracking information list and the currently detected three-dimensional position, and the detection processing interval (time). At this time, it is assumed that a vehicle having a relative speed above a certain level is not detected as a rear vehicle. For example, in this embodiment, if the relative speed is substantially the same as the own vehicle speed, it is not detected as a rear vehicle because it is not a rear vehicle (at least not running).

  The tracking process 516 will be described with reference to FIGS.

  FIG. 14 is a flowchart of the tracking process 516.

  In FIG. 14, reference numeral 1401 denotes an overlapping area tracking process, 1402 denotes an overlapping area list update process, 1403 denotes a non-overlapping area tracking process, and 1404 denotes a non-overlapping area list updating process.

  The overlapping area tracking process 1401 is a process of comparing the list of vehicles already detected in the overlapping area with the vehicles detected in the current overlapping area. This comparison process compares the three-dimensional position of the vehicle detected this time and the vehicle detected previously (vehicles in the list), and vehicles within a certain distance range (for example, from the size of one vehicle). Are detected to be the same vehicle). On the other hand, newly detected vehicles that are not so are added to the list. In addition, a vehicle that was detected last time but not detected this time is transmitted to the non-overlapping area tracking process as one of the tracking candidates in the non-overlapping area.

  The overlap area list update process 1402 updates the list based on the result of the overlap area tracking process 1401 and outputs a priority process candidate instruction 514.

  The non-overlapping area tracking process 1403 is a process for performing tracking based on a list of vehicles already detected in the non-overlapping area and a template image of the target vehicle acquired in the overlapping area. In addition, the first tracking is attempted for the tracking candidate in the non-overlapping area transmitted from the overlapping area tracking processing 1401. That is, template matching is performed.

  The non-overlapping area list update process 1404 is a process for updating the list based on the result of the non-overlapping area tracking process 1403.

  Here, the data structure of the overlapping area list and the non-overlapping area list will be described. Both lists include a vehicle ID, a detection position (on the image), a detection position (three-dimensional), a relative speed with the host vehicle, and a template image. The vehicle ID is a number assigned to the detection target, and is used to distinguish the detection target on the list. The detection position (on the image) and the detection position (three-dimensional) indicate the coordinate position of the detection target. The relative speed with the own vehicle is a speed to be detected and a relative speed with the own vehicle, and is calculated based on the previous and current detection positions (three-dimensional). The template image is a template image in front of the vehicle.

  Here, with reference to FIG. 19, the structure of a list indicating non-overlapping areas and overlapping areas relating to tracking, and a list indicating priority candidates will be described.

  FIG. 19A shows a non-overlapping area list and an overlapping area list related to tracking. This list has a structure as shown in the figure, and one vehicle entry corresponds to one line. FIG. 19B is a list for instructing priority candidates. In this list, one vehicle entry corresponds to one line, and each line stores one coordinate on the image. This list is arranged in descending order of priority as candidates.

  With reference to FIG. 16, the template-related processing performed in the overlapping area tracking processing will be described.

  In FIG. 16, 1601 is a template acquisition process, and 1602 is a template update process.

  The template acquisition process 1601 is a process performed for a vehicle newly registered in the overlapping area tracking list, and is a process of acquiring an image of the front part of the vehicle. The template update process 1602 is a process of updating (re-acquiring) a template of a vehicle that has already been acquired while the vehicle is tracking in the overlapping area. In this embodiment, it is assumed that a plurality of frames are acquired and stored in the same vehicle for the template image. In general, during traveling, the acquired image differs depending on the traveling environment (such as the sun or shade). Therefore, in order to cope with the change in the environment, a system is obtained in which a plurality of templates are acquired for the same vehicle, and a plurality of templates can be sequentially used as necessary at the time of template matching.

  In this system, when there is an object (vehicle) in the overlapping area, the image is held as a template, and when the object moves to the non-overlapping area, the subsequent behavior is continuously tracked by template matching. Further, the template is acquired when the target object is in the overlapping region. Template matching is easier to apply if it is rich in geometric features. In the case of a car, the front side of the vehicle (headlight, license plate, etc.) is rich in characteristics, but the side of the vehicle is poor in characteristics.

  In this embodiment, there are two non-overlapping areas, but the appearance of the detection target vehicle differs depending on the non-overlapping area in the rear input image and the non-overlapping area in the side input image. Come. In the former case (1503 in FIG. 15), not only the front side of the vehicle but also the side is visible, whereas in the latter case (1502 in FIG. 15), the side is not visible only on the front side of the vehicle. Therefore, in the template update processing 1602, for a vehicle approaching the position 1503 in FIG. 15, a certain pixel at the right end of the vehicle image is not acquired, that is, to the left (to acquire the vehicle front surface). Is acquired and retained as a template. On the other hand, for a vehicle that is approaching the area 1502 in FIG. 15, a process of holding the template as it is detected as the front surface is performed. This is a process that takes into account the relative positional relationship between the camera mounting position and the detected vehicle. Although not described in the present embodiment, in template matching, a method may be considered in which each pixel of the template is weighted and the weight is set in consideration of the relative positional relationship.

  FIG. 17 shows a screen displayed by the display process 304.

  In FIG. 17, 1801 is a display screen, 1802 is a mark indicating the vehicle, 1803, 1804, 1805 and 1806 are lines indicating the boundaries of the area, and 1807 is a mark indicating the detected vehicle.

  When it is detected that the rear vehicle is in the region C in FIG. 2, the region surrounded by the boundary lines 1803 and 1804 is lit in yellow. Similarly, when the rear vehicle is in the region A in FIG. On the other hand, when the rear vehicle is in the region B in FIG. 2, the region surrounded by the boundary lines 1804 and 1805 is lit in light blue, and the relative position to the own vehicle is a rectangle (1807) in the light blue region. Is displayed. The difference in operation between the case where it is lit in yellow and the case where it is lit in light blue and has a rectangular display reflects the difference in detection accuracy of the rear vehicle. As for the detection accuracy, since the non-overlapping area is detected not by stereo but by a monocular image, the distance accuracy of the detection target decreases. These displays are performed by a microcomputer, and are processed based on a list of detection results of overlapping areas and non-overlapping areas. It is also possible to output sound as necessary during or during detection. In addition, when the input image is determined not to be suitable for the processing in the input image confirmation processing 302 of FIG. 3, a display indicating that the input error state is present is displayed on the display system.

  Next, a case where matching processing is performed using feature points of an image will be described with reference to FIG. Reference numerals 1801 and 1802 denote feature point calculation processing, and 1803 denotes feature point matching processing.

  A feature point calculation process 1801 performs a calculation process that extracts a feature point such as an edge or a dot, which is generally used in image processing. The traveling path detection feature point calculation processing 1802 is the same. The feature point matching process 1803 is a process for matching feature points. By performing processing based on the feature points, it is possible to expect further improvement in resistance to changes in brightness in two camera images.

The block diagram of the whole system. The flowchart which shows operation | movement of the whole system. The figure for demonstrating whole operation | movement. The flowchart for demonstrating an input image confirmation process. The figure for demonstrating a detection process. The figure for demonstrating distortion correction. The figure for demonstrating an image preparation process. The figure for demonstrating projective transformation. The figure for demonstrating projective transformation. The figure for demonstrating a template matching process. The figure for demonstrating a stereo process. The figure for demonstrating a back vehicle detection process. The figure for demonstrating a back vehicle detection process. The figure for demonstrating tracking processing. The figure for demonstrating tracking processing. The figure for demonstrating a template process. The figure for demonstrating a display screen. The flowchart in the case of performing a matching process using the feature point of an image. The figure which shows the list | wrist of a non-overlapping area | region and overlapping area | region regarding a tracking, and a priority candidate.

Explanation of symbols

102,103 In-vehicle camera 105 CPU
106 RAM
107 ROM
108 Distortion Correction Unit 109 Image Processing Unit 110 Display Unit 111 Display Interface 112 Audio Output Unit 113 Audio Output Interface 114 Bus

Claims (8)

An input unit for inputting an overlapping region and a non-overlapping region between a first region imaged by a first monocular camera and a second region imaged by a second monocular camera;
A distance calculation unit that recognizes another vehicle from the image in the overlapping region and calculates a distance between the host vehicle and the other vehicle;
An image processing system comprising a tracking unit that estimates a relative position of the other vehicle with respect to the host vehicle based on the distance calculated by the distance calculation unit when the other vehicle moves from the overlapping region to the non-overlapping region. . An extraction unit for extracting a feature amount in the overlapping region;
The image processing system according to claim 1, wherein the distance calculation unit calculates the distance based on the feature amount extracted by the extraction unit.   The image processing system according to claim 2, wherein the feature amount indicates an edge or a quadrilateral.   The image processing system according to claim 1, further comprising a correction unit that performs distortion correction and viewpoint conversion on at least one of the first and second regions.   The image processing system according to claim 4, wherein the correction unit adjusts apertures of the first and second monocular cameras so that the first and second monocular cameras can capture images with the same brightness.   The image processing system according to claim 5, wherein the correction unit adjusts the diaphragm based on an image of a road surface.   The image processing system according to claim 5, wherein the tracking unit performs different processing on each of an image captured by the first monocular camera and an image captured by the second monocular camera. The first monocular camera is attached to the rear of the host vehicle, and the second monocular camera is
The image processing system of Claim 1 attached to the side of the said own vehicle.

Images