JP2007158639A - Car driving support apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a car driving support apparatus capable of obtaining images having less distortions with less labor, after a camera and a lens have been replaced. <P>SOLUTION: The car driving support apparatus has the camera 10a or the like, photographing the periphery of a car through a fish-eye lens, a photographed-image storage section 12a or the like for storing a photographed image data and a VRAM 23. The car driving support apparatus further has mapping-table storage sections 32 and 36 for storing a mapping table displaying the relationship of a reading address at a time, when the image data are read from the photographed-image storage section 12a or the like, and a writing address at the time, when the read image data is stored in the VRAM 23. The car driving support apparatus has a top-view image conversion section 20 for writing the image data to the VRAM 23, on the basis of the mapping table and a mapping-table conversion section 34 for correcting the contents of the mapping table, on the basis of the place of a reference set to the photographed image by using the camera 10a or the like. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a vehicle driving support device that supports driving of a vehicle by photographing the periphery of the vehicle using a camera with a fisheye lens attached and displaying it on a screen.

2. Description of the Related Art Conventionally, there has been known an apparatus that corrects a distorted image captured using a camera equipped with a fisheye lens to obtain a normal image (see, for example, Patent Document 1). In this apparatus, image data photographed through a fisheye lens is stored in an image memory, and this image data is read out using an address converted in a lookup table, thereby converting a distorted image into a normal image. Processing is performed.
Japanese National Patent Publication No. 11-510341 (page 6-17, FIG. 1-8)

  By the way, in the apparatus disclosed in Patent Document 1 described above, the correspondence between the address of the distorted image and the address of the corrected image is performed using a look-up table, but this attachment shifts the attachment position of the fisheye lens. Therefore, if the entire camera or the fisheye lens is replaced, there is a possibility that a correct image from which distortion is removed cannot be obtained. In order to avoid such an inconvenience, it is necessary to exchange the memory storing the look apple table, and there is a problem that it takes a lot of trouble when exchanging the camera and the lens.

  The present invention was created in view of the above points, and an object of the present invention is to provide a vehicle driving support device that can obtain an image without distortion with little effort after replacing a camera or a lens. is there.

  In order to solve the above-described problem, a vehicle driving support apparatus according to the present invention includes a camera that captures the periphery of a vehicle through a wide-angle lens, captured image storage means that stores image data obtained by capturing with the camera, and a storage address. Display memory corresponding to the display position on the display screen, an address indicating the relationship between the read address when reading the image data from the captured image storage means and the write address when storing the read image data in the display memory For address conversion information storage means storing conversion information, image data read / write means for writing image data read from the photographed image storage means to the display memory based on the address conversion information, and an image photographed using a camera To determine the reference position for which the relative position is set, and based on this reference position, And a correcting means for correcting the. If the camera or the wide-angle lens attached to it is replaced, the shooting position shifts due to the change in the mounting position. The amount of shooting position shift is calculated, and the content of the address conversion information is calculated according to this shift amount. Since it is only necessary to make a change, it is not necessary to replace the entire memory in which the address change information is stored, and an image without distortion can be obtained with little effort when replacing the camera or the wide-angle lens.

  Further, it is desirable that the correction means described above determines the circular center as a reference position for a circular image obtained by photographing through a wide-angle lens. Although a circular image can be obtained by using a wide-angle lens such as a fisheye lens, it is easy to find the center of the circle for the circular image, and the center of the circle is uniquely determined for the circular image. In addition, the amount of deviation of the reference position can be determined.

  Further, the correction means described above calculates an amount of deviation of the reference position when the wide-angle lens is actually attached to the reference position when the wide-angle lens is attached to a predetermined position of the camera, and converts the address. It is desirable to perform correction by shifting the address for reading image data from the captured image storage means included in the information by this amount of deviation. The process of shifting the read address by the shift amount can be realized by simple subtraction (or addition), the burden required for the correction process can be reduced, and an inexpensive processing device such as a CPU can be used. .

  Further, the correction by the correction means described above is preferably performed at the time of camera installation, after replacement, or after replacement of the wide-angle lens attached to the camera. Since it is only necessary to perform the correction process once after the vehicle driving support device is mounted on each vehicle or after the camera or the like is exchanged, it is possible to avoid hindering subsequent operations.

  Hereinafter, a vehicle driving support apparatus according to an embodiment to which the present invention is applied will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration of a vehicle driving support device according to an embodiment. 1 includes a camera 10a to 10d, captured image storage units 12a to 12d, a top view image conversion unit 20, a display processing unit 22, a display device 24, a camera characteristic determination unit 30, and a mapping table storage unit. 32 and 36, and a mapping table conversion unit 34.

  The cameras 10a to 10d photograph the surroundings of the host vehicle through a fisheye lens as a wide angle lens. FIG. 2 is a diagram illustrating the installation positions of the four cameras 10a to 10d. The camera 10a is installed in the door mirror on the right side of the vehicle 200 that is the host vehicle. A camera 10b is installed on the left door mirror. A camera 10c is installed in the rear trunk door. A camera 10d is installed in the vicinity of the front bumper. Each of these cameras 10a to 10d photographs the periphery of the vehicle 200 through a fisheye lens having a viewing angle of 180 degrees. By these cameras 10a to 10d, a horizontal side view image including the ground is obtained with respect to the vehicle 200. The side view image data output from each of the cameras 10a to 10d is stored in each of the captured image storage units 12a to 12d.

  The top view image conversion unit 20 performs viewpoint conversion processing for generating a top view image based on the side view images stored in the captured image storage units 12a to 12d. Specifically, since the side view image taken through the fisheye lens is distorted, the coordinates of the top view image after reading out the data for each pixel constituting the distorted image and removing the distortion The viewpoint conversion process is performed by writing the data of each pixel at (address). That is, a top view image is generated by changing an address for reading data and a writing address for each pixel, and this address conversion is performed using a mapping table as address conversion information. The mapping table will be described later. The top view image conversion unit 20 writes the data for each pixel read from each of the captured image storage units 12 a to 12 d in a VRAM (video RAM) 23 in the display processing unit 22. The VRAM 23 is a display memory, and a storage position and a display position are associated with each other. In the present embodiment, since an image around the vehicle is a display target, the top view image generated based on the side view image captured by the camera 10a is displayed on the right side of the display screen, and the side view image captured by the camera 10b. The top view image generated based on the side view image is captured on the left side of the display screen, and the top view image generated based on the side view image captured by the camera 10c is displayed on the lower side of the display screen. Writing to the VRAM 23 is performed so that the top view images generated based on the images are respectively arranged on the upper side of the display screen.

  The display processing unit 22 reads the top view image data stored in the VRAM 23 and converts it into a predetermined format video signal (for example, an NTSC video signal). This video signal is output to the display device 24, and a top view image is displayed on the screen. The display device 24 is configured by using, for example, a liquid crystal display device (LCD), and is installed at a vehicle interior position where the driver can easily see.

  The camera characteristic determination unit 30 determines the shooting ranges of the cameras 10a to 10d as camera characteristics. Specifically, the camera characteristic determination unit 30 determines the attachment position of the fisheye lens attached to each of the cameras 10a to 10d as the camera characteristic. FIG. 3 is an explanatory diagram of the attachment position of the fisheye lens. In FIG. 3, a rectangular frame A is an area corresponding to an image sensor such as a CCD (Charge Coupled Device) provided in the camera 10a. In this embodiment, a subject passing through a fisheye lens is photographed. Only an area included in the image is an effective shooting range. Further, since the relative position of the circle B with respect to the rectangular frame A is slightly shifted depending on the attachment position of the fisheye lens, a unique value for each of the cameras 10a to 10d or each fisheye lens when the fisheye lens is replaced. It becomes. The camera characteristic determination unit 30 determines the coordinates (x0, y0) of the center O of the circle B shown in FIG. 3 as the attachment position of the fisheye lens. The determination of the attachment position is performed for each of the cameras 10a to 10d. In this embodiment, the center O of the circle B is treated as a reference position in which a relative position is set with respect to a circular image photographed through the fisheye lens by the camera 10a or the like.

  The mapping table storage unit 32 determines in which address of the VRAM 23 the image data read by designating an address from the captured image storage unit 12a when the fisheye lens is mounted at an ideal position as designed. Stores the mapping table shown. FIG. 4 is an explanatory diagram of the mapping table, and shows the contents relating to the camera 10a. As shown in FIG. 4, an image photographed by the camera 10a is stored in a photographed image storage unit 12a whose storage address is indicated by a rectangular frame A. Since the camera 10a captures the right side surface of the vehicle, the VRAM 23 corresponds to the right region D of the display range indicated by the rectangular frame C. As described above, the image data photographed through the fisheye lens and stored in the photographed image storage unit 12 a corresponds to a part of the area in the VRAM 23, and the image data is read from the photographed image storage unit 12 a and stored in the VRAM 23. By adjusting the relationship between the read / write addresses (for example, the read address corresponding to P1 and the write address corresponding to Q1 are made to correspond, and the read address corresponding to P2 and the write address corresponding to Q2 are made to correspond to each other) The mapping data can be used to convert the distorted image data obtained through the fisheye lens into image data without distortion. The mapping table shows the relationship between the read / write addresses. It is. The mapping table storage unit 32 includes a mapping table corresponding to this design value, that is, the center of the rectangular frame A corresponding to the captured image storage unit 12a and the center O of the circle B corresponding to the fisheye lens as shown in FIG. Stores a mapping table created assuming that they match.

  However, in practice, the attachment position of the fisheye lens is deviated from the design value for each camera. Therefore, when the mapping table stored in the mapping table storage unit 32 is used, distortion occurs with the displacement of the attachment position. The mapping table conversion unit 34 performs a process of correcting the contents of the mapping table stored in the mapping table storage unit 32 in order to remove distortion caused by a shift in the mounting position of the fisheye lens, and performs mapping in consideration of the mounting position of the fisheye lens. Create a table. The corrected mapping table is stored in the mapping table storage unit 36.

  FIG. 5 is a diagram showing an overview of mapping table correction processing by the mapping table conversion unit 34. In FIG. 5, a circle B represents a range photographed through a fisheye lens with an ideal attachment, and a circle B ′ represents a range photographed through a fisheye lens actually attached. As described above, the actual attachment position deviates from the design value by Δx in the horizontal direction and Δy in the vertical direction when viewed from the center (circular center) coordinates (O → O ′). These values Δx and Δy are calculated based on the attachment position of the fisheye lens determined by the camera characteristic determination unit 30. The mapping table storage unit 32 shifts the values of the read addresses included in the mapping table stored in the mapping table storage unit 32 by these values Δx and Δy (actually Δx from the horizontal coordinate value of each read address). Is subtracted from the vertical coordinate value), a corrected mapping table is created and stored in the mapping table storage unit 36. The top view image conversion unit 20 performs viewpoint conversion processing using the corrected mapping table created in this way, and generates a top view image.

  The above-described photographed image storage units 12a to 12d are mapped to the photographed image storage unit, the VRAM 23 to the display memory, the mapping table storage unit 32 to the address conversion information storage unit, and the top view image conversion unit 20 to the image data read / write unit. The table conversion unit 34 corresponds to each correction unit.

  The vehicle driving support apparatus according to the present embodiment has such a configuration. Next, an operation for correcting the mapping table will be described. The mapping table is corrected when the necessity arises, for example, when any of the cameras 10a to 10d is replaced, or when a fisheye lens attached to any of the cameras 10a to 10d is replaced.

  FIG. 6 is a flowchart showing an operation procedure for correcting the mapping table. The camera characteristic determination unit 30 determines whether or not a mapping table resetting instruction has been issued (step 100), and repeats this determination by making a negative determination until an instruction is given. After the cameras 10a to 10d or the fisheye lenses attached thereto are exchanged, if the user operates the operation unit (not shown) to instruct the resetting of the mapping table, an affirmative determination is made in step 100. . In the reset instruction, it is instructed which camera to reset the mapping table. For example, the following description will be made assuming that the camera 10a has been replaced.

  Next, the camera characteristic determination unit 30 captures a captured image from the captured image storage unit 12a corresponding to the camera 10a to be exchanged (step 101), and sets the center O ′ of the circle B ′ corresponding to the captured image shown in FIG. The coordinates are determined (step 102).

  Next, the mapping table conversion unit 34 calculates deviation amounts Δx and Δy of the center O ′ of the circle B ′ corresponding to the actual attachment position with respect to the center O of the circle B when ideal attachment is performed. (Step 103), conversion processing for correcting the contents of the mapping table is performed using the deviation amounts Δx and Δy (Step 104). In this way, the mapping table is corrected once after the camera or fisheye lens is replaced.

  As described above, in the vehicle driving support device according to the present embodiment, when the cameras 10a to 10d and the fisheye lens attached to the cameras 10a to 10d are replaced, the photographing position shifts due to the change in the mounting position. Therefore, it is only necessary to change the contents of the mapping table (address conversion information) according to the amount of deviation, so there is no need to replace the entire memory storing the mapping table, and the cameras 10a to 10d and the fisheye lens. It is possible to obtain a distortion-free image with little effort when exchanging. In particular, the center of the circle is determined as a reference position for a circular image obtained by photographing through a fisheye lens. Although a circular image can be obtained by using a fisheye lens, it is easy to obtain the center of a circle for the circular image, and the center of the circle is uniquely determined for the circular image. The amount of deviation can be determined.

  In addition, the mapping table content correction is performed by calculating the amount of deviation of the center of the circular image actually obtained after camera replacement or the like, and shifting the read address included in the mapping table by this amount of deviation. ing. The process of shifting the read address by the shift amount can be realized by simple subtraction (or addition), the burden required for the correction process can be reduced, and an inexpensive processing device such as a CPU can be used. .

  The mapping table is corrected at the time of camera installation, after replacement, or after replacement of a wide-angle lens attached to the camera. Since it is only necessary to perform the correction process once after the vehicle driving support device is mounted on each vehicle or after the camera or the like is exchanged, it is possible to avoid hindering subsequent operations.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation implementation is possible within the range of the summary of this invention. In the embodiment described above, the case where the four cameras 10a to 10d are installed around the vehicle has been described, but the number of cameras may be one or a plurality other than four.

  In the above-described embodiment, the center (circular center) O of the circular image obtained through each of the cameras 10a to 10d is set as the reference position. May be set as the reference position. For example, an intersection point between the circumference of the circular image and a horizontal line passing through the center of the circle may be set as the reference position, or a position separated from the center of the circle by a predetermined distance in the horizontal direction and the vertical direction may be set as the reference position.

It is a figure which shows the structure of the vehicle driving assistance device of one Embodiment. It is a figure which shows the installation position of four cameras. It is explanatory drawing of the attachment position of a fisheye lens. It is explanatory drawing of a mapping table. It is a figure which shows the outline | summary of the correction process of the mapping table by a mapping table conversion part. It is a flowchart which shows the operation | movement procedure which correct | amends a mapping table.

Explanation of symbols

10a to 10d Cameras 12a to 12d Captured image storage unit 20 Top view image conversion unit 22 Display processing unit 23 VRAM
24 Display device 30 Camera characteristic determination unit 32, 36 Mapping table storage unit 34 Mapping table conversion unit

Claims (4)

A camera that captures the surroundings of the vehicle through a wide-angle lens;
Photographic image storage means for storing image data obtained by photographing with the camera;
A display memory whose storage address corresponds to the display position on the display screen; and
Address conversion information storage means storing address conversion information indicating the relationship between a read address when reading the image data from the captured image storage means and a write address when storing the read image data in the display memory When,
Image data read / write means for writing the image data read from the captured image storage means to the display memory based on the address conversion information;
Correction means for determining a reference position in which a relative position is set with respect to an image captured using the camera, and correcting the content of the address conversion information based on the reference position;
A vehicle driving support device comprising: In claim 1,
The vehicle driving support device according to claim 1, wherein the correcting means determines a circular center of a circular image obtained by photographing through the wide-angle lens as a reference position. In claim 1 or 2,
The correction means calculates a deviation amount of the reference position when the wide-angle lens is actually attached to the camera with respect to the reference position when the wide-angle lens is attached to a predetermined position of the camera. A vehicle driving support apparatus that performs a correction to shift an address for reading image data from the captured image storage means included in the address conversion information by the amount of deviation. In any one of Claims 1-3,
The correction by the correction means is performed at the time of installation of the camera, after replacement, or after replacement of the wide-angle lens attached to the camera.

Images