JP2006060460A - Image correction apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image correction apparatus carrying out image conversion with a small memory capacity and a small processing load. <P>SOLUTION: An image converter 1 for a vehicle is connected to a camera module 2 for imaging its surrounding object to generate image data and includes a table storage section 15 which stores in advance an address conversion table wherein address information of input image data with an address space with a resolution greater than that of a monitor 3 longitudinally or laterally and address information of display image data by the resolution of the monitor 3 are described. When the converter 1 receives the input image data imaged by the camera module 2, an image conversion section 14 references the address conversion table, segments image data of the amount equivalent to the resolution to be displayed on the monitor 3 from the input image data, converts the address information of the segmented image data into the address information of the display image data, and outputs the converted input image data to the monitor 3 as the display image data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image correction apparatus that captures, for example, a vehicle periphery, converts an image generated by capturing the image, and presents the image to a driver or the like.

  Conventionally, vehicle surroundings images in a driving support system to assist the driver's view in situations such as when the vehicle is moving backwards, such as in the garage or width adjustment of the vehicle, or when the vehicle enters an intersection or T-junction with poor visibility Display devices are known.

As such a technique, as described in Patent Document 1 below, a rear blind spot of a vehicle is captured by a camera, and the camera image is converted into an image viewed from a virtual camera position different from the actual camera position. I am letting. That is, the image conversion apparatus described in Patent Document 1 changes the presentation area of the output image by converting the input image captured from the actual camera position.
JP 2001-163132 A

  However, in the above-described conventional technology, the optical axis of the camera lens may deviate from a predetermined position due to a camera manufacturing error, a camera mounting error by an installation worker, vibration due to vehicle traveling, or the like. Such a shift in the optical axis is synonymous with a shift in the camera imaging range, and it is necessary to correct the shift in the optical axis.

  On the other hand, when an address conversion table for converting a camera image to correct the optical axis deviation is mounted in the memory in order to correct and display the optical axis deviation of the camera by the image conversion device, the optical axis deviation direction Since it is necessary to prepare an address conversion table for each degree of deviation and to prepare an enormous number of address conversion tables, the memory capacity of the image conversion apparatus increases.

  In addition, as a technique for suppressing such an increase in memory capacity, when an address conversion table is generated in real time by an image conversion device in consideration of an optical axis shift, the amount of calculation for generating the address conversion table is large. As a result, the processing load on the CPU increases. In such a state, for example, when an input delay of image data from a plurality of cameras or an output delay to the monitor occurs, the processing of the CPU may be delayed, and there is a possibility that the image cannot be displayed. .

  Therefore, the present invention has been proposed in view of the above-described circumstances, and an object thereof is to provide an image correction apparatus capable of performing image conversion with a small memory capacity and processing load.

  An image correction apparatus according to the present invention is connected to an imaging unit that captures an image of surroundings and generates image data. In order to display the image data on a display unit, the image correction apparatus is preliminarily stored in a vertical direction rather than the resolution of the display unit Alternatively, an address conversion table describing address information of input image data having an address space with a large resolution in the horizontal direction and address information of display image data corresponding to the resolution of the display means is stored.

  Then, when the input image data picked up by the image pickup means is input by the input means, the image correction device refers to the address conversion table by the image processing means and displays an image corresponding to the resolution displayed on the display means from the input image data. The data is cut out, the address information of the cut out image data is converted into the address information of the display image data, and the image data cut out by the image processing means by the output means and converted into the address information is used as display image data. Output to the display means.

  In such an image correction apparatus, when image data for the resolution to be displayed on the display means is cut out from the input image data, the image data to be converted can be moved by moving the area of the address conversion table to be referenced. Even with one address conversion table, it is possible to correct a shift in the imaging range of the imaging means.

  According to the image correction apparatus of the present invention, an address conversion table having an address space larger than the address space corresponding to the resolution of the display unit is prepared, and the position to be cut out from the input image data by the address conversion table is the physical position of the imaging unit. By setting to correct the optical axis deviation, it is not necessary to prepare an address conversion table with an enormous capacity to display an image in which the imaging range of the imaging means is corrected, and the imaging range can be obtained with simple processing. Can be corrected.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  The present invention is applied to a vehicular image conversion apparatus 1 configured as shown in FIG. 1, for example.

[First Embodiment]
[Configuration of vehicle surroundings display system]
The vehicle surrounding display system including the vehicle image conversion device 1 includes a plurality of camera modules 2A and 2B (hereinafter simply referred to as “camera module 2” when collectively referred to) and a vehicle image conversion device 1. The monitor 3 is connected. In this example, a case where two camera modules 2A and 2B are provided will be described, but an arbitrary number may be used.

  The camera module 2A and the camera module 2B are installed in different parts of the vehicle body and have different fixed imaging directions. The camera module 2 includes an imaging lens 21 and a CCD (charge coupled device) 22. The camera module 2 includes, for example, an NTSC (National Television System Committee) camera, and outputs image data to the vehicular image conversion apparatus 1 according to the NTSC system.

  The vehicular image conversion apparatus 1 includes input buffers 12A and 12B (hereinafter collectively referred to as a unit “input buffer 12”), a CPU (Central Processing Unit) 13, an image conversion unit 14, A table storage unit 15 and an output buffer 16 are connected.

  The input buffer 12 is provided corresponding to the number of camera modules 2, the input buffer 12A is connected to the camera module 2A, and the input buffer 12B is connected to the camera module 2B. The input buffer 12 temporarily stores NTSC input image data, and reads the input image data at the image conversion timing by the image conversion unit 14 in accordance with the control of the CPU 13.

  The table storage unit 15 stores address conversion tables 15a, 15b, 15c,... For each image layout presented to the driver. That is, for example, when the right side image and the left side image are displayed on the monitor 3 on one screen, an address conversion table according to the image layout is used, and the right side image, the left side image, and the lower image are combined. When displaying on the screen, an address conversion table according to the image layout is used.

  The address conversion table stored in the table storage unit 15 includes a memory address (address information) of input image data having an address space having a resolution larger in the vertical direction or the horizontal direction than the resolution of the monitor 3, and the resolution of the monitor 3. This is a table describing the memory address (address information) of the output buffer 16 for image data for display. That is, the address conversion table describes the memory address of the output buffer 16, that is, the display coordinates of the monitor 3 and the coordinates of the input buffer 12. Such an address conversion table is created in advance based on the specifications of the camera modules 2A and 2B, the mounting position, the direction in the imaging direction (optical axis), and the like.

  These address conversion tables may be not only an address space corresponding to the output resolution of the image displayed on the monitor 3, but also an address space wider in the vertical direction or the horizontal direction than the address space corresponding to the output resolution of the monitor 3. Alternatively, the address space may be wider in both the vertical direction and the horizontal direction than the address space corresponding to the output resolution of the monitor 3.

  Under the control of the CPU 13, the image conversion unit 14 reads one of the address conversion tables from the table storage unit 15, refers to the address conversion table, and stores the input image data stored in the input buffer 12 in the output buffer 16. Let Here, since the address conversion table describes the memory address of the output buffer 16 according to the image layout, the display image data replaced by the output buffer 16 by the image conversion unit 14 is the same image as the used address conversion table. The image has been converted to layout.

  At this time, the image conversion unit 14 determines an image area corresponding to the output resolution of the monitor 3 that is actually used for address conversion with reference to the address conversion table, and outputs the pixels of the input buffer 12 included in the image area to the output buffer 16. Performs address conversion, which is a replacement process to replace with. This is realized by designating a head pointer (address information) of an address conversion table for extracting an image area corresponding to the output resolution of the monitor 3 used for address conversion. Note that the image conversion unit 14 may be configured by an LSI, FPGA, DSP, or the like, or may be performed by the CPU 13.

  As a result, the image conversion unit 14 performs a process of correcting the shift of the image displayed on the monitor 3 due to the physical shift of the optical axis from the initial stage when the camera module 2 is attached, as will be described later.

  The output buffer 16 stores the image data address-converted by the image conversion unit 14, that is, the image data for display on the monitor 3, and outputs it to the monitor 3 under the control of the CPU 13.

  The CPU 13 recognizes an image layout instruction or an image switching instruction determined by operating an operation input unit (not shown) by the driver, and determines an address conversion table used for image conversion processing in the image conversion unit 14. Further, the CPU 13 switches the image to be displayed on the monitor 3 by controlling the image conversion timing from the input buffer 12 of the image conversion unit 14 and the data output timing of the output buffer 16 to the monitor 3.

[Specific example of address translation processing]
Next, an address conversion process performed by the vehicle image conversion apparatus 1 configured as described above will be described with a specific example.

  In this description, the resolution of the CCD 22, that is, the resolution of image data stored in the input buffer 12, is VGA (640 × 480 pixels), the resolution of the monitor 3 is QVGA (320 × 240 pixels), and VGA input image data. It is assumed that an image is cut out and an address conversion process is performed. Therefore, the address conversion table has a larger image area in the vertical direction or the horizontal direction and in both directions than the resolution of 320 × 240 of the monitor 3.

  In this example, it is assumed that the address space to be subjected to the address conversion process is 400 × 300 pixels. As a result, as shown in FIG. 2, an image 201 (address conversion target image 201) for 400 × 300 pixels out of camera images (input image data) 200 (640 × 480 pixels) captured in the camera imaging range is addressed. It becomes a conversion target. That is, the address conversion table stores address information for 400 × 300 pixels and address information for 320 × 240 pixels of the monitor 3.

  Here, the address conversion table is prepared for each camera module 2 having a fixed optical axis. When the address conversion target image 201 to be used is brought close to the contour portion of the camera image 200, the address conversion table is located near the end of the address conversion target image 201. Image distortion occurs. Therefore, the address conversion table sets the range of the address conversion area limited to a range that is larger than the resolution of the monitor 3 and that does not cause image distortion.

  The vehicular image conversion apparatus 1 sets the head pointer of the address conversion table for each camera module 2. That is, for example, the address information at the upper left end of the address conversion area is set as a head pointer, and an image of 320 × 240 pixels on the monitor 3 is cut out from the head pointer. Thereby, when reading the camera image from the input buffer 12, the image conversion unit 14 reads the read source input buffer 12, that is, the head pointer and the address conversion table corresponding to the camera module 2. The image conversion unit 14 can read the image area 202A corresponding to the monitor resolution, replace the display image 203A with the output buffer 16, and display the image on the monitor 3.

  In addition, when the camera image 200 is shifted upward in FIG. 2 due to a physical optical axis shift or the like of the camera module 2, the vehicle image conversion apparatus 1 follows the head pointer for each camera module 2. The image area 202B shifted from the image area 202A is cut out instead of the image area 202A that should be cut out originally, and the display image 203B is displayed instead of the display image 202A that should be output to the monitor 3 originally.

  In contrast, the vehicular image conversion device 1 can correct the head pointer so as to shift it and cut out the image area 202A and display the display image 203A without changing the existing address conversion table. As described above, in the vehicular image conversion apparatus 1, a camera image 200 (640 × 480 pixels) having a size corresponding to the camera imaging area is captured by the camera module 2 and stored in the input buffer 12. The display image 203B shown in FIG. 2 is switched to the display image 203A according to the target deviation.

  In addition, the head pointer corrected for such an optical axis misalignment is set to be used in common regardless of the image layout displayed on the monitor 3, so that the camera module 2 can be used for each image layout presented to the driver. Eliminates the need for image area correction.

  Further, the vehicular image conversion apparatus 1 stores the head pointer of the address conversion table set to correct the optical axis shift of the camera module 2 in a nonvolatile memory. As a result, the stored contents can be retained even when the image conversion apparatus 1 for the vehicle is turned off, thereby eliminating the need to perform the correction of the imaging range of the camera module 2 each time it is used.

[Effect of the first embodiment]
As described above in detail, according to the vehicular image conversion apparatus 1 to which the present invention is applied, an address conversion table having an address space larger than the address space for the monitor resolution is prepared, and a camera image is generated by the address conversion table. By setting the position of the top pointer to be cut out from the camera module 2 so as to correct the physical optical axis shift of the camera module 2, an address conversion table having a huge capacity is displayed to display an image in which the imaging range of the camera module 2 is corrected. There is no need to prepare, and the imaging range can be corrected by simple processing.

  In the vehicular image conversion device 1, a case has been described in which the image for the monitor resolution specified by the head pointer of the address conversion table is cut out from the camera image stored in the input buffer 12 and the image is corrected. For the camera image, a write address conversion process may be performed in which the address conversion table is sequentially referred to and stored in the output buffer 16.

  On the other hand, the address conversion table is provided with an address space of 320 × 240 which is the monitor resolution, and the arbitrary coordinates of the output buffer 16, that is, the monitor 3, and the arbitrary input buffer 12, that is, the arbitrary coordinates of the camera image are obtained. Consider a case where an address conversion table in which a correspondence relationship with coordinate information for reading and replacing is stored is used. This address conversion table is also created based on information such as the specifications, mounting position, and orientation (optical axis) of the camera module 2. When the image conversion unit 14 creates an image to be presented to the driver using the address conversion table, if the camera module 2 is physically displaced from the original position, the camera The imaging range of the module 2 is shifted, and the image presentation range after the address conversion process is also shifted. In order to avoid this, if an address conversion table with different camera position and orientation (optical axis) is prepared for each layout to correct the camera imaging range, a huge number of address conversion tables are required, and the table capacity Will become enormous. In addition, when an address conversion table is generated in real time after taking into account the physical deviation of the camera module 2, the amount of calculation increases and the processing load increases.

  Therefore, the vehicular image conversion apparatus 1 to which the present invention is applied includes an address conversion table for the area expanded in the vertical direction or the horizontal direction and in both directions with respect to the monitor resolution. By cutting out the image corresponding to the monitor resolution in consideration of the shift, the physical shift of the camera module 2 can be absorbed only by the processing of the image conversion unit 14.

  In addition, according to the vehicular image conversion apparatus 1, the camera pointer range for cutting out the address conversion area is stored for each camera module 2 to correct each camera imaging range for each layout presented to the driver. There is no need. Further, by using a non-volatile memory as a memory for storing the head pointer of the address conversion table, the stored contents can be maintained even when the power of the vehicle image conversion device 1 is turned off, and the camera imaging range needs to be corrected at each activation. Disappears.

"Image correction processing based on vehicle information"
Next, in the vehicle image conversion apparatus 1 configured as described above, description will be given of performing image correction based on vehicle information indicating the situation of the host vehicle.

  This image correction process changes the image area cut out from the camera image based on the vehicle information. At this time, the image conversion unit 14 changes the image area to be cut out with reference to the address conversion table by changing the head pointer of the address conversion table.

  Here, the address conversion table stored in the table storage unit 15 includes parameters such as the mounting position and orientation (optical axis) of the camera module 2 in a state where no weight is applied to the host vehicle when no passengers or cargo is loaded. It is created based on. Accordingly, when the vehicle height changes due to some factor such as when many occupants get on the own vehicle or many loads are loaded, the camera imaging range is shifted. That is, if the address conversion area is cut out from the head pointer stored together with the initially set address conversion table, the image displayed on the monitor 3 is shifted.

  On the other hand, the vehicle image conversion apparatus 1 installs a set of sensors for detecting the behavior of the host vehicle, for example, a vehicle height sensor (not shown) in the vicinity of the mounting position of the camera module 2 as vehicle information. The CPU 13 reads a sensor signal from a vehicle height sensor (not shown) and recognizes the current vehicle height of the host vehicle. Then, the CPU 13 obtains a difference between the current height read from the vehicle height sensor and the weight when the vehicle is not loaded, that is, the height at the time of creating the address conversion table, and the camera imaging range is displaced by the number of pixels in the vertical direction. Calculate what you are doing. Then, the image conversion unit 14 moves the address conversion area to be cut out by referring to the address conversion table by moving the head pointer by the displacement pixel amount of the camera imaging range calculated by the CPU 13, thereby shifting the camera imaging range. Correct. The calculation of the height by the CPU 13 and the movement of the head pointer of the address conversion table by the image conversion unit 14 may be performed in real time and are stored in the table storage unit 15 together with the address conversion table and the head pointer. May be.

  As a result, the vehicular image conversion device 1 is a case where a short-term camera imaging range shift occurs due to the behavior of the own vehicle, such as when the position, weight, etc. of the passenger boarding the own vehicle are different. In addition, by automatically correcting the shift of the camera imaging range for each camera module 2 based on the vehicle signal that affects the camera imaging range, an image in the appropriate imaging range can be presented to the driver.

[Second Embodiment]
Next, the vehicle image conversion device 1 according to the second embodiment will be described. In addition, about the part similar to the above-mentioned 1st Embodiment, the detailed description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  The vehicular image conversion apparatus 1 according to the second embodiment corrects a display position when displaying information (overlay data) for assisting vehicle driving by superimposing it with a camera image captured by the camera module 2. It is characterized by this. Examples of the overlay data include a trajectory guideline indicating a backward trajectory superimposed on an image behind the host vehicle when the host vehicle moves backward within a predetermined parking line frame and stops. This trajectory guideline is created by an existing technology based on the steering angle of the host vehicle, for example, by a vehicle driving support device (overlay data creation means) (not shown), and has been converted by the image conversion unit 14 by an image creation device (not shown). Are superimposed on the image and stored in the output buffer 16. That is, the vehicle surrounding display system including the vehicle image conversion device 1 includes a vehicle driving support device that creates overlay data, and an image creation device that superimposes overlay data on image data stored in the output buffer 16. Become.

  Here, the image superimposition position of the trajectory guideline which is overlay data is set based on parameters such as the mounting position and orientation (optical axis) of the camera module 2 in a state where the host vehicle is not heavy. Therefore, when the camera imaging range is deviated due to some factor such as the weight of the host vehicle, the positional relationship between the image converted by the image conversion unit 14 and the vehicle travel locus guideline is deviated, and the locus guideline is obeyed. The inconvenience that it is impossible to reverse is generated, and the reliability as a trajectory guideline is lowered.

  On the other hand, in the vehicle surrounding display system, the vehicle guideline is generated by using the camera image larger than the monitor resolution image captured by the camera module 2 by the vehicle driving support device. As a result, the vehicle surrounding display system uses the address conversion target image 201 shown in FIG. 3A to display overlay data having an area larger than the image area 202 corresponding to the monitor resolution as shown in FIG. An image 300 including the image is created. Similar to the address conversion target image 201 and the address conversion table, the image 300 including the overlay data has a large image area in the vertical direction or the horizontal direction and in both directions as compared with the image for the monitor resolution.

  Then, the vehicle surrounding display system stores the image 300 including the overlay data in a buffer larger than the image area 202 corresponding to the monitor resolution, and the image creation apparatus generates an image corresponding to the monitor resolution from the image 300 including the overlay data. An overlay data superimposed image 302 including overlay data 301 to be superimposed on the region 202 is cut out. The overlay data superimposed image 302 is the same image area as the image area 202 for the monitor resolution, and is an image for the monitor resolution. At this time, the vehicle periphery display system cuts out the overlay data superimposed image 302 with the same start pointer as that used to cut out the image area 202 corresponding to the monitor resolution.

  As a result, the overlay pointer 302 for the overlay data superimposed image 302 is obtained by using the top pointer for cutting out the image area 202 for the monitor resolution as the top pointer of the address conversion table in consideration of the shift of the imaging range of the camera module 2 as described above. The head pointer of the address conversion table to be cut out is matched with the head pointer in consideration of the shift of the imaging range of the camera module 2.

  It should be noted that the address conversion target image 201 and the image 300 including overlay data do not necessarily need to match, and if the amount of displacement from the default value of the head pointer of the address conversion table is known, the same amount of overlay data can be obtained. What is necessary is just to displace.

[Effects of Second Embodiment]
As described above in detail, according to the vehicular image conversion apparatus 1 according to the second embodiment to which the present invention is applied, as with the address conversion target image 201, the image 300 including overlay data is an image corresponding to the monitor resolution. The overlay data superimposed image 302 is cut out by using the one having a larger area and using the same start pointer as the start pointer for cutting out the image area 202 corresponding to the monitor resolution. Minute image area 202 can be superimposed and presented. Therefore, according to the vehicular image conversion apparatus 1, the position of the overlay data superimposed image 302 can be corrected and presented with respect to the shift of the imaging range of the camera module 2, and the overlay data 301 is presented at an accurate position. can do.

  Further, the vehicular image conversion device 1 displays the overlay data superimposed image 302 in conjunction with the movement amount of the image area 202 corresponding to the monitor resolution even when detecting the shift of the image area 202 corresponding to the monitor resolution in real time. By moving and superimposing, overlay data 301 can be presented at a more accurate position.

[Third Embodiment]
Next, the vehicular image conversion apparatus 1 according to the third embodiment will be described. Note that parts similar to those in the above-described embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  The vehicular image conversion apparatus 1 according to the third embodiment has an absolute reference position that is an asymmetrical object such as a vehicle body or a bumper of the own vehicle when the imaging range of the camera module 2 includes a part of the own vehicle. The imaging range of the camera module 2 is corrected based on the overlay data indicating the above.

  As shown in FIG. 4A, the vehicular image conversion apparatus 1 has a camera module 2 attached to the right end portion in the address conversion target image 201 at a position where the vehicle body of the subject vehicle is imaged. Here, when an asymmetrical object such as the own vehicle, that is, a vehicle body or a bumper is included in the camera imaging range, the asymmetrical object such as the vehicle body is determined from the known specification, mounting position, and orientation (optical axis) of the camera module 2. It is known which region of the camera image 200 appears.

  Therefore, the vehicular image conversion apparatus 1 prepares in advance an overlay data image 400 as shown in FIG. 4B showing the contour of the vehicle body appearing at the right end of the image in FIG. The overlay data image 400 only needs to have the same size as the image area 202 corresponding to the monitor resolution in the address conversion target image 201, and is based on a parameter such as a mounting position of the camera module 2 and a reference position such as a vehicle body. Thus, a head pointer for cutting out the address conversion target image 201 is set.

  When correcting the position of the image area 202 corresponding to the monitor resolution, the vehicular image conversion apparatus 1 provided with such an overlay data image 400 has an address in a state where the overlay data image 400 is superimposed on the address conversion target image 201. The head pointer of the overlay data image 400 when the vehicle body in the conversion target image 201 matches the overlay data 401 is acquired. Then, the image area 202 for the monitor resolution is moved based on the overlay data 401 by using the head pointer as the head pointer of the image area 202 for the monitor resolution. At this time, the camera imaging range is corrected by displacing the head pointer of the address conversion table vertically, horizontally, and pixel by pixel from the image in which the address conversion target image 201 and the overlay data image 400 are superimposed.

  Specifically, the image area 202 corresponding to the monitor resolution is matched with the overlay data 401. At this time, the overlay data image 400 and the image area 202 corresponding to the monitor resolution may be matched in accordance with the manual operation of the driver, or may be automatically performed by the vehicle image conversion apparatus 1.

  When manually performed by the driver, the image conversion unit 14 detects a signal from an operation system (not shown), and changes the position of the image area 202 corresponding to the monitor resolution by moving the top pointer of the address conversion table up, down, left, and right. Displace in units. On the other hand, when the automatic conversion is performed by the vehicle image conversion apparatus 1, the image conversion unit 14 recognizes the contour coordinates of the asymmetric object included in the address conversion target image 201 by an image recognition unit (not shown), and the contour coordinates and overlay data are recognized. The head pointer of the address conversion table is displaced vertically, horizontally, and pixel by pixel so as to match 401. The image recognition unit stores color information of an asymmetric object such as a vehicle body that is set in advance, and detects an image portion that matches the color information as contour coordinates of the asymmetric object.

  As a result, the output buffer 16 corrects the imaging range of the camera module 2 by matching the vehicle body in the image area 202 corresponding to the monitor resolution and the overlay data 401 as shown in FIG. Can do.

[Effect of the third embodiment]
As described above in detail, according to the vehicular image conversion apparatus 1 according to the third embodiment to which the present invention is applied, the display screen of the monitor 3 is included in the imaging range of the camera module 2 such as a vehicle body or a bumper. The imaging range of the camera module 2 is moved by moving the image area 202 corresponding to the monitor resolution so that the overlay data 401 indicating the object displayed at the predetermined position matches the object in the address conversion target image 201. It can be used as the head pointer of the address conversion table in which the deviation is corrected. Therefore, according to this vehicular image conversion apparatus 1, the shift of the imaging range of the camera module 2 can be more reliably corrected, and the image area 202 for the monitor resolution that is always a constant imaging range is presented. be able to.

[Fourth Embodiment]
Next, a vehicle image conversion apparatus 1 according to the fourth embodiment will be described. Note that parts similar to those in the above-described embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

[Configuration of Vehicle Image Conversion Device 1]
As shown in FIG. 5, the vehicular image conversion apparatus 1 according to the fourth embodiment includes a table storage unit 31 and an image conversion unit 14 in place of the table storage unit 15 of the vehicular image conversion apparatus 1 shown in FIG. An alternative address conversion unit 34 is provided, and a table area determination unit 32 and a table reconstruction unit 33 are further provided.

  The table storage unit 31 stores one address conversion table for each image layout to be displayed on the monitor 3 and for each camera module 2. Therefore, the address conversion table has an address space larger than the image corresponding to the monitor resolution, which is an image displayed in the image layout displayed on the monitor 3.

  The table area determination unit 32 determines an area for the monitor resolution that is actually used for address conversion from the address conversion table for each camera module 2 of the image layout presented to the user. That is, the table area determination unit 32 reads out an address conversion table corresponding to both the image layout displayed on the monitor 3 and the camera module 2 that captures an image included in the image layout.

  Specifically, when the image layout is such that the image captured by the camera module 2A is displayed on the right half of the monitor 3 and the image captured by the camera module 2B is displayed on the left half of the monitor 3, the table area determining unit 32 Reads the address conversion table for the monitor resolution used for address conversion to display the camera image stored in the input buffer 12A on the right half imaged by the camera module 2A, and the camera image stored in the input buffer 12B An address conversion table corresponding to the monitor resolution used for address conversion to be displayed on the left half imaged by the module 2B is read. Here, an image area to be extracted from an image captured by each camera module 2 is assigned according to the image layout, but the address conversion table read according to the image layout has a resolution higher than the monitor resolution assigned according to the image layout. ing.

  The table reconstruction unit 33 combines an arbitrary number of address conversion tables extracted by the table area determination unit 32 and performs reconfiguration to create an address conversion table including an area (address space) for the monitor resolution. Specifically, an address conversion table having a half area for the monitor resolution for display on the right half of the monitor 3 and an address conversion having a half area for the monitor resolution for display on the left half of the monitor 3 By combining the table, an address conversion table for the monitor resolution corresponding to one screen of the monitor 3 is reconstructed. At this time, the table reconstruction unit 33 extracts an address conversion table corresponding to the monitor resolution allocated according to the image layout from the address conversion table extracted by the table area determination unit 32. Then, the table reconfiguration unit 33 sends the reconfigured address conversion table to the address conversion unit 34 via the internal bus 11.

  The address conversion unit 34 refers to the reconstructed address conversion table sent from the table reconstruction unit 33 and performs a replacement process for replacing the camera image stored in the input buffer 12 with the output buffer 16.

  As shown in FIG. 6, the vehicular image conversion apparatus 1 includes a camera image 501 on the right rear side of the host vehicle captured by the camera module 2A, a camera image 503 directly below the rear of the host vehicle captured by the camera module 2B, A combined image 500 is displayed on the monitor 3 by combining the left and right camera images 505 captured by the camera module 2C (not shown).

  At this time, when the image layout for displaying the right rear side, the left rear side, and the rear right below of the host vehicle is determined, the vehicular image conversion apparatus 1 corresponds to the image layout, and the right rear side, the left The table area determination unit 32 reads out the address conversion tables 31a, 31b, and 31c corresponding to the camera module 2 having the imaging direction in the rear side and directly below the rear. At this time, each of the address conversion tables 31 a, 31 b, and 31 c is an address space that is larger than the image for the monitor resolution, that is, the resolution for the image layout including the three images constituting the composite image 500.

  Next, the table reconstruction unit 33 monitors the monitor resolution corresponding to the resolution necessary for creating the composite image 500 among the camera images 501, 503, and 505 captured by each camera module 2 from the determined image layout. Minute image areas 502, 504, and 506 are determined. This is synonymous with displacing the head pointer of the address conversion table, and the address conversion tables 31a ′, 31b ′, and 31c ′ having the address spaces of the image areas 502, 504, and 506 corresponding to the determined monitor resolution are created. . In other words, the table reconstructing unit 33 extracts the address conversion tables 31a, 31b, and 31c larger than the monitor resolution in accordance with the image layout in order to cut out the image areas 502, 504, and 506 corresponding to the monitor resolution. Minute address conversion tables 31a ′, 31b ′, and 31c ′. Thereby, the area of the address conversion table used for creating the composite image 500 is determined.

  Next, the table reconstruction unit 33 reconstructs the address conversion table for the composite image 500 by combining the address conversion tables 31a ′, 31b ′, and 31c ′ for the monitor resolution allocated according to the image layout according to the image layout. To do. Thereby, the address conversion table 507 that is referred to when the address conversion unit 34 actually performs the address conversion process is created.

  Next, the address conversion unit 34 refers to the address conversion table 507, cuts out the image 502 for the monitor resolution from the camera image 501, cuts out the image 504 for the monitor resolution from the camera image 503, and monitors the monitor resolution from the camera image 505. Image 506 is cut out, and the cut-out image for the monitor resolution is replaced with the output buffer 16 with reference to the address conversion table 507 to create a composite image 500.

  It should be noted that not only when a composite image 500 is created from camera images 501, 503, and 505 captured by a plurality of camera modules 2, but a plurality of images are cut out from the camera images captured by a single camera module 2 and combined. An image 500 may be created.

[Image correction processing]
Next, in the vehicular image conversion apparatus 1 that reconstructs the address conversion table as described above, a process for individually correcting the image area to be presented in the image for each camera module 2 will be described with reference to the flowchart of FIG. .

  This vehicular image conversion apparatus 1 has a mode for presenting a vehicle surrounding image to the driver and a mode for correcting the image layout presented to the driver (hereinafter referred to as calibration mode). In step S1, the CPU 13 If it is determined that the calibration mode is selected, the process proceeds to step S2.

  Next, in step S2, the CPU 13 determines the camera module 2 to be corrected. At this time, the CPU 13 determines a signal to be corrected by the driver by determining a signal from an operation system (not shown), and determines the camera module 2 that is capturing the image.

  Next, in step S3, the CPU 13 extracts an address conversion table corresponding to the image layout to be presented to the driver for each camera module 2, and temporarily stores it in a memory (not shown). At this time, the CPU 13 extracts an address conversion table having a resolution higher than that for the monitor resolution with respect to the address conversion table for the camera module 2 determined to be the correction target in step S2. In this address conversion table, the head pointer is set as a default value.

  Next, in step S4, the table area determination unit 32 acquires the head pointer of the default address conversion table set for each camera module 2. Since there are cases where a plurality of image layouts are prepared for an image captured by one camera module 2, each camera layout is corrected for each image layout by holding this head pointer for each camera module 2. There is no need.

  Next, in step S5, the table reconstruction unit 33 refers to the head pointer for each camera module 2 acquired in step S4, and from the head pointer of the address conversion table for each image layout, the monitor resolution of each camera module 2. Get the address space of. Then, the table reconstruction unit 33 reconstructs an address conversion table for one screen used for actual address conversion by combining the address conversion tables for each of the plurality of camera modules 2.

  Next, in step S6, the address conversion unit 34 refers to the address conversion table reconstructed in step S5, and after the address conversion process of replacing the pixels of the input buffer 12 with the output buffer 16, outputs from the output buffer 16 to the monitor 3. Then present it to the driver.

  Next, when the calibration mode is not canceled by the driver (step S7) and the CPU 13 detects an operation related to calibration, for example, an operation for shifting the image area of a certain camera module 2 leftward (step S8). Then, the head pointer of the address conversion table of the camera module 2 to be calibrated is displaced vertically and horizontally in accordance with the operation contents (step S9).

  Then, by performing the processing of step S5 and step S6, an image corresponding to the operation related to calibration is presented, and the processing is repeated until the calibration mode is canceled.

[Effect of Fourth Embodiment]
As described above in detail, according to the vehicular image conversion apparatus 1 according to the fourth embodiment to which the present invention is applied, depending on the image layout presented to the driver, each camera module 2 has a vertical direction or a horizontal direction. And an address conversion table equal to or higher than the monitor resolution in both directions, of the address conversion table for each camera module 2, an area for the monitor display area allocated according to the image layout is determined and used to address Reconfigure the address conversion table for one screen used for conversion. Thereby, even when correcting the shift of each image included in the image layout, it is not necessary to prepare an address conversion table corresponding to the level of the shift, and a reduction in memory capacity can be realized.

  In addition, according to the vehicular image conversion apparatus 1, the physical shift of the camera module 2 is corrected based on the address-converted image with reference to the reconfigured monitor conversion address conversion table. It is possible to correct the shift of the imaging range for each of the plurality of camera modules 2 automatically and manually.

  Furthermore, according to the vehicular image conversion apparatus 1, the monitor display area for each camera module 2 included in the image layout can be individually corrected, so that the memory capacity can be reduced and the address conversion table generation process by the CPU can be performed. Can be approximated. In addition, it is possible to provide an imaging range that matches the user's preference.

[Fifth Embodiment]
Next, a vehicle image conversion device 1 according to a fifth embodiment will be described. Note that parts similar to those in the above-described embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  The vehicular image conversion apparatus 1 according to the fifth embodiment is characterized in that the image layout is automatically corrected by combining the images taken by the plurality of camera modules 2 without interruption.

  As shown in FIG. 8A, the vehicle image conversion apparatus 1 includes a camera module 2A and a camera module 2B installed at the rear part of the host vehicle, and an imaging range 2a of the camera module 2A and an imaging range 2b of the camera module 2B. The imaging ranges of the camera module 2A and the camera module 2B are set so as to overlap.

  Then, as shown in FIGS. 8B and 8C, the vehicle image conversion apparatus 1 has an image layout in which the camera image 601A of the camera module 2A and the camera image 601B of the camera module 2B are displayed side by side. Then, the address conversion table for the camera module 2A is read, and the address conversion table for the camera module 2B is read. Then, the vehicle image conversion apparatus 1 cuts out the address conversion table for the camera module 2A and the address conversion table for the camera module 2B so that the camera image of the camera module 2A and the camera image of the camera module 2B are continuous, and addresses Reconstruct the translation table.

  As a result, when the camera module 2A and the camera module 2B have no optical axis shift or mounting position shift, as shown in FIG. 8B, the camera image 601A and the camera image 601B are continuous via the partition line. It becomes an image. On the other hand, when the camera module 2A or the camera module 2B has an optical axis shift or a mounting position shift, the camera image 601A and the camera image 601B are continuous across the partition lines as shown in FIG. 8C. End up with no image.

  When the image shift occurs at the joint between the camera image 601A and the camera image 601B, the vehicle image conversion device 1 captures an object whose arrangement of feature points is known, and the image after address conversion. The displacement of the imaging range of the camera module 2 is corrected by displacing the use area of the address conversion table using the object included in the.

  The processing for correcting the shift of the imaging range of the camera module 2 will be described assuming that the following conditions are assumed for the sake of simplicity.

  The monitor output resolution is 16 × 16 pixels.

  ・ Assuming correction under the situation shown in FIG. Image clipping processing is performed by referring to the address conversion table from the images captured by the camera module 2A and the camera module 2B, and is assigned to the left and right, horizontal (horizontal) 8 × vertical (vertical) 16 pixel areas of the monitor, respectively. (See FIG. 9).

  The pixel information output from the camera is in RGB format (24 bits). A YCbCr format or the like may be used.

  The information size for the surroundings of the vehicle per pixel captured by a plurality of cameras is assumed to be equal. In the case of FIG. 8, the camera module 2A and the camera module 2B have the same specs, are symmetrically mounted, and have the same orientation (optical axis), but the camera modules 2A and the camera modules 2B have different specs. Even when the mounting position, orientation (optical axis), and the like are different, it is only necessary that the information size per pixel in the monitor image area allocated to the monitor 3 from each camera imaging range is equal.

  A known object is assumed to be a diagonally orthogonal pattern (see FIG. 9), and color information that does not exist in the imaging environment is used as the color information of the pattern. In this example, the color information of the object is black (0x000000), and the color information around the black part is white (0xFFFFFF). It is only necessary that the pixel information can be discriminated. The pattern of the asymmetric object is not limited to the orthogonal pattern.

  When such conditions are provided, as shown in FIG. 9, when there is no optical axis shift, a symmetrical pattern is formed between the image 701A captured by the camera module 2A and the image 701B captured by the camera module 2B. Is detected (FIG. 9A). On the other hand, when there is an optical axis shift, as shown in FIGS. 9B, 9C, and 9D, between the image 701A captured by the camera module 2A and the image 701B captured by the camera module 2B. The symmetry of the pattern is lost and the continuity is lost. Therefore, the vehicular image conversion apparatus 1 corrects the image shift at the joint as shown in FIG. 8C by detecting such a change in the pattern of the object.

  In this process, as shown in FIG. 10, the object is imaged by another camera module 2 different from the camera module 2A and the camera module 2B, and the feature pattern of the object is stored. This feature pattern is a feature pattern when an image is taken and output to the monitor 3 in a situation where there is no physical deviation between the camera module 2A and the camera module 2B. Here, the object for acquiring the feature pattern is an asymmetric object installed on the physical central axis between the camera module 2A and the camera module 2B, and is a camera module different from the camera module 2A and the camera module 2B. Suppose that it exists in 2 imaging range.

  First, in step S11, the vehicular image conversion apparatus 1 determines whether or not it is in the calibration mode. If it is in the calibration mode, in step S12, the table area determination unit 32 causes the camera module 2A and camera module 2B to function. The head pointer of each default address conversion table is acquired.

  Next, in step S13, the table reconfiguration unit 33 acquires the address conversion table for the monitor resolution from the head pointer acquired in step S12, and reconfigures the address conversion areas of the camera module 2A and the camera module 2B. An address conversion table for the monitor output resolution used for actual address conversion is reconstructed.

  Next, in step S14, the address conversion unit 34 refers to the address conversion table for the monitor resolution reconstructed in step S13, and performs address conversion to replace the pixel information in the input buffer 12 with the output buffer 16.

  Next, in step S15, the CPU 13 extracts the color information (0x000000) of the symmetric object (feature pattern) from the pixel information stored in the output buffer 16 and the coordinates of the pixel where the color information is detected.

  Next, in step S16, the CPU 13 compares the coordinates of the pixels constituting the feature pattern detected in step S15 with the coordinates of the pixels of the feature pattern acquired before starting the processing from step S11 onward. It is determined whether or not. If it is determined that they match, the camera module 2A and the camera module 2B determine that there is no physical deviation, the process ends, and if it is determined that they do not match, The shift direction and shift pixel amount of the feature pattern are acquired.

  Next, in step S17, the CPU 13 sets one camera module 2 of the camera module 2A and the camera module 2B as a calibration target from the deviation direction and the deviation pixel amount of the feature pattern acquired in step S16. Here, the camera module 2 that has not been subjected to calibration is set as a relative reference for correction.

  At this time, the CPU 13 may select the camera module 2 with the larger amount of deviation of the feature pattern acquired in advance as the camera module 2 to be calibrated, or may be one camera module 2 set in advance. At this time, the image data converted in the address in step S14 and stored in the output buffer 16 is separately stored in a memory (not shown).

  Next, in step S18, the CPU 13 extracts an address conversion table corresponding to the image layout presented to the driver for each camera module 2, and temporarily stores it in a memory (not shown). At this time, the CPU 13 extracts an address conversion table having a resolution larger than that for the monitor resolution with respect to the address conversion table for the camera module 2 determined to be the calibration target in step S17. At this time, the head pointer is set as a default value in the address conversion table for the camera module 2 that is not subject to calibration.

  Next, in step S <b> 19, the CPU 13 sets the feature pattern stored in advance and the feature pattern included in the image captured by the camera module 2 to be calibrated as a correction target. A displacement amount of a feature pattern included in an image captured by a certain camera module 2 is obtained in units of pixels. Such a displacement amount is a displacement amount of the head pointer of the address conversion table for the camera module 2 to be corrected, and is stored in a memory (not shown), and the process is terminated (step S20). In the subsequent address conversion, the address conversion table is used by using the displaced leading pointer.

  In the above-described embodiment, the case where the image joint between the camera module 2A and the camera module 2B is corrected has been described. However, the present invention is not limited to this, and the present invention can also be applied to correction of two or more camera images.

[Effect of Fifth Embodiment]
As described above in detail, according to the vehicular image conversion apparatus 1 according to the fifth embodiment to which the present invention is applied, the head pointer for determining the use area of the address conversion table corresponding to the camera module 2 to be corrected is set. By determining with reference to the image after address conversion of another camera module 2, the imaging range for each camera module 2 can be corrected using a relative standard. Specifically, for example, since an image of a vehicle body, a bumper or the like, which is an absolute reference, is not used for correcting the imaging range, it is not necessary to provide a correction processing mechanism for each vehicle type.

  Further, according to the vehicular image conversion apparatus 1, when the vehicle periphery is presented in an image layout in which images captured by a plurality of camera modules 2 are combined, a previously acquired feature pattern and a plurality of camera modules 2 capture images. By comparing the image with the feature pattern after address conversion of the image, detecting a shift in the imaging range of the camera module 2, and automatically displacing the use area of the address conversion table based on the shift in the imaging range. The imaging range of the camera module 2 can be corrected.

  Further, according to the vehicle image conversion device 1, each camera module 2 is based on the relative position between the feature pattern imaged by the camera module 2 to be calibrated and the feature pattern imaged by a separate camera. The shift of the imaging range can be corrected. For example, image information such as a vehicle body and a bumper, which are absolute standards, is not used for correcting the shift of the imaging range, so that it is not necessary to provide a correction processing function for each vehicle type.

  The above-described embodiment is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and various modifications can be made depending on the design and the like as long as the technical idea according to the present invention is not deviated from this embodiment. Of course, it is possible to change.

  That is, the above-described vehicular image conversion apparatus 1 is not limited to a camera image cutout process, but also an enlargement / reduction process for manipulating an image presentation area, a distortion conversion process considering lens distortion, and a viewpoint for conversion into an image from a virtual viewpoint. The present invention can also be applied to an address conversion table corresponding to conversion processing or the like.

It is a block diagram which shows the structure of the vehicle periphery display system which concerns on 1st Embodiment to which this invention is applied. It is a figure for demonstrating the process which displays the display image of a different position from the address conversion object image in the vehicle periphery display system which concerns on 1st Embodiment to which this invention is applied. It is a figure for demonstrating the process which moves and displays the overlay data superimposed on the image area | region for monitor resolution in the vehicle periphery display system which concerns on 2nd Embodiment to which this invention is applied. In the vehicle periphery display system which concerns on 3rd Embodiment to which this invention is applied, it is a figure for demonstrating the process which moves the image area for monitor resolution with reference to the overlay data previously displayed on the predetermined position of a monitor. is there. It is a block diagram which shows the structure of the vehicle periphery display system which concerns on 4th Embodiment to which this invention is applied. It is a figure for demonstrating the process which reconfigure | reconstructs an address conversion table and displays the image according to an image layout in the vehicle periphery display system which concerns on 4th Embodiment to which this invention is applied. It is a flowchart which shows the process sequence of the process which reconfigure | reconstructs an address conversion table in the vehicle periphery display system which concerns on 4th Embodiment to which this invention is applied. It is a figure for demonstrating the process for displaying the image of a several camera module continuously in the vehicle periphery display system which concerns on 5th Embodiment to which this invention is applied. In the vehicle periphery display system which concerns on 5th Embodiment to which this invention is applied, it is a figure for demonstrating the shift | offset | difference of the image according to the presence or absence of the optical axis offset at the time of imaging the same target object with a some camera module. It is a flowchart which shows the process sequence of the process which correct | amends the optical axis offset of several camera modules according to the characteristic of a target object in the vehicle periphery display system which concerns on 5th Embodiment to which this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image converter for vehicles 2 Camera module 3 Monitor 11 Internal bus 12 Input buffer 13 CPU
14 Image Converter 15 Table Storage 16 Output Buffer 21 Imaging Lens 22 CCD
31 Table storage unit 32 Table area determination unit 33 Table reconstruction unit 34 Address conversion unit

Claims (12)

An image correction apparatus that is connected to an imaging unit that captures an image of surroundings and generates image data, and converts the image data into display image data that is displayed on a display unit,
Input means for inputting input image data imaged by the imaging means;
Address conversion describing address information of the input image data having an address space with resolution larger in the vertical direction or the horizontal direction than the resolution of the display means, and address information of display image data corresponding to the resolution of the display means Storage means for storing the table;
Image processing means for extracting image data corresponding to the resolution of the display means from the input image data with reference to the address conversion table, and converting address information of the extracted image data into address information of the display image data When,
An image correction apparatus comprising: output means for outputting image data cut out by the image processing means and converted in address information to the display means as display image data.   The image processing means sets a head pointer of an address conversion table for cutting out image data corresponding to the resolution of the display means for each image pickup means, and cuts out image data from the head pointer of the address conversion table. The image correction apparatus according to claim 1, wherein the image correction apparatus is an image correction apparatus. The imaging means generates an image data by imaging a surrounding of a predetermined host vehicle,
The image processing unit inputs a vehicle signal indicating the behavior of the host vehicle, cuts out from the input image data according to the vehicle signal, moves image data corresponding to the resolution of the display unit, and moves around the predetermined host vehicle. The image correction apparatus according to claim 1, wherein image data including the image data is cut out. Overlay data creating means for creating overlay data that is superimposed on the image data cut out by the image processing means and has an address space whose resolution is larger in the vertical or horizontal direction than the resolution of the display means. Prepared,
The image processing means cuts out overlay data to be superimposed on image data corresponding to the resolution of the display means cut out from the input image data from the overlay data created by the overlay data creating means. The image correction apparatus according to 1.   The image processing means stores in advance overlay data to be displayed at a predetermined position on the display screen of the display means, a predetermined position for displaying the overlay data, and a position of an object included in the input image data. The image correction apparatus according to claim 1, wherein a position where image data corresponding to the resolution of the display unit is cut out is moved so as to match.   The image correction apparatus according to claim 5, wherein the overlay data to be displayed at a predetermined position on the display screen of the display means is a part of the host vehicle. The imaging means includes a plurality of imaging ranges different from each other,
The storage means combines the image data picked up by a plurality of image pickup means in the vertical direction or the horizontal direction with respect to the resolution of the display image data for each of the image pickup means assigned according to the image layout to be displayed on the display means. An address conversion table having a large resolution address space,
The image processing means includes
Table area determining means for extracting a plurality of address conversion tables for each of the imaging means according to an image layout to be displayed on the display means;
For each address conversion table extracted by the table area determining means, an area for extracting image data corresponding to the resolution of the display image data allocated according to the image layout is determined, and a plurality of areas in which the area is determined Table reconstructing means for reconfiguring an address translation table having an address space corresponding to the resolution of the display means in combination with an address translation table;
With reference to the address conversion table reconstructed by the table reconstructing means, image data for the resolution assigned according to the image layout is cut out for each input image data imaged by each imaging means, The image correction apparatus according to claim 1.   The table reconstructing means includes a head pointer of an address conversion table corresponding to the image pickup means to be used for correcting input image data picked up by the image pickup means among the address conversion tables extracted by the table area determining means. The image correction apparatus according to claim 7, wherein a region of an address conversion table for cutting out image data for a resolution assigned in accordance with an image layout is moved.   8. The image correction according to claim 7, wherein the storage unit stores, for each of the imaging units, a head pointer of an address conversion table for extracting image data having a resolution assigned according to an image layout. apparatus.   The table reconstructing means moves each head pointer of an address conversion table for each of the imaging means, and cuts out image data for a resolution assigned according to an image layout for each of the imaging means. The image correction device according to claim 8 or 9.   The table reconstruction means takes out image data for the resolution assigned according to the image layout from the input image data picked up by one of the image pickup means, converts it into display image data, and refers to the display image data The image correction apparatus according to claim 7, wherein each head pointer of the address conversion table of the other imaging unit is moved to cut out image data for a resolution assigned in accordance with the image layout. The image processing means acquires features of an object included in an imaging range of a plurality of imaging means, and performs address conversion for cutting out image data for a resolution assigned according to an image layout so that the features are continuous. 8. The image data for a resolution assigned in accordance with an image layout is cut out from a plurality of input image data picked up by a plurality of image pickup means by moving a head pointer of the table. Image correction device.