JP2015065579A - Picture data conversion device and operation support device, navigation device, and camera device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a picture data conversion device capable of avoiding deterioration in picture quality in creating an overhead picture.SOLUTION: A coordinate conversion unit 35 extracts a pixel in a range corresponding to an overhead picture of the ground, by performing calculation processing on imaging data for identifying pixel information for each pixel with respect to an imaged picture of a subject projected from a fisheye lens 21. A buffer storage unit 29 is connected to the coordinate conversion unit 35. The buffer storage unit 29 has capacity smaller than an information amount of information on all pixels of the imaged picture, and stores pixel information on the extracted pixel. The coordinate conversion unit 35 extracts a pixel in a range corresponding to an overhead picture of the ground of all pixels of the imaged picture. An information amount on pixel information stored in the buffer storage unit 29 is reduced.

Description

  The present invention relates to an image data conversion apparatus that stores pixel information relating to a captured image of a subject projected from a fisheye lens, performs image conversion, and outputs the result to a monitor.

  A driving support device that displays an overhead image behind the vehicle body is generally known. In the driving support device, for example, a subject is projected onto an image sensor from a fisheye lens. Therefore, the further away from the fisheye lens, the smaller the image. An overhead image is generated from such a captured image. The distant image is magnified. The farther away, the greater the magnification of the image. Significant image quality degradation is observed.

JP 2001-145012 A

  At present, a display having a resolution of about WVGA is used for the display of the driving assistance device. An image sensor having the same resolution is used. Therefore, when the captured image is enlarged twice or three times when generating the overhead view image, the pixel information can be acquired only by the number of pixels of 1/2 or 1/3 of the number of pixels of the display display. I can't. The image quality will deteriorate. In particular, the image quality deteriorates noticeably because the image quality deteriorates partly within one display screen.

  According to some aspects of the present invention, it is possible to provide an image data conversion apparatus that can avoid deterioration in image quality when generating an overhead image.

  One aspect of the present invention is a coordinate conversion unit that extracts pixel information for each pixel in a range corresponding to an overhead image from imaging data that specifies pixel information for each pixel with respect to a captured image of a subject projected from a fisheye lens. The present invention relates to an image data conversion device including a buffer storage unit that has a capacity smaller than the information amount of information about all pixels of the captured image and stores pixel information about the extracted pixels.

  Imaging data of the captured image is processed by the coordinate conversion unit prior to storage in the buffer storage unit. In the coordinate conversion unit, pixels are extracted in a range corresponding to the bird's-eye view image of the ground among all the pixels of the captured image. Thus, the information amount of the pixel information is reduced. Although the buffer storage unit has a capacity smaller than the information amount of information regarding all the pixels of the captured image, the overhead image can be generated reliably. Even if the resolution of the captured image is increased, an increase in the capacity of the buffer storage unit can be avoided. If the resolution of the picked-up image is increased, even if the picked-up image is partially enlarged when generating the overhead view image, deterioration of the image quality can be largely avoided.

  The image data conversion apparatus includes a storage unit, and the coordinate conversion unit may extract pixel information for each pixel in a range corresponding to the overhead image based on a mathematical formula stored in the storage unit.

  The storage unit includes a first equation for rotating the captured image around a horizontal axis in a virtual three-dimensional space to generate an angle-converted image, and the angle-converted image on a two-dimensional plane including the captured image. And a second expression for generating a projected image, and a range corresponding to the overhead image is cut out from the projected image, and the cut-out projected image is transformed according to the number of pixels of the display screen to generate a buffer image The third equation can be stored.

  The image data conversion device generates a first restored image from the buffer image in a range cut from the projected image, and reversely rotates the first restored image around the horizontal axis in the virtual three-dimensional space. A restored image is generated, the second restored image is projected onto the two-dimensional plane including the captured image in the virtual three-dimensional space to generate a third restored image, and an image caused by the lens in the third restored image It is possible to provide an overhead image processing unit that performs a distortion correction process for correcting the distortion, generates an original image before the overhead view, and generates the overhead image based on the first expression and the second expression.

  In general, in a captured image of a subject projected from a fisheye lens, the subject is distorted as the distance from the optical axis of the fisheye lens increases. By simply performing linear coordinate transformation on the captured image, distortion of the subject remains in the overhead image. The distortion correction corrects the distortion of the subject. As a result, the appearance of the overhead view image is improved.

  The overhead image processing unit can generate the overhead image with a smaller number of pixels than the number of pixels of the captured image. A display having a resolution lower than that of the image sensor can be used. In such a case, even if the captured image is enlarged when generating the overhead view image, the lack of pixel information can be compensated for when the image is enlarged. As a result, deterioration of image quality on the display can be avoided.

  The image data conversion device as described above can be incorporated in the driving support device. At this time, the driving support device includes an image data conversion device. Such a driving support device can be used by being mounted on a vehicle, for example.

  In addition, the image data conversion device as described above can be incorporated in a navigation device. At this time, the navigation device includes an image data conversion device. Such a navigation device can be used by being mounted on a vehicle, for example.

  In addition, the image data conversion apparatus as described above can be incorporated in the camera apparatus. At this time, the camera device includes a data conversion device. Such a camera device can be used by being mounted on a vehicle.

  As described above, according to the embodiment of the present invention, it is possible to avoid deterioration in image quality when generating an overhead image.

1 is a perspective view schematically showing an overall image of a vehicle, that is, an automobile according to an embodiment. It is a block diagram which shows the structure of a driving assistance device roughly. It is a figure which shows roughly the concept of 1st image data, 2nd image data, and 3rd image data. It is a figure which shows the concept of 4th image data roughly. It is a photograph which shows one specific example of the captured image of the subject projected from the fisheye lens. It is a figure which shows the concept of the image conversion of a three-dimensional angle conversion part. It is a photograph which shows a specific example of a bird's-eye view image. It is a photograph of the original image which shows roughly the magnification of the bird's-eye view image concerning the 1st comparative example. It is a photograph of the original image which shows roughly the magnification of the bird's-eye view image concerning the 2nd comparative example. It is a photograph which shows the bird's-eye view image which concerns on another specific example.

  Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

  FIG. 1 schematically shows a vehicle, that is, an automobile 11 according to an embodiment. A driving support device 13 is mounted on the vehicle body 12 of the automobile 11. The driving support device 13 includes a navigation device 14. The navigation device 14 has a display panel 15. The display panel 15 has a resolution of 640 × 480, for example. The display panel 15 can be incorporated into a dashboard, for example. A map and other information can be displayed on the display screen of the display panel 15.

  The driving support device 13 includes a camera device 16. The camera device 16 is installed at the rear portion of the vehicle body 12, for example. The camera device 16 is attached above the rear bumper, for example. The optical axis 17 of the camera device 16 is set at a predetermined angle with respect to the horizontal plane, for example. The optical axis 17 is inclined so as to approach the ground on which the automobile 11 is placed as it moves backward from the vehicle body 12. The camera device 16 images the state behind the vehicle body 12. Here, a captured image of the subject is acquired in an angle of view range of at least ± 90 ° in both the horizontal and vertical directions with respect to the optical axis 17.

  The camera device 16 generates an image signal of the display image (hereinafter referred to as “display image signal”) based on the captured image. The display image signal specifies an image of the camera viewpoint and an overhead view image of the ground with respect to the rear of the vehicle body. The image or the overhead image may be switched arbitrarily. The camera device 16 is connected to the navigation device 14. A display image signal is supplied from the camera device 16 to the navigation device 14. On the display screen of the display panel 15, an image behind the vehicle body 12 can be displayed based on the display image signal. In the driving support device 13, for example, the driver can check the state behind the vehicle body 12 when entering the parking space backward in a parking lot. In this way, the driving support device 13 can support the driver's backward confirmation.

  As shown in FIG. 2, the camera device 16 includes a fisheye lens 21. The fish-eye lens 21 defines, for example, an incident surface of a subject with a hemispherical surface having an equator plane orthogonal to the optical axis 17. As a result, the light reflected from the subject is incident on the optical axis 17 in the horizontal and vertical directions in an angle of view range of at least ± 90 °. The incident light forms an image of the subject on the equator plane. The subject image is partitioned by a circular outline. In the image, the subject shrinks away from the center of the circle.

  The camera device 16 includes an image sensor 22. The image sensor 22 converts an image of a subject projected from the fisheye lens 21 into an electric signal. The image sensor 22 includes pixels arranged in a matrix. Here, the image sensor 22 has a resolution of 1280 × 960. Therefore, the resolution of the display panel 15 is lower than the resolution of the image sensor 22. The position of each pixel is specified according to a two-dimensional coordinate system set on the imaging surface of the image sensor 22. For example, in the Bayer array, the color of the subject is expressed by an electrical signal of RGGB (red green green blue).

  The camera device 16 includes an image processing unit 23. The image processing unit 23 is electrically connected to the image sensor 22. The image processing unit 23 reads out the electrical signals of the pixels while sequentially specifying the pixel positions, that is, addresses. The image processing unit 23 outputs a digital imaging signal, that is, imaging data. The imaging data specifies image information for each pixel.

  An image data conversion device 24 is incorporated in the camera device 16. The image data conversion device 24 receives imaging data from the image processing unit 23. The image data conversion device 24 includes an image generation unit 25 and an overhead image generation unit 26. The image generation unit 25 generates a normal image at the camera viewpoint based on the imaging data. The overhead image generation unit 26 generates an overhead image based on the imaging data. In the image data converter 24, processing for generating an overhead image and a distortion-corrected image is performed by image conversion.

  The navigation device 14 includes an arithmetic processing unit 27. The arithmetic processing unit 27 is electrically connected to the image data conversion device 24. The arithmetic processing unit 27 can have, for example, an MPU (microprocessor unit) and a memory. For example, the MPU can operate according to software stored in the memory. The arithmetic processing unit 27 can acquire image information after coordinate conversion from the camera device 16 in accordance with processing executed by software. The arithmetic processing unit 27 can switch between the navigation operation and the driving support operation of the navigation device 14. For example, when the gear of the automobile 11 is put in the back, the arithmetic processing unit 27 can switch the navigation device 14 from the navigation operation to the driving support operation. For example, according to the operation of the navigation device 14, the arithmetic processing unit 27 can select supply of one or both of the normal image and the overhead image.

  The display panel 15 is electrically connected to the arithmetic processing unit 27. Image information is supplied from the arithmetic processing unit 27 to the display panel 15. Depending on the supplied image information, a map or an image behind the vehicle body 12 can be displayed on the display screen of the display panel 15.

  The overhead image generation unit 26 is connected to the storage unit 28. The storage unit 28 includes a buffer storage unit 29. A first buffer area 31 and a second buffer area 32 are formed in the buffer storage unit 29. Input image data is stored in the first buffer area 31 and the second buffer area 32 of the buffer storage unit 29. However, it is not necessary to store all the pixels of the input image in each of the first buffer area 31 and the second buffer area 32 of the buffer storage unit 29, and only the image portion necessary for generating the overhead image may be stored. The first buffer area 31 and the second buffer area 32 are switched for each frame and store image information. The first buffer area 31 has a capacity corresponding to the information amount of pixel information regarding all the pixels of the display panel 15. The second buffer area 32 has the same capacity as the first buffer area 31. The pixel information is stored in the first buffer area 31 for each pixel in the determined input order. When the pixel information is stored in the first buffer area 31 for all the pixels of the display panel 15, the pixel information starts to be output from the first buffer area 31. The reception of the pixel information is switched to the second buffer area 32. The pixel information is stored in the second buffer area 32 for each pixel in the determined input order. When the pixel information is stored in the second buffer area 32 for all the pixels of the display panel 15, the pixel information starts to be output from the second buffer area 32. The reception of the pixel information is switched to the first buffer area 31 again.

The storage unit 28 further includes a mathematical expression storage unit 33 and a lookup table storage unit 34. The formula storage unit 33 stores the first to eighth formulas. In the first to eighth formulas, the following formulas are set according to the three-dimensional coordinates as will be described later. Here, the coordinate values (x, y, z) are specified according to the three-dimensional coordinate system of the virtual three-dimensional space.
A lookup table 37 is stored in the lookup table storage unit 34. The look-up table 37 specifies coordinate transformation according to a three-dimensional coordinate system. The coordinate position is converted for each pixel in the virtual three-dimensional space.

  The overhead image generation unit 26 further includes a coordinate conversion unit 35 and an overhead image processing unit 36. The coordinate conversion unit 35 is connected to the image processing unit 23. The coordinate conversion unit 35 extracts pixels in a range corresponding to the overhead image from the imaging data. A buffer storage unit 29 is connected to the coordinate conversion unit 35. Pixel information regarding the extracted pixels is sequentially stored in the buffer storage unit 29. An overhead image processing unit 36 is connected to the buffer storage unit 29. The overhead image processing unit 36 extracts pixel information from the buffer storage unit 29. The overhead image processing unit 36 performs a distortion correction process for correcting the distortion of the image caused by the lens on the pixel information and an overhead image conversion process for converting the image into an image looking down from directly above to generate an overhead image.

  The coordinate conversion unit 35 includes a three-dimensional angle conversion unit 41, a cut processing unit 42, and a reduction processing unit 43. The three-dimensional angle conversion unit 41 is connected to the image processing unit 23. The three-dimensional angle conversion unit 41 performs arithmetic processing on the imaging data based on the first and second expressions. The first formula and the second formula are acquired from the formula storage unit 33. The three-dimensional angle conversion unit 41 generates the first image data. The cut processing unit 42 is connected to the three-dimensional angle conversion unit 41. The cut processing unit 42 performs arithmetic processing on the first image data based on the third equation. The third formula is acquired from the formula storage unit 33. The second image data is generated by the cut processing unit 42. The reduction processing unit 43 is connected to the cut processing unit 42. The reduction processing unit 43 performs arithmetic processing on the second image data based on the fourth equation. The fourth equation is acquired from the equation storage unit 33. The reduction processing unit 43 generates the third image data. The third image data is stored in the buffer storage unit 29. The concept of the first, second and third image data will be described later.

  The overhead image processing unit 36 includes an inverse three-dimensional angle conversion unit 45, a distortion correction unit 46, and a three-dimensional angle conversion unit 47. The inverse three-dimensional angle conversion unit 45 is connected to the buffer storage unit 29. The inverse three-dimensional angle conversion unit 45 performs arithmetic processing on the third image data based on the fifth formula, the sixth formula, the seventh formula, and the eighth formula. The fifth formula, the sixth formula, the seventh formula, and the eighth formula are acquired from the formula storage unit 33. The fourth image data is generated by the inverse three-dimensional angle conversion unit 45. The distortion correction unit 46 is connected to the inverse three-dimensional angle conversion unit 45. The distortion correction unit 46 performs coordinate conversion on the fourth image data based on the lookup table 37. The distortion correction unit 46 is connected to the lookup table 37. The distortion correction unit 46 refers to the lookup table storage unit 34 for coordinate conversion. The fifth image data is generated by the distortion correction unit 46. The three-dimensional angle conversion unit 47 is connected to the distortion correction unit 46. The three-dimensional angle conversion unit 47 performs arithmetic processing on the fifth image data based on the first and second expressions. The first formula and the second formula are acquired from the formula storage unit 33. Sixth image data is generated by the three-dimensional angle converter 47. The sixth image data specifies an overhead image.

  Here, the concept of the first to sixth image data will be described. As shown in FIG. 3A, a virtual three-dimensional space is set on the two-dimensional surface of the imaging surface 51. The xy plane of the virtual three-dimensional space is superimposed on the two-dimensional surface of the imaging surface 51. On the imaging surface 51, the pixel position is specified in the row direction (horizontal direction) by the x coordinate, and the pixel position is specified in the column direction (vertical direction) by the y coordinate. A rotation center 52 is set at a distance z from the xy plane.

  As shown in FIG. 3B, when the arithmetic processing is performed based on the first expression in the coordinate conversion unit 35, the captured image rotates around the horizontal axis in the virtual three-dimensional space. Thus, the angle-converted image 53 is generated. The horizontal axis extends through the center of rotation 52 and parallel to the xy plane. As illustrated in FIG. 3C, when the coordinate conversion unit 35 performs the calculation process based on the second expression, the angle-converted image 53 is projected onto the two-dimensional plane including the imaging surface 51. Thus, the projection image 54 is generated. The rotation angle t around the horizontal axis may be set so that the subject appears in a bird's eye view on the projected image 54. The first image data specifies the projection image 54.

  When calculation processing is performed on the first image data based on the third expression in the coordinate conversion unit 35, a range 55 corresponding to the overhead image is cut out from the projection image 54 as shown in FIG. In this way, the cut image 56 is generated. The cut image 56 is aligned with the display screen 57. Here, the constant a indicates the distance between the center of the range 55 and the center of the display screen 57 in the x-axis direction, and the constant b indicates the distance between the center of the range 55 and the center of the display screen 57 in the y-axis direction. The second image data specifies the cut image 56.

  When the coordinate conversion unit 35 performs the calculation process on the second image data based on the fourth expression, the cut image 56 is deformed in accordance with the number of pixels of the display screen 57 as shown in FIG. Here, the image is reduced in the y-axis direction and the image is enlarged in the x-axis direction. Thus, the buffer image 58 is generated. Here, the constant c indicates the enlargement ratio of the cut image 56 with respect to the display screen 57 in the x-axis direction, and the constant d indicates the enlargement ratio of the cut image 56 with respect to the display screen 57 in the y-axis direction. The third image data specifies the buffer image 58.

  When the overhead image processing unit 36 performs arithmetic processing on the third image data based on the fifth equation, the buffer image 58 is restored to the cut-out image 56 as shown in FIGS. . When the overhead image processing unit 36 performs a calculation process on the clipped image 56 based on the sixth equation, the clipped image 59 is aligned in the virtual three-dimensional space as shown in FIG. Thus, the first restored image 61 is generated within the range 55. The first restored image 61 corresponds to the projection image 54 described above. When the overhead image processing unit 36 performs arithmetic processing on the first restored image 61 based on the seventh equation, the aligned first restored image 61 is displayed in the virtual three-dimensional space as shown in FIG. In reverse around the horizontal axis. In this way, the second restored image 62 is generated. When the overhead image processing unit 36 performs a calculation process on the second restored image 62 based on the eighth equation, as shown in FIG. 4E, the second restored image 62 is captured on the imaging surface in the virtual three-dimensional space. 51 is projected onto a two-dimensional plane 63 including 51. In this way, the third restored image 64 is generated. The third restored image 64 corresponds to a subject in a range cut out as an overhead image on the captured image. The fourth image data specifies the third restored image 64.

  Distortion correction is performed on the fourth image data. According to the lookup table 37, the third restored image 64 is subjected to coordinate transformation. As a result, distortion correction processing is performed on the fourth restored image. In this way, the original image before the overhead view is generated. In the original image before the overhead view, the distortion of the subject is corrected. The fifth image data specifies the original image before the overhead view.

  Three-dimensional angle conversion is performed on the fifth data. The coordinate conversion unit 35 performs arithmetic processing on the original image before the bird's-eye view based on the first and second expressions. An overhead image is thus generated. The sixth image data specifies an overhead image.

  The image data converter 24 generates an overhead image from the captured image. As shown in FIG. 5, in the captured image 66, the subject is reduced as the distance from the optical axis 17 toward the edge of the screen increases. As a result, the straight line shape of the subject in real space is lost and the straight line shape is curved. The reference range 67 corresponds to the original image of the overhead image. A cut range 68 is cut out corresponding to the reference range 67 by the cut processing unit 42. Cutout area 68 completely includes reference area 67. Here, the cut-out range 68 is partitioned into a trapezoid. The lower base of the trapezoid coincides with the lower side of the captured image 66. The upper base and legs of the trapezoid are set outside the reference range 67 and parallel to the contour of the reference range 67.

  FIG. 6 shows the concept of image conversion of the three-dimensional angle conversion unit 41. The captured image rotates around the horizontal axis based on the arithmetic processing of the first and second formulas in the coordinate conversion unit 35. As the image is rotated, the trapezoidal cutout area of the projection image 69 overlaps a rectangle or a square. The rotation angle t is set by the three-dimensional angle conversion unit 41 so as to overlap.

  As shown in FIG. 7, in the lower side 71 a of the overhead image 71, the pixels of the overhead image 71 and the pixels of the captured image 66 correspond one-to-one. The resolution is maintained. In the upper side 71b of the overhead image 71, the captured image 66 is enlarged at a magnification of about 1.5 times. In other words, the number of pixels of 1 / 1.5 in the captured image 66 is extracted with respect to the number of pixels of the overhead image 71. Degradation of resolution is suppressed. On the side edges 71c and 71d of the overhead image 71, the captured image 66 is enlarged at a magnification of about 1.2 times. With respect to the number of pixels of the bird's-eye view image 71, a pixel having a number of pixels of 1/2 in the captured image 66 is extracted. Degradation of resolution is suppressed. In the vertical line 71v at the center position in the horizontal direction and the horizontal line 71h at the center position in the vertical direction, the pixels of the overhead image 71 and the pixels of the captured image 66 correspond approximately one to one. The resolution is maintained.

  FIG. 8 shows the magnification of the overhead image according to the first comparative example. The image pickup element has a resolution of 640 × 480 in forming the overhead view image. All the pixels of the captured image 72 are used. Distortion correction and three-dimensional angle conversion are performed on the captured image 72. In the lower side 72a of the overhead image, the pixels of the overhead image correspond to the pixels of the captured image 72 on a one-to-one basis. In the upper side 72b of the overhead image, the captured image 72 is enlarged at a magnification of about 3.0 times. In other words, one-third pixels are extracted from the captured image with respect to the number of pixels in the overhead image. The resolution is degraded. On the side sides 72c and 72d of the overhead image, the captured image 72 is enlarged at a magnification of about 2.2 times. In the captured image 72, the number of pixels of 1 / 2.2 is extracted with respect to the number of pixels of the overhead image. The resolution is degraded. On the vertical line 72v at the center position in the horizontal direction, the captured image 72 is enlarged at a magnification of about 1.9 times. That is, a pixel having a number of pixels of 1.9 is extracted from the captured image with respect to the number of pixels of the overhead image. The resolution is degraded. In this case, the image quality deteriorates remarkably partially in one display screen, so that the image quality deterioration becomes conspicuous.

  FIG. 9 shows an overhead image according to the second comparative example. In forming the bird's-eye view image, the image sensor 22 has a resolution of 1280 × 960. All pixels of the captured image 66 are used. Distortion correction and three-dimensional angle conversion are performed on the captured image 66. In the lower side 73a of the overhead image, the captured image 66 is enlarged at a magnification of about 0.5 times. In other words, twice as many pixels are extracted in the captured image 66 as the number of pixels in the overhead image. Although pixel information of two pixels is used for one pixel, the pixel information is not deteriorated, so that the resolution can be maintained. In the upper side 73b of the overhead image, the captured image 66 is enlarged at a magnification of about 1.5 times. That is, in the captured image 66, the number of pixels that is 1 / 1.5 of the number of pixels of the overhead image is extracted. Degradation of resolution is suppressed. The captured image 66 is enlarged at a magnification of about 1.1 times on the sides 73c and 73d of the overhead image. In the captured image 66, the number of pixels of 1 / 1.times. Is extracted from the number of pixels of the overhead image. Degradation of resolution is suppressed. In the horizontal line 73v in the horizontal direction, the captured image 66 is enlarged at a magnification of about 0.9 times. In other words, about 1.1 times as many pixels are extracted in the captured image 66 as the number of pixels in the overhead image. The resolution is maintained. In this case, pixel information of all the pixels of the captured image 66 is temporarily stored in the buffer storage unit. Therefore, the increase in the number of pixels of the captured image 66 requires an increase in capacity in the buffer storage unit. At present, there is a correlation between the capacity of the memory and the size of the memory, so that the increase in capacity induces an increase in the size of the buffer memory. When the size of the buffer memory increases in this way, it is difficult to incorporate the buffer memory in the camera device 16.

  In the image data conversion device 24, the image data of the captured image 66 is processed by the coordinate conversion unit 35 before being stored in the buffer storage unit 29. The coordinate conversion unit 35 extracts pixels in a range corresponding to the overhead view image 71 of the ground among all the pixels of the captured image 66. Thus, the information amount of the pixel information is reduced. Although the buffer storage unit 29 has a capacity smaller than the information amount of information regarding all the pixels of the captured image 66, the overhead image 71 can be reliably generated. Even if the resolution of the captured image 66 is increased, an increase in the capacity of the buffer storage unit 29 can be avoided. If the resolution of the picked-up image 66 is increased, even when the picked-up image 66 is partially enlarged when the overhead image 71 is generated, deterioration of the image quality can be largely avoided.

  As described above, the coordinate conversion unit 35 performs arithmetic processing according to the mathematical formula. At this time, the capacity of the buffer storage unit 29 can be kept small. In particular, simple mathematical formulas are used as described above when using mathematical formulas. As a result, the processing load can be reduced as much as possible.

  In general, in the captured image 66 of the subject projected from the fisheye lens 21, the subject is distorted as the distance from the optical axis 17 of the fisheye lens 21 increases. If the captured image 66 is simply subjected to linear coordinate transformation, distortion of the subject remains in the overhead image. The distortion correction corrects the distortion of the subject. As a result, the appearance of the overhead view image 71 is improved.

  A display panel 15 having a resolution lower than that of the image sensor 22 is used. In such a case, even if the captured image 66 is enlarged when the overhead image 71 is generated, the lack of pixel information can be compensated for when the image is enlarged. As a result, degradation of image quality can be avoided.

  In addition, as shown in FIG. 10, the capacities of the first buffer area 31 and the second buffer area 32 may be increased from the above-mentioned 640 × 480 to 768 × 576. By doing so, the number of pixels of the captured image 66 used for the overhead image 74 can be increased even if the size of the cutout range 55 is the same. As a result, the pixels of the bird's-eye view image 74 and the pixels of the captured image 66 have a one-to-one correspondence on the lower side 74a of the overhead image 74, the vertical line 74v in the horizontal center, and the horizontal line 74h in the vertical center. The resolution is maintained. In the upper side 74b of the overhead image 74, the captured image 66 is enlarged at a magnification of about 1.5 times. In the side edges 74c and 74d of the overhead image 74, the captured image 66 is enlarged at a magnification of about 1.1 times. In this way, degradation of resolution can be further effectively suppressed.

  DESCRIPTION OF SYMBOLS 11 Vehicle (automobile), 13 Driving assistance apparatus, 14 Navigation apparatus, 16 Camera apparatus, 21 Fisheye lens, 24 Image data conversion apparatus, 29 Buffer memory | storage part, 35 Coordinate conversion part, 36 Overhead image processing part, 53 Image after angle conversion, 54 projected image, 55 range, 57 display screen, 58 buffer image, 61 first restored image, 62 second restored image, 64 third restored image, 66 captured image, 71 overhead view image.

Claims (8)

A coordinate conversion unit that extracts pixel information for each pixel in a range corresponding to an overhead image from imaging data that specifies pixel information for each pixel with respect to a captured image of a subject projected from a fisheye lens;
An image data conversion apparatus comprising: a buffer storage unit that has a capacity smaller than an information amount of information about all pixels of the captured image and stores pixel information about the extracted pixels.   The image data conversion device according to claim 1, wherein the image data conversion device includes a storage unit, and the coordinate conversion unit is a pixel for each pixel within a range corresponding to an overhead image based on a mathematical formula stored in the storage unit. An image data converter characterized by extracting information.   3. The image data conversion device according to claim 2, wherein the storage unit includes a first equation for rotating the captured image around a horizontal axis in a virtual three-dimensional space to generate an angle-converted image, and the imaging A second expression for projecting the angle-converted image onto a two-dimensional plane including the image to generate a projection image, and a range corresponding to the overhead image is cut out from the projection image, and the cut projection image is displayed on the display screen. An image data conversion device characterized by storing the third equation for generating a buffer image by deforming according to the number of pixels.   4. The image data conversion device according to claim 3, wherein a first restored image is generated from the buffer image in a range cut from the projection image, and the first restored image around the horizontal axis in the virtual three-dimensional space. The second restored image is generated by projecting the second restored image onto the two-dimensional plane including the captured image in the virtual three-dimensional space by generating the third restored image An image data conversion apparatus comprising: a bird's-eye view image processing unit that performs distortion correction processing to generate an original image before bird's-eye view and generates the bird's-eye view image based on the first expression and the second expression.   5. The image data conversion apparatus according to claim 4, wherein the overhead image processing unit generates the overhead image with a smaller number of pixels than the number of pixels of the captured image.   A driving support apparatus comprising the image data conversion apparatus according to claim 1.   A navigation apparatus comprising the image data conversion apparatus according to claim 1.   A camera apparatus comprising the image data conversion apparatus according to claim 1.