JP2011182040A - Method of compressing data for image processing, and compressor and imaging apparatus including the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve sampling equal to that of a prior art by reducing a capacity of data for image processing through compression and decompression of the data for use. <P>SOLUTION: When an image deformation unit 7 outputs X-coordinates data and Y-coordinates data that are criteria for determining addresses read from an image buffer unit 4 by a raster image generation unit 5 in synchronization with a vertical synchronization signal and a horizontal synchronization signal, the image deformation unit 7 reads data recorded in a lookup table 8 and performs linear interpolation to the read vertex data, thus determining correspondence between a sensor image of the image buffer unit 4 and an output image at a pixel level. In the vertex data of the lookup table 8 used for the determination, a differential value in a horizontal direction of the vertex data, and a differential value in a vertical direction are calculated each, and lookup data compressed by calculating a difference are stored. The stored lookup data are decompressed and used, and a sampling having an interval narrower than before regardless of the same data capacity, is then achieved to improve an appearance quality of an image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to image processing data used in devices such as a still camera and a video camera that perform image processing, and more particularly to a compression method and a compression device for the data, and an imaging device including the data compression method.

  Conventionally, in a camera system such as an in-vehicle camera, when image correction is performed on a captured image, such as distortion correction, rotation correction, look-down correction, and other necessary image transformation processing, as image processing data in advance for speeding up the processing A lookup table that stores the correspondence between the coordinate values of input and output pixels is used.

  Patent Document 1 discloses correction of image distortion caused by the imaging lens, correction of image tilt due to tilting or overhead view, viewpoint change, mirror image inversion, and electronic zoom processing in imaging data obtained through the imaging lens. As an image processing method for obtaining output image data in which at least one of the above is performed, the output image data corresponds to addresses every integer a in the horizontal scanning direction (X direction) and every integer b in the vertical scanning direction (Y direction). A lookup table storing the address of the imaging data to be stored is prepared in advance, and if the address of the imaging data corresponding to the address of the output image data does not exist on the lookup table, it is calculated by the first interpolation process. The captured image data address or the image data address on the lookup table is used to calculate the output image data from the image data. Image data corresponding to the address generated by the second interpolation processing, it is described that the output image data.

  Further, in Patent Document 2, in order to correct a distorted image and reduce the amount of data stored in the storage unit, the data amount of each sampling point is calculated using the difference value of the coordinate data between the output image and the input image. It describes compression and decompression of the compressed data.

  As in Patent Document 1, image processing that performs linear interpolation using a lookup table that stores the address of imaged data requires a very small amount of computation. For example, when a certain part of the sensor image is to be enlarged (digital zoom) and displayed, the pixel values of the vertices indicated by 7 × 5 gray dots shown in the sensor image of FIG. 3A are stored in the lookup table. In addition, the corresponding pixels of the output image as shown in FIG. Then, when generating the output image, if there is a corresponding pixel, the value of the pixel value of the sensor image as it is is calculated by performing linear interpolation from the surrounding corresponding pixels if there is no corresponding pixel. Complex image processing represented by digital zoom is realized with a small circuit.

  However, this method can generate an output image without error when the output image is generated by performing linear conversion such as enlargement / reduction on the sensor image, but nonlinear conversion such as lens distortion correction is possible. When performing the above, an error occurs when generating the output image. Furthermore, the smaller the sampling interval for each integer a in the X direction and every integer b in the Y direction, the better the appearance of the image (in the example shown in FIG. 3A, the number of gray dots is 7 × 5 → 8). However, since the size of the lookup table becomes large, there is a limit in reducing the sampling interval in terms of data capacity. Specifically, there is a problem that when the number of samples is doubled, the data capacity is doubled.

  Also, recent image sensors have a very large number of pixels, and the storage capacity required for the look-up table tends to increase correspondingly. In addition, it is necessary to prepare a look-up table for each image deformation process, and a wide variety of image deformation processes are required, which increases the scale and cost of the look-up table.

  Moreover, although what is described in Patent Document 2 is considered to be useful when simply performing distortion correction, a normal view, a high angle view, and the like such as an in-vehicle camera are not limited to distortion correction, but more various. This is not suitable for performing various image deformations.

  The present invention solves the above-described problems of the prior art, by compressing image processing data (lookup data) stored in a data storage means (lookup table) and using the lookup data obtained by decompressing the data. Image processing data compression method and compression apparatus, and image pickup apparatus including the same, which can realize sampling with a narrower interval than the prior art even with the same data capacity and can improve the appearance of the image The purpose is to provide.

  In order to achieve the above object, according to a first aspect of the present invention, there is provided a method for compressing image processing data, the address for each integer a in the horizontal scanning direction (X direction) of the imaging data, and the vertical scanning direction ( Lookup table in which the address of the imaging data corresponding to the address of each integer b is stored in advance in the Y direction), and the lookup data used when performing interpolation processing with a plurality of coordinate data with reference to the lookup table For the coordinate data in the X direction of the lookup data, the difference between the coordinate data in the horizontal scanning direction is compressed and stored, and the coordinate data in the Y direction is stored in the vertical scanning direction. It is characterized in that the difference is taken and compressed and stored.

  This method can reduce the amount of data to be stored in the lookup table.

  The invention described in claims 2 and 3 is the image processing data compression method according to claim 1, wherein all the differences d in the X direction of the look-up data are d ≧ 0 (or all d ≦ 0). Furthermore, the difference d in the Y direction of the lookup data is all d ≧ 0 (or all d ≦ 0).

  According to this method, when the sign bit is constant, the data amount can be further reduced by reducing the sign bit in each data after being compressed by the difference.

  According to a fourth aspect of the present invention, there is provided the image processing data compression method according to any one of the first to third aspects, wherein the lookup data stores the rotational conversion amount of the image at the same time. To do.

  By this method, it is possible to perform a deformation process in which an image is rotated when the top-to-bottom direction does not coincide with human consciousness, which occurs when the vehicle is installed vertically on a side mirror with an in-vehicle camera or the like.

  Further, the image processing data compression apparatus according to any one of claims 5 to 8 is different only in category from the compression method according to claims 1 to 4 described above, and the imaging apparatus according to any one of claims 9 to 12. Is provided with the compression device according to claims 5 to 8 to which the compression method according to claims 1 to 4 is applied, and since it has the same configuration and operation as described above, its description is omitted.

  According to the present invention, it is possible to reduce the amount of lookup data to be stored in the lookup table, and further, when the sign bit is constant, the code bit of the differential data after compression is reduced, The amount of data can be reduced, and even if the direction of the top and bottom does not match the human consciousness due to the installation position of the vehicle-mounted camera or the like, it is possible to match the images by rotating and deforming the images.

1 is a block diagram illustrating a schematic configuration of an imaging apparatus including a compression apparatus to which an image processing data compression method according to an embodiment of the present invention is applied. The figure which divided | segmented the image size of 640x480 into 16 blocks of 160x120 in the output image in this embodiment, and described the vertex number The figure which shows the output image (b) which only cut out and expanded the sensor image (a) in this embodiment The figure which shows the output image (b) which cut and enlarged by performing distortion correction to the sensor image (a) in this embodiment The figure which shows the sensor image and output image of the normal view (a) of a vehicle-mounted camera in this embodiment, and the sensor image and output image of a high angle view (b). The figure which shows the output image (b) which performed rotation conversion to the sensor image (a) in this embodiment Flowchart of processing method related to data compression in this embodiment The figure which shows the output image (b) of Picture by Picture produced | generated by the sensor image (a) in this embodiment

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of an imaging apparatus including a compression apparatus to which an image processing data compression method according to an embodiment of the present invention is applied. The imaging apparatus shown in FIG. 1 includes a control unit, an operation unit, an image storage unit, an image display unit, and the like, but the description is omitted.

  In FIG. 1, an image sensor 1 is constituted by, for example, a CCD or a CMOS sensor for converting a captured optical image into an electric signal (pixel data). The image sensor 1 is provided with a Bayer array color filter, and the Bayer array RGB pixel data is sequentially output.

  The A / D converter 2 converts the RGB pixel data of the Bayer array as an analog signal output from the image sensor 1 into a digital signal and sends the digital signal to the Bayer interpolation unit 3. In this embodiment, the digital signal is assumed to be composed of 8 bits for each of RGB. In general, an AGC circuit is provided in the previous stage of the A / D converter 2, but the description thereof is omitted here.

  The Bayer interpolation unit 3 receives the RGB pixel data of the Bayer array converted into a digital signal, generates pixel data of all coordinate positions by linear interpolation independently for each RGB color, and sends the pixel data to the image buffer unit 4.

  The image buffer unit 4 stores the image data (sensor image) sent from the Bayer interpolation unit 3 for a predetermined time.

  The raster image generation unit 5 calculates an address value based on the X coordinate and the Y coordinate received from the image deformation unit 7 and acquires pixel data from the image buffer unit 4. That is, the raster image generation unit 5 has a function of determining the correspondence between the sensor image stored in the image buffer unit 4 and the output image at the image level.

  The D / A converter 6 converts the digital signal into an analog signal and transmits a signal such as NTSC to an external display or the like.

  The image transformation unit 7 outputs X coordinate data and Y coordinate data, which are a basis for determining an address read by the raster image generation unit 5 from the image buffer unit 4, in synchronization with the vertical synchronization signal and the horizontal synchronization signal. At that time, the image transformation unit 7 reads the data recorded in the lookup table 8 of the data storage means, and performs linear interpolation on the read vertex data, so that the sensor stored in the image buffer unit 4 It is responsible for determining the correspondence between the image and the output image at the pixel level.

  For example, as shown in FIG. 2, an output image having a size of 640 × 480 is divided into 16 blocks having a size of 160 × 120, and the coordinate data of the four corners of each block is read from the lookup table 8 to perform linear interpolation. By doing so, the coordinate data inside the block is generated. With respect to this linear interpolation method, an actual image can be obtained by using the image generation method described in Patent Document 1 described above.

  The present invention relates to a method for compressing image processing data (lookup data) stored in the lookup table 8, and an example of image processing using the lookup table 8 will be described below.

  FIG. 3 is an example in which the sensor image in FIG. 3A is simply cut out and enlarged to obtain the output image in FIG. 3B. FIG. 4 shows a distortion in the sensor image in FIG. This is an example in the case where the sensor image is cut out and enlarged after correction to obtain the output image of FIG. Since lens distortion occurs in the target system with respect to the center of the image, when performing distortion correction, the sample vertex in the sensor image of FIG. 4A draws a curve. FIG. 2 described above shows the output images shown in FIGS. 3B and 4B with vertex numbers.

  Regarding the coordinate data in the X direction at each vertex shown in FIG.

Holds. That is, with respect to the X coordinate data, the relationship of (Equation 1) is established in the horizontal direction.

  Regarding the coordinate data in the Y direction, (Equation 2)

Holds. That is, with respect to the Y coordinate data, the relationship of (Equation 2) is established in the vertical direction.

  These relationships are not limited to the examples shown in FIGS. 3 and 4, but also hold in a normal normal view and a high angle view (see FIGS. 5A and 5B). The in-vehicle camera requires complicated image deformation because it uses a complex lens system called a fisheye lens, and various image deformations are performed. It is easy to see if it is deformed so that the lower side and the far side are the upper side of the image. In addition, since an image photographed with a fisheye lens is generally installed so that the sky is on the upper side and the ground is on the lower side, the above-described (Equation 1) and (Equation 2) are established in the image deformation of the in-vehicle camera. Most cases.

  In addition, even when an in-vehicle camera is installed on the side mirror and the image is taken vertically downward, the image transformation that satisfies (Equation 1) and (Equation 2) can be achieved only by rotating the captured image. (FIGS. 6A and 6B).

  When (Equation 1) and (Equation 2) hold, if compression is performed using difference data, code data becomes unnecessary, and therefore one bit can be compressed with respect to the number of bits of each difference data. .

  Next, a processing method related to data compression in the present embodiment will be described with reference to the flowchart of FIG.

  A coordinate position is determined (S1). As shown in FIG. 2, coordinates V1 (x1, y1), V2 (x2, y2),... V25 (x25, y25) of the coordinate system in the sensor image corresponding to the 25 vertices are determined. If it is only desired to enlarge the obtained sensor image, the vertices shown in FIGS. 3A and 3B may be associated with each other. The association may be performed manually or automatically.

  The rotation angle is calculated (S2). This is for calculating the rotation angle α satisfying the above-described (Equation 1) and (Equation 2). The coordinate V1 ′ (x1 ′, y1 ′) to V25 of the vertices obtained by performing the rotation transformation R on the coordinates V1 to V25 calculated by determining the coordinate position in the process S1, where R is the two-dimensional rotation matrix of the rotation angle α. Consider '(x25', y25 ').

  At this time, the rotation angle α satisfying the following (Equation 3) is searched. In general, the rotation angle α satisfying (Equation 3) is not uniquely determined, but it is sufficient if a certain rotation angle α can be determined. However, when (Equation 3) holds when the rotation angle α = 0, it is preferable to set the rotation angle α = 0.

When decompressing the data, reverse rotation conversion (rotation angle -α) is performed.

Difference data is calculated (S3). The difference value is calculated using “x1 ′ to x25 ′” and “y1 ′ to y25 ′” determined by calculating the rotation angle in the process S2.
With respect to “x1 ′ to x25 ′”, a difference is taken in the horizontal direction, and 20 pieces of difference information are acquired as in the following (Equation 4).

  Further, regarding “y1 ′ to y25 ′”, a difference is taken in the vertical direction, and 20 pieces of difference information are acquired as shown in the following (Equation 5).

Thereby, each value of the horizontal difference value of the X coordinate and the vertical difference value of the Y coordinate is calculated.

  The compressed data is generated (S4). First, in the example shown in FIG. 2, the start block number (= 1), the number of X direction blocks (= 4), the number of Y direction blocks (= 4), the rotation angle α, and the start point of the horizontal X coordinate (x1 ′) , X6 ′, x11 ′, x16 ′, x21 ′), the start point of the Y coordinate in the vertical direction (each coordinate of y1 ′, y2 ′, y3 ′, y4 ′, y5 ′), the difference value of the X coordinate Each value of (dx1 to dx20) and Y coordinate difference values (dy1 to dy20) is written as one data in a storage device such as SPI (Serial parallel Interface).

  Here, the start block number represents the number of the block located at the upper left of the blocks in the output image (see FIG. 2), and the number of X-direction blocks represents the number of horizontal blocks and the number of Y-direction blocks in the output image. Represents the number of blocks in the vertical direction in the output image.

Furthermore, the number of X direction blocks, the number of Y direction blocks, the number of elements at the start point of the X coordinate in the horizontal direction, the number of elements at the start point of the Y coordinate in the vertical direction, the number of elements of the difference value of the X coordinate, The relationship between the difference value and the number of elements is as follows: “the number of elements at the start point of the horizontal X coordinate = the number of blocks in the X direction + 1”
Number of elements at start point of Y coordinate in vertical direction = number of blocks in Y direction + 1
Number of elements of X coordinate difference value = (Number of blocks in X direction + 1) × Number of blocks in Y direction Number of elements of difference value in Y coordinate = (Number of blocks in Y direction + 1) × Number of blocks in X direction ”
become that way.

  In the present embodiment, the method of performing only the compression related to the present invention is shown, but further compression may be performed using other data compression methods. For example, the difference data for each start point of the horizontal X coordinate may be calculated and stored, or the difference data for each start point of the Y coordinate in the vertical direction may be calculated and stored. May be. In addition, data to which further data compression processing is added may be written in the SPI. However, when these processes are added, a circuit for decompressing data becomes complicated. Therefore, it is desirable to determine the compression method after considering the trade-off with the circuit scale.

  In the present embodiment, a method of outputting an image of one screen has been described. However, a Picture by Picture output image as illustrated in FIG. 8 can be generated. For that purpose, the data regarding the area | region described with the gray dot of FIG. 8 is produced according to the procedure mentioned above. Next, data relating to a region described with black dots is created. Finally, Picture by Picture can be realized by integrating gray dot data and black dot data as a single piece of data. Similarly, data for Picture in Picture can be created.

  As described above, by compressing the look-up data used for image processing stored in the look-up table 8 and decompressing and using the same, it is possible to realize sampling with a narrower interval than in the past even with the same data capacity. The appearance of the image can be improved.

  The image processing data compression method and apparatus according to the present invention and an image pickup apparatus including the same can reduce the amount of lookup data to be stored in the lookup table, and further, when the sign bit is constant The amount of data can be reduced by reducing the amount of bits, and the image can be rotated and deformed to match when the orientation of the top and bottom does not match the human consciousness depending on the installation position of the camera, which is useful for image processing. is there.

DESCRIPTION OF SYMBOLS 1 Image pick-up element 2 A / D conversion part 3 Bayer interpolation part 4 Image buffer part 5 Raster image generation part 6 D / A conversion part 7 Image deformation part 8 Look-up table

JP 2009-10730 A Japanese Patent No. 4144292

Claims (12)

A lookup table is prepared in advance, which stores addresses for each integer a in the horizontal scanning direction (X direction) of the imaging data and addresses of the imaging data corresponding to addresses for each integer b in the vertical scanning direction (Y direction). , Referring to the lookup table, a compression method related to lookup data used when performing interpolation processing with a plurality of coordinate data,
The coordinate data in the X direction of the lookup data is compressed and stored in the horizontal scanning direction, and the coordinate data in the Y direction is compressed and stored in the vertical scanning direction. A method for compressing image processing data, wherein   2. The image processing data compression method according to claim 1, wherein all the differences d in the X direction of the lookup data satisfy d ≧ 0 (or all d ≦ 0). Compression method.   3. The method for compressing image processing data according to claim 1, wherein all the differences d in the Y direction of the lookup data are d ≧ 0 (or all d ≦ 0). The data compression method.   The image processing data compression method according to any one of claims 1 to 3, wherein the look-up data also stores an image rotation conversion amount at the same time. A lookup table is prepared in advance, which stores addresses for each integer a in the horizontal scanning direction (X direction) of the imaging data and addresses of the imaging data corresponding to addresses for each integer b in the vertical scanning direction (Y direction). , Referring to the look-up table, in the compression device for processing the look-up data used when performing the interpolation process with a plurality of coordinate data,
With respect to the coordinate data in the XY directions of the lookup data, the X direction is the horizontal coordinate in the horizontal scanning direction and X coordinate horizontal difference value calculation means for compressing the coordinate data, and the Y direction is the vertical data in the vertical scanning direction to be compressed. An apparatus for compressing image processing data, comprising: Y-coordinate vertical difference value calculation means.   6. The apparatus for compressing image processing data according to claim 5, wherein all the differences d in the X direction of the lookup data are d ≧ 0 (or all d ≦ 0). .   7. The image processing data compression apparatus according to claim 5, wherein all the differences d in the Y direction of the lookup data satisfy d ≧ 0 (or all d ≦ 0). Compression device.   8. The image processing data compression apparatus according to claim 5, wherein the look-up data also stores an image rotation conversion amount at the same time. A lookup table is prepared in advance, which stores addresses for each integer a in the horizontal scanning direction (X direction) of the imaging data and addresses of the imaging data corresponding to addresses for each integer b in the vertical scanning direction (Y direction). , An imaging device having lookup data used when performing interpolation processing with a plurality of coordinate data with reference to the lookup table,
Regarding the coordinate data in the X and Y directions of the lookup data, the X direction takes the difference in the coordinate data in the horizontal scanning direction, and the Y direction takes the difference in the coordinate data in the vertical scanning direction to store the lookup data compressed. An imaging device comprising a table.   The imaging apparatus according to claim 9, wherein all the differences d in the X direction of the compressed lookup data satisfy d ≧ 0 (or all d ≦ 0).   11. The imaging apparatus according to claim 9, wherein all the differences d in the Y direction of the compressed lookup data satisfy d ≧ 0 (or all d ≦ 0).   The imaging apparatus according to claim 9, wherein the compressed lookup data includes an image rotation conversion amount.

Images