JP6225137B2 - In-vehicle camera image processing device - Google Patents

Description

  The present invention relates to an in-vehicle camera image processing apparatus that processes images captured by a plurality of imaging units.

  As a background art of this technical field, Patent Document 1 discloses that when image processing is performed on a wide-angle captured image by stereo viewing, an image input from each camera is parallel to the straight line on which each camera is arranged. An image projecting unit that projects onto a cylindrical surface having an axis, and a pixel position detecting unit that detects a pixel position corresponding to the object in the image projected onto the cylindrical surface; Techniques for reducing processing time are shown.

  In Patent Document 2, camera characteristic data for correcting variations in characteristics of a pair of cameras is stored in advance in the first nonvolatile memory in the stereo imaging unit, and the image processing unit has an internal memory. In this second non-volatile memory, there is shown a technique in which circuit characteristic data for correcting variations in characteristics of circuits that process two systems of image signals output from a pair of cameras is stored in advance.

JP 2009-139246 A Japanese Patent Laid-Open No. 11-211469

  However, just changing the central projection to the cylindrical projection as in the technique disclosed in Patent Document 1 is effective only when the captured images have the same size, and calibration data such as distortion correction increases. As the memory usage increases, the time to access the calibration data will increase accordingly, the cost will increase, the time required for the original image processing will decrease, and the image processing performance will increase. The effect cannot be expected.

  Further, as in the technique disclosed in Patent Document 2, if only a nonvolatile memory for storing correction data is separated, the amount of memory used for storing calibration data such as shading correction increases, and accordingly, the calibration data is included in the calibration data. The access time will increase, the cost will increase, the time required for the original image processing will decrease, and no effect can be expected to improve the image processing performance.

  In the future, in order to improve the functions and performance of the in-vehicle camera, the number of pixels of the image sensor and the amount of correction data will increase. With such means, the correction is performed in the memory storing the correction data. The proportion of data increases, and image data access necessary for original image processing is restricted. Adopting a large-capacity and high-speed memory as a solution means is disadvantageous from the viewpoint of cost and power consumption.

  The present invention has been made in view of the above points, and an object thereof is to provide an in-vehicle camera image processing apparatus capable of reducing the data amount of correction data stored in a memory unit as much as possible. It is.

In order to solve the above problems, the present invention employs, for example, the configurations described in the claims.
The present invention includes a plurality of means for solving the above-mentioned problems. To give an example, a shading correction unit that corrects shading of image data captured by two camera units, and correction of one camera unit, respectively. A memory unit for storing a correction table having a difference value or an average value between the table and the correction table of the other camera unit, and a correction table of the one camera unit and the other of the correction table having the difference value or the average value A correction table generation unit that generates at least one of the correction tables of the camera unit, wherein the shading correction unit performs shading correction on the image data using the correction table generated by the correction table generation unit. To do.

  According to the present invention, the amount of correction data stored in the memory unit can be reduced as much as possible, and the efficiency of image processing can be improved. Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

1 is a configuration diagram of an in-vehicle camera image processing apparatus in Embodiment 1. FIG. The figure explaining the method of carrying out the shading correction | amendment of the captured image using a shading correction table. FIG. 6 is a diagram illustrating a configuration example of a shading correction table according to the first embodiment. FIG. 10 is a diagram illustrating a configuration example of a shading correction table in Embodiment 2. FIG. 10 is a diagram illustrating a configuration example of a shading correction table in Embodiment 3. FIG. 10 is a diagram illustrating a configuration example of a shading correction table in Embodiment 4. FIG. 10 is a diagram illustrating a configuration example of a shading correction table according to a fifth embodiment. FIG. 10 is a diagram illustrating a configuration example of a shading correction table according to a sixth embodiment. FIG. 10 is a diagram illustrating a configuration example of a shading correction table according to a seventh embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

[Example 1]
FIG. 1 is a configuration diagram of an in-vehicle camera image processing apparatus according to the first embodiment.
In the present embodiment, a case of a stereo camera having two camera units will be described as an example of the in-vehicle camera image device. The in-vehicle camera image processing apparatus 100 includes a pair of left and right camera sections (left) 101 and camera section (right) 102, and image data 121 and 122 captured by the camera section (left) 101 and camera section (right) 102, respectively. The shading correction units 103 and 104 that perform shading correction, the image data units 111 and 112 of the memory unit 106 that store the image data 123 and 124 after shading correction, and the image data 125 from the image data units 111 and 112 of the memory unit 106. An image processing microcomputer 107 that takes in and performs image processing is included.

  One camera unit (left) 101 and the other camera unit (right) 102 are imaging elements such as a CCD image sensor and a CMOS image sensor, and have a configuration in which a plurality of pixels are arranged in a matrix. In the camera unit (left) 101 and the camera unit (right) 102, generally, an object is imaged via an optical lens, and becomes an electrical signal of the image data 121 and 122 by an image sensor or the like. In principle, the edge of the screen appears dark due to the characteristics of the lens. This is because the so-called peripheral light amount falls. When matching processing is performed on the left and right image data 121 and 122 acquired by the camera unit (left) 101 and the camera unit (right) 102 as in a stereo camera, the in-plane brightness is constant. Is ideal, the image data 121 and 122 need to be corrected so that the in-plane luminance is constant. This is called shading correction.

  The correction data used for the shading correction is stored in, for example, the flash memory unit 108 which is a non-volatile memory, and is read out by the image processing microcomputer 107 when the in-vehicle camera image processing apparatus 100 is activated. 106 stored in the shading correction table sections 113 and 114. Each time image data captured from the camera unit (left) 101 and the camera unit (right) 102 is acquired, the image data is read from the shading correction table units 113 and 114 for each frame, for example, and is read by the shading correction units 103 and 104. Used for correction calculation.

FIG. 2 is a diagram for explaining a method for shading correction of a captured image using a shading correction table.
The shading correction table 302 has correction data that becomes one-to-one for each pixel for each pixel i (m, n) of the captured image (image data before shading correction) 301. When each correction data in the shading correction table 302 is expressed as j (m, n), the shading correction data j of the pixel data i at the pixel position coordinate (m, n) is applied to the correction calculation.

As a shading correction, when the captured image 301 is multiplied by the shading correction table, the image 303 after the shading correction is
k (m, n) = i (m, n) × j (m, n)
m = 1 to M (M is the horizontal size of the image), n = 1 to N (N is the vertical size of the image)
It is expressed.

Similarly, when offsetting, the image 303 after shading correction is
k (m, n) = i (m, n) + j (m, n)
m = 1 to M (M is the horizontal size of the image), n = 1 to N (N is the vertical size of the image)
It is expressed. This is an example, and the concept is the same for subtraction and division.

  In this way, the image data 121 and 122 become the image data 123 and 124 after the shading correction via the shading correction units 103 and 104 that perform the shading correction. For example, the image data in the memory unit 106 is temporarily stored. Stored in the sections 111 and 112. Thereafter, the image processing microcomputer 107 reads out the image data 125 at an arbitrary timing and performs desired image processing.

  What is characteristic in the present embodiment is that the flash memory unit 108 includes a shading correction table (left) of the left camera unit 101, a shading correction table (left) of the left camera unit 101, and a shading correction table of the right camera unit 102 (left). The left and right shading correction tables (left and right differences) are stored, and the correction data decoding unit 105 stores the shading correction table (left) of the left camera unit 101 and the shading correction table of the left camera unit 101 ( A shading correction table (right) of the right camera unit 102 is generated using a shading correction table (left / right difference) having a difference value between the left) and the shading correction table (right) of the right camera unit 102.

FIG. 3 is a diagram illustrating a configuration example of the shading correction table in the first embodiment.
In the shading correction units 103 and 104, a shading correction table (left) in accordance with optical characteristics represented by the respective imaging elements and lenses of one camera unit (left) 101 and the other camera unit (right) 102. Using A and the shading correction table (right) B, the image data 121 and 122 are subjected to shading correction, and the image data 123 and 124 after the shading correction are provided to the memory unit 106.

At this time, if the correction data is, for example, unsigned 12 bits / pixel and the image size is SXGA (1280 pixels × 1024 pixels), the shading correction tables A and B are respectively
A = 1280 × 1024 × 12bit = 1.875MB
B = 1280 × 1024 × 12bit = 1.875MB
It becomes.

  On the other hand, a new shading correction table (left-right difference) C is calculated using the difference between the shading correction table (left) A and the shading correction table (right) B. Like the stereo camera, when the optical characteristics represented by the left and right camera units 101 and 102 and the lenses are substantially equal, the left and right shading correction tables are characterized by close values. The above difference can be expressed by the number of bits sufficiently smaller than the unsigned 12 bits / pixel which is the original number of data bits. For example, here, it is assumed that it can be expressed by signed 6 bits.

The data amount of the shading correction table C at this time is
C = A−B = 1280 × 1024 × 6 bits = 0.9375 MB
It becomes.

Therefore, in this embodiment, the flash memory unit 108 stores a shading correction table (left) A and a shading correction table (left-right difference) C. If there is a shading correction table (left) A and a shading correction table (left-right difference) C, they are read from the memory unit 106 and stored in the shading correction table units 113 and 114 of the memory unit 106, and then the correction data decoding unit 105 ,
A−C = A− (A−B) = B
By doing so, the shading correction table (right) B can be generated. Thus, the correction data can be restored to the original, and the left and right image data can be subjected to shading correction by the shading correction units 103 and 104, respectively.

As an effect, the capacity of the flash memory unit 108 and the capacity occupied by the shading correction table in the memory unit 106 can be reduced by 25% by the following equation.
(A + C) / (A + B) = 2.8125 MB / 3.75 MB = 75% (25% reduction)

  As described above, in this embodiment, the flash memory unit 108 stores the shading correction table (left) A of the camera unit (left) 101 and the shading correction table (left-right difference) C having difference values, and stores the image. After being read by the processing microcomputer 107, it is stored in the shading correction table units 113 and 114 of the memory unit 106.

  The shading correction table (left) A has correction data set for each pixel for a plurality of pixels of the camera unit (left) 101, and the shading correction table (left-right difference) C is stored in the camera unit (left). ) Corrects a difference value between correction data set for each pixel with respect to a plurality of pixels included in 101 and correction data set for each pixel with respect to a plurality of pixels included in the camera unit (right) 102. It has as data.

  Then, the correction data decoding unit 105 generates a shading correction table (right) B of the camera unit (right) 102 by subtracting the shading correction table (left-right difference) C and the shading correction table (left) A. (Correction table generation unit). Then, the shading correction table (left) A is supplied to the shading correction unit 103, and the shading correction table (right) B is supplied to the shading correction unit 104.

  The shading correction unit 103 performs shading correction on the image data 121 of the camera unit (left) 101 using the shading correction table (left) A supplied from the correction data decoding unit 105, and the shading correction unit 104 includes a correction data decoding unit. The image data 122 of the camera unit (right) 102 is subjected to shading correction using the shading correction table (right) B generated in 105.

  Instead of the shading correction table (right) B, the flash memory unit 108 stores a shading correction table (left-right difference) C having a smaller data amount than the shading correction table (right) B. And the capacity occupied by the shading correction table in the memory unit 106 can be reduced. In addition, since the amount of data read from the flash memory unit 108 is reduced and the amount of data stored in the shading correction table units 113 and 114 is reduced, the startup time of the in-vehicle camera image processing apparatus 100 is shortened.

  Furthermore, since the amount of data for reading the shading correction table from the memory unit 106 can be reduced simply by adding the correction data decoding unit 105 having the simple configuration described above, power consumption can be reduced, Of the access amount to the memory unit 106, the read amount from the image processing microcomputer 107 for the original image processing performance can be increased, which can lead to performance improvement.

  In the future, in order to improve the functions and performance of in-vehicle cameras, the number of pixels in the image sensor and the amount of correction data will increase, and the proportion of correction data will increase in the memory that stores correction data. However, there is a problem that the image data access necessary for the original image processing is limited. However, according to the present invention, the amount of correction data stored in the memory can be reduced as much as possible. The problem can be solved without adopting a high-speed memory.

  In the present embodiment, the shading correction table (left) of the left camera unit 101, the shading having a difference value between the shading correction table (left) of the left camera unit 101 and the shading correction table (right) of the right camera unit 102. The shading correction table of the right camera unit 102 is generated using the correction table (left and right difference). Instead, the shading correction table (right) of the right camera unit 102 and the shading correction table (left) of the left camera unit 101 are generated. ) And the shading correction table (left difference) between the right camera unit 102 and the shading correction table (right) of the right camera unit 102 may be used to generate the shading correction table (left) of the left camera unit 101.

  In this embodiment, the case where the flash memory unit 108 and the memory unit 106 are separated has been described as an example. However, the flash memory unit 108 is omitted, and the shading correction table is stored only in the memory unit 106. It is good also as the structure made to do.

  The following embodiments describe several methods for reducing the data amount of the shading correction table described above. Note that the present invention itself is to generate difference data between two shading correction tables even if the method is not described in the following embodiments.

[Example 2]
FIG. 4 is a diagram illustrating a configuration example of a shading correction table in the second embodiment.
In the present embodiment, the shading correction table (left) A described in the first embodiment and the compressed shading correction tables A ′ and C ′ obtained by compressing the shading correction table (left-right difference) C are stored in the flash memory unit 108. It is characterized by that.

  A data compression technique of a general method is applied to each compression. As a compression method, it is desirable to employ a method in which the shading correction tables (left) A and (right) B generated by the correction data decoding unit 105 are similar correction data, and may be lossless compression. Lossless compression may be used. As a lossless compression method, for example, a method such as run length compression, compression using a Huffman code, or LZSS compression can be used. As a lossy compression method, for example, a JPEG compression method can be used.

If the correction data can be compressed to 70% by data compression, for example, the shading correction tables A ′ and C ′ are respectively
A ′ = 1.875 MB × 0.7≈1.31 MB
C ′ = 0.9375 MB × 0.7≈0.66 MB
It becomes.

The compression shading correction table (left) A ′ and the compression shading correction table (left / right difference) C ′ are stored in the flash memory unit 108, read by the memory unit 106, and stored in the shading correction table units 113 and 114 of the memory unit 106. Remember me. If these are present, the correction data decoding unit 105 performs a reverse compression (decompression) process,
A ′ × reverse compression = A, C ′ × reverse compression = C,
A−C = A− (A−B) = B
By doing so, the shading correction tables (left) A and (right) B can be generated. Thereby, the correction data can be returned to the original, and the left and right shading correction can be performed.

The capacity of the flash memory unit 108 storing the shading correction table and the capacity occupied by the shading correction table in the memory unit 106 can be reduced by 47% according to the following equation.
(A ′ + C ′) / (A + B) ≈1.97 MB / 3.75 MB≈53% (47% reduction)

Thereby, the capacity can be further reduced as compared with the first embodiment (25% reduction).
For example, as shown in FIG. 2, the shading correction table (left) A shown in FIG. 4 is data that makes the entire image uniform with respect to the pixels where the amount of light around the lens is reduced. Most of the areas including are characterized by a lot of the same or similar data.

  Therefore, in the compression method of the shading correction table (left) A, a higher effect is obtained when a Huffman code is used, which increases the compression efficiency when there is a difference in the number of appearances of each data in the entire data.

  The shading correction table (left-right difference) C shown in FIG. 4 is assumed to be a case where the correction data varies, and therefore, whether the data string matches the previous data string or not. When the LZSS compression method, which is a method of expressing using the coincidence position and the coincidence length, is used, a higher effect can be obtained.

  Considering the shading correction table as an image, as shown in FIG. 2, since the center of the image data is dark and the periphery is bright, it is possible to use, for example, a JPEG compression method that is irreversible compression. However, in the JPEG compression method, an image is divided into 8 × 8 pixel square blocks, and information on the change of the image is extracted in that unit, so a part of the information is discarded.

  Whether to select reversible or irreversible compression method is decided after considering the features of the optical system including the image sensor and lens, and the functions and system requirements of the image processing recognition application. The

[Example 3]
FIG. 5 is a diagram illustrating a configuration example of a shading correction table in the third embodiment.
In this embodiment, new shading correction tables D and E are calculated using the difference between the two shading correction tables (left) A and the shading correction table (right) B. The shading correction table D is a table obtained by dividing the sum of the shading correction table (left) A and the shading correction table (right) B by 2. The shading correction table E is a table obtained by dividing the difference between the shading correction table (left) A and the shading correction table (right) B by 2.

  Similar to the first embodiment, when the optical characteristics represented by the left and right camera units and the lenses are substantially equal, like the stereo camera, the left and right shading correction tables have characteristics that are close to each other. For example, the shading correction data E that takes the difference between the two correction tables can be expressed by a sufficiently smaller number of bits than the original number of data bits of 12 bits / pixel.

  For example, in the first embodiment, it can be expressed by 6 bits with a sign, but here, since it is divided by 2, it can be further reduced by 1 bit. Since the shading correction table D is an addition of unsigned 12 bits / pixel, it increases by 1 bit, but is divided by 2, so the result is unsigned 12 bits / pixel and the data amount does not increase.

The data amount of the shading correction table D and the shading correction table E at this time is
D = 1280 × 1024 × 12bit = 1.875MB
E = 1280 x 1024 x 5 bits ≒ 0.78MB
It becomes.

The shading correction table D and the shading correction table E are stored in the flash memory unit 108, read by the memory unit 106, and stored in the shading correction table units 113 and 114 of the memory unit 106. If these are present, after reading from the memory unit 106, the correction data decoding unit 105
D + E = (A + B) / 2 + (A−B) / 2 = A
D−E = (A + B) / 2− (A−B) / 2 = B
By doing so, the shading correction table (left) A and the shading correction table (right) B can be generated. Thereby, the correction data can be restored to the original, and the shading correction units 103 and 104 can perform the left and right shading corrections.

The capacity of the flash memory unit 108 storing the shading correction table and the capacity occupied by the shading correction table in the memory unit 106 can be reduced by 29% according to the following equation.
(D + E) / (A + B) ≈2.66MB / 3.75MB≈71% (29% reduction)
Therefore, the data capacity can be reduced as compared with the first embodiment (25% reduction).

  In this embodiment, the flash memory unit 108 has a shading correction having an average value of the sum of the shading correction table (left) A of the camera unit (left) 101 and the shading correction table (right) B of the camera unit (right) 102. A shading correction table E having an average value of differences between the table D and the shading correction table (left) A of the camera unit (left) 101 and the shading correction table (right) B of the camera unit (right) 102 is stored. The data is read out by the image processing microcomputer 107 at a predetermined timing and stored in the shading correction table units 113 and 114 of the memory unit 106.

  The shading correction table (left) A of the camera unit (left) 101 has correction data set for each of the plurality of pixels of the camera unit (left) 101, and The shading correction table (right) B has correction data set for each pixel with respect to a plurality of pixels of the camera unit (right) 102.

  The shading correction table D having the average value of the sum is obtained by correcting correction data set for each of a plurality of pixels included in the camera unit (left) 101 and a plurality of pixels included in the camera unit (right) 102. Thus, the correction data is a value obtained by dividing the sum of the correction data set for each pixel by two. The shading correction table E having an average difference value includes correction data set for each pixel of the plurality of pixels included in the camera unit (left) 101 and a plurality of pixels included in the camera unit (right) 102. In contrast, a value obtained by dividing the difference from the correction data set for each pixel by 2 is included as correction data.

  Then, the correction data decoding unit 105 adds the shading correction table D having the average value of the sum and the shading correction table E having the average value of the difference to thereby add the shading correction table (left) of the camera unit (left) 101. ) A is generated, and a shading correction table (right) B of the camera unit (right) 102 is generated by subtracting the shading correction table E having the average difference value from the shading correction table D having the average value of the sum. To do.

  The shading correction unit 103 performs shading correction on the image data 121 of the camera unit (left) 101 using the shading correction table (left) A supplied from the correction data decoding unit 105, and the shading correction unit 104 includes a correction data decoding unit. The image data 122 of the camera unit (right) 102 is subjected to shading correction using the shading correction table (right) B generated in 105.

  In the flash memory unit 108, instead of the shading correction table (left) A and the shading correction table (right) B, a shading correction table D having the same data amount as the shading correction table (left) A and a shading correction table ( Right) Since the shading correction table E having a smaller data amount than B is stored, the capacity of the flash memory unit 108 storing the shading correction table and the capacity occupied by the shading correction table in the memory unit 106 are reduced. be able to. Therefore, the same effect as in the first embodiment can be obtained.

[Example 4]
FIG. 6 is a diagram illustrating a configuration example of the shading correction table in the fourth embodiment.
The present embodiment is characterized in that the shading correction table D described in the third embodiment and the compressed shading correction tables D ′ and E ′ obtained by compressing the shading correction table E are stored in the flash memory unit 108.

  A data compression technique of a general method is applied to each compression. The compression method may be lossless compression or lossy compression. As a lossless compression method, for example, a method such as run length compression, compression using a Huffman code, or LZSS compression can be used. As a lossy compression method, for example, a JPEG compression method can be used.

If the correction data can be compressed to 70% by data compression, for example, the shading correction tables D ′ and E ′ are respectively
D ′ = 1.875 MB × 0.7≈1.31 MB
E ′ = 0.78 MB × 0.7≈0.55 MB
It becomes.

The shading correction table has the compression shading correction table D ′ and the compression shading correction table E ′. If there are these, after reading from the memory unit 106, the correction data decoding unit 105 performs a reverse compression (decompression) process,
D ′ × reverse compression = D, E ′ × reverse compression = E,
D + E = (A + B) / 2 + (A−B) / 2 = A
D−E = (A + B) / 2− (A−B) / 2 = B
By doing so, the shading correction tables (left) A and (right) B can be generated. Thereby, the correction data can be returned to the original, and the left and right shading correction can be performed.

The capacity of the flash memory unit 108 for storing the shading correction table and the capacity occupied by the shading correction table in the memory unit 106 can be reduced by 50% using the following equation.
(D ′ + E ′) / (A + B) ≈1.86 MB / 3.75 MB≈50% (50% reduction)

Thereby, the capacity can be further reduced as compared with the first embodiment (25% reduction).
Whether to select reversible or irreversible compression method is decided after considering the features of the optical system including the image sensor and lens, and the functions and system requirements of the image processing recognition application. The

[Example 5]
FIG. 7 is a diagram illustrating a configuration example of a shading correction table in the fifth embodiment.
In this embodiment, before calculating a difference between two shading correction tables, a data compression technique of a general method is applied to the shading correction table (left) A and the shading correction table (right) B, respectively. . Then, a new compression shading correction table (left-right difference) F is calculated with respect to the compression shading correction table (left) A ′ and the compression shading correction table (right) B ′ generated by data compression.

For example, here, it is assumed that the compression shading correction table (left) A ′ and the compression shading correction table (right) B ′ can be compressed to 70% compared to the one before compression, and the compression shading correction table ( If the data amount of the left / right difference) F is half of the compression shading correction table (left) A ′, the data amount of the compression shading correction table (left / right difference) F at this time is
F = A ′ × 50% = A × 70% × 50% ≈1.31 MB × 50% ≈0.66 MB
It becomes.

The compression shading correction table (left) A ′ and the compression shading correction table (left-right difference) F are stored in the flash memory unit 108, read by the image processing microcomputer 107, and stored in the shading correction table unit 113 of the memory unit 106. Remember me. Therefore, after reading from the memory unit 106, in the correction data decoding unit 105,
A′−F = A ′ − (A′−B ′) = B ′
A ′ × reverse compression = A, B ′ × reverse compression = B,
By doing so, the shading correction tables (left) A and (right) B can be generated. Thereby, the correction data can be returned to the original, and the left and right shading correction can be performed.

The capacity of the flash memory unit 108 for storing the shading correction table and the capacity occupied by the shading correction table in the memory unit 106 can be reduced by 47% using the following equation.
(A ′ + F) / (A + B) ≈1.97 MB / 3.75 MB≈53% (47% reduction)
Thereby, the capacity can be further reduced as compared with the first embodiment (25% reduction).

  The compression method may be either lossless compression or lossy compression. Whether to select reversible or irreversible compression method is decided after considering the features of the optical system including the image sensor and lens, and the functions and system requirements of the image processing recognition application. The

  In this embodiment, after the shading correction tables (left) A and (right) B are compressed, a compression shading correction table (left / right difference) F, which is the difference between them, is calculated and stored in the flash memory unit 108. In contrast, in the second embodiment, a shading correction table (left / right difference) C, which is the difference between the shading correction table (left) A and (right) B, is calculated and then compressed, and the compressed shading correction table (left / right difference) is calculated. C ′ is stored in the flash memory unit 108. In this way, whether the compression is performed first or later can be selected depending on the characteristics of the optical system.

  For example, when the features of the optical system are good, such as when the tolerance between the camera unit (left) 101 and the camera unit (right) 102 is small, similar data is obtained on the left and right. The amount of data can be reduced by calculating the difference and compressing it later. On the other hand, if the features of the optical system are poor, such as a large tolerance between the camera unit (left) 101 and the camera unit (right) 102, the difference is calculated after compression first as in this embodiment.

[Example 6]
FIG. 8 is a diagram illustrating a configuration example of a shading correction table in the sixth embodiment.
What is characteristic in this embodiment is that the shading correction table (left-right difference) C is divided into a plurality of areas, and the amount of correction data is set for each area.

  In the first embodiment, the amount of correction data is made constant over the entire surface of the shading correction table (left-right difference) C. When the optical characteristics represented by the left and right camera units, such as a stereo camera, and the lenses are substantially equal, the left and right shading correction tables have characteristics that are close to each other. (Left) A shading correction table (left / right difference) C, which is a difference value between A and the shading correction table (right) B, can be expressed by a sufficiently smaller number of bits than the original unsigned 12-bit / pixel. As described above, for example, it can be expressed by signed 6 bits.

The data amount of the shading correction table (left-right difference) C at this time is
C = A−B = 1280 × 1024 × 6 bits = 0.9375 MB
It became.

  In the present embodiment, the shading correction table (left / right difference) C is divided into a plurality of areas, and the amount of correction data is set by giving the number of bits for each area. Difference) The data amount of C as a whole is reduced, and the capacity of the flash memory unit 108 and the capacity occupied by the shading correction table in the memory unit 106 are reduced. For example, the total data amount of the shading correction table having the difference value can be set to 0. 0 of the first embodiment by changing the number of bits necessary for the difference data depending on the area from the characteristics of the optical performance represented by the lens. It can be reduced from 9375 MB.

  In the present embodiment, the shading correction table (left / right difference) C is divided into a central area C-4 arranged at the center of the image data, a left / right area C-2 arranged at the left and right of the central area C-4, and a central area. The area is divided into an upper and lower area C-3 disposed above and below C-4, and a left and right upper and lower oblique area C-1 disposed on both the left and right sides and obliquely above and below the central area C-4. Then, the number of data bits is increased in the order of the central area C-4, the upper and lower areas C-3, the left and right areas C-2, and the left and right upper and lower diagonal areas C-1, so that the amount of correction data is increased. .

  Since the center of the image data is bright, the difference between the camera unit (left) 101 and the camera unit (right) 102 is small, and the periphery of the image data is dark and easily affected by noise. The difference from the camera unit (right) 102 is large. Therefore, the number of bits at the center of the shading correction table (left-right difference) C can be made smaller than the number of bits around.

  Further, when the captured image is horizontally long, the difference is easily generated because the horizontal direction is longer than the vertical direction. Therefore, the number of bits around the vertical direction of the shading correction table (left-right difference) C can be made smaller than the number of bits around the horizontal direction.

  Thereby, for example, the shading correction table (left-right difference) C is divided into a plurality of areas C-4 to C-1 as described above, and the amount of correction data increases in the order of areas C-4 to C-1. By setting so as to be, the capacity of the flash memory unit 108 storing the shading correction table and the capacity occupied by the shading correction table in the memory unit 106 can be appropriately reduced.

  In this embodiment, the case where the shading correction table (left-right difference) C is divided into a plurality of areas has been described. However, the shading correction table D having the average value of the sums in the third and fourth embodiments and the average difference. It can also be applied to a shading correction table E having a value.

[Example 7]
FIG. 9 is a diagram illustrating a configuration example of a shading correction table in the seventh embodiment.
What is characteristic in this embodiment is that a representative value of correction data is set for each predetermined pixel size in the shading correction table (left-right difference) C.

  In the first embodiment, in the shading correction table (left-right difference) C, correction data that is set to be one-to-one for each pixel is set for each pixel of the captured image data. And, like a stereo camera, when the optical characteristics represented by the left and right camera units and the lenses are almost equal, the left and right shading correction tables are characterized by close values. A shading correction table (left-right difference C), which is a difference value between the table (left) A and the shading correction table (right) B, is expressed by a sufficiently smaller number of bits than the original unsigned 12-bit / pixel. For example, it has been described that it can be expressed in signed 6 bits.

The data amount of the shading correction table C at this time is
C = A−B = 1280 × 1024 × 6 bits = 0.9375 MB
It became.

  In this embodiment, paying attention to an arbitrary pixel size in the screen, when correction data at adjacent pixel coordinates are substantially the same numerical value, by setting a representative value to a predetermined pixel size, The total data amount of the shading correction table (left-right difference / representative value) G can be reduced from 0.9375 MB.

For example, if the correction data is one representative value for 2 × 2 pixels, the data amount is further reduced to ¼.
G = 0.9375MB × (1/4) = 0.23MB
It becomes.

Therefore, the capacity of the flash memory unit 108 for storing the shading correction table and the capacity occupied by the shading correction table in the memory unit 106 can be reduced by 44% according to the following equation.
(A + G) / (A + B) ≈2.11 MB / 3.75 MB≈56% (44% reduction)
Thereby, the capacity can be further reduced as compared with the first embodiment (25% reduction).

  In the present embodiment, the case where the shading correction table having the difference value is divided into a plurality of areas has been described, but the shading correction table having the average value of the sum in the third and fourth embodiments and the average value of the difference are used. The present invention can also be applied to a shading correction table having the same.

  Further, in the present embodiment, a case has been described in which correction data is uniformly set to one representative value per 2 × 2 pixels over the entire surface of the shading correction table (left / right difference) C, but the number of pixels depends on the position. May be changed. For example, in the center of the shading correction table (left-right difference) C, a lot of similar correction data is gathered, so one representative value for 3 × 3 pixels and one representative value for 2 × 2 pixels in the periphery. It is good.

  Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various designs can be made without departing from the spirit of the present invention described in the claims. It can be changed. For example, the above-described embodiment has been described in detail for easy understanding of the present invention, and is not necessarily limited to one having all the configurations described. Further, a part of the configuration of an embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of an embodiment. Furthermore, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

101 Camera section (left)
102 Camera (right)
103 Shading correction unit 104 Shading correction unit 105 Correction data decoding unit (correction table generation unit)
106 Memory unit 107 Image processing microcomputer 108 Flash memory unit 111, 112 Image data unit 113, 114 Shading correction table unit 121, 122 Image data 123, 124 Image data after shading correction 125 Captured image data 131, 132 Shading correction table 133, 134 Compression shading correction data 235 Shading table data 236 Shading table data 301 Captured image (image data before shading correction)
302 Shading correction table 303 Image data after shading correction

Claims (8)

A shading correction unit for correcting shading of image data captured by two camera units,
A memory unit for storing a correction table having a difference value or an average value between the correction table of one camera unit and the correction table of the other camera unit;
A correction table generating unit that generates at least one of the correction table of the one camera unit and the correction table of the other camera unit from the correction table having the difference value or the average value;
The shading correction unit performs shading correction on the image data using the correction table generated by the correction table generation unit ,
The in-vehicle camera characterized in that the correction table having the difference value or the average value represents the difference value or the average value with a smaller number of data bits than the correction table of one camera unit and the correction table of the other camera unit. Image processing device. The memory unit stores a correction table having the difference value and a correction table of the one camera unit,
The correction table generation unit generates the correction table of the other camera unit using the correction table having the difference value and the correction table of the one camera unit,
The shading correction unit performs shading correction on image data of the other camera unit using the correction table of the other camera unit generated by the correction table generation unit, and stores the one camera stored in the memory unit The in-vehicle camera image processing apparatus according to claim 1, wherein the image data of the one camera unit is subjected to shading correction using a correction table of the unit. The correction table of the one camera unit has correction data set for each pixel with respect to a plurality of pixels of the one camera unit,
The correction table having the difference value includes correction data set for each pixel of a plurality of pixels included in the one camera unit, and pixel-by-pixel correction for a plurality of pixels included in the other camera unit. Has the difference from the set correction data as correction data,
The correction table generation unit generates the correction table of the other camera unit by subtracting the correction data of the correction table having the difference value from the correction data of the correction table of the one camera unit. The in-vehicle camera image processing device according to claim 2, wherein The memory unit includes a correction table having an average value of a sum of the correction table of the one camera unit and the correction table of the other camera unit, and the correction table of the one camera unit and the correction of the other camera unit. Storing a correction table having an average value of differences from the table;
The correction table generation unit generates a correction table of the one camera unit and a correction table of the other camera unit using a correction table having the average value of the sum and a correction table having the average value of the difference,
2. The shading correction unit according to claim 1, wherein the image data is subjected to shading correction using the correction table of the one camera unit and the correction table of the other camera unit generated by the correction table generation unit. The on-vehicle camera image processing apparatus described. The correction table of the one camera unit has correction data set for each pixel with respect to a plurality of pixels of the one camera unit,
The correction table of the other camera unit has correction data set for each pixel with respect to a plurality of pixels of the other camera unit,
The correction table having the average value of the sum includes correction data set for each of the plurality of pixels of the one camera unit and one pixel for the plurality of pixels of the other camera unit. The correction data is a value obtained by dividing the sum of the correction data set for each time by 2 and
The correction table having the average value of the differences includes correction data set for each of the plurality of pixels of the one camera unit and one pixel for the plurality of pixels of the other camera unit. A value obtained by dividing the difference from the correction data set every time by 2 is provided as correction data.
The correction table generation unit generates the correction table of the one camera unit by adding the correction data of the correction table having the average value of the sum and the correction data of the correction table having the average value of the difference The correction table of the other camera unit is generated by subtracting the correction data of the correction table having the average value of the difference from the correction data of the correction table having the average value of the sum. Item 5. The on-vehicle camera image processing device according to Item 4. A shading correction unit for correcting shading of image data captured by two camera units,
A memory unit for storing a correction table having a difference value or an average value between the correction table of one camera unit and the correction table of the other camera unit;
A correction table generating unit that generates at least one of the correction table of the one camera unit and the correction table of the other camera unit from the correction table having the difference value or the average value;
The shading correction unit performs shading correction on the image data using the correction table generated by the correction table generation unit,
Correction table having the difference value or the average value, the image data for each divided area into a plurality of areas, car mounting a camera image processing apparatus, characterized in that the data amount is set in the correction data.   The correction table having the difference value or the average value includes a central area arranged at the center of the image data, left and right areas arranged at the left and right of the central area, and upper and lower areas arranged above and below the central area. , Is divided into left and right upper and lower diagonal areas arranged on both left and right sides and diagonally above and below the central area, and in the order of the central area, the upper and lower areas, the left and right areas, and the left and right upper and lower diagonal areas. The in-vehicle camera image processing apparatus according to claim 6, wherein the correction data is set to have a large data amount. A shading correction unit for correcting shading of image data captured by two camera units,
A memory unit for storing a correction table having a difference value or an average value between the correction table of one camera unit and the correction table of the other camera unit;
A correction table generating unit that generates at least one of the correction table of the one camera unit and the correction table of the other camera unit from the correction table having the difference value or the average value;
The shading correction unit performs shading correction on the image data using the correction table generated by the correction table generation unit,
Correction table having the difference value or the average value, the car mounting a camera image processing apparatus, characterized in that the representative value of the correction data for each predetermined pixel size is set.

Images