JP2018026619A - Imaging device and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device in which measurement accuracy of distance to a subject was improved.SOLUTION: An imaging device includes multiple imaging units consisting of a filter array where filters having the same characteristics or different characteristics are arranged two-dimensionally, and a pixel array where multiple pixels to which the light from the filter enters are arranged two-dimensionally, and generating an image obtained from multiple pixels, and a distance calculation unit for calculating the distance to a subject by matching between images generated by multiple imaging units. In the filter array of at least one imaging section, filters having the same characteristics in the search direction in matching are arranged side by side B, C.SELECTED DRAWING: Figure 6

Description

  The present disclosure relates to an imaging apparatus and an image processing method, and more particularly, to an imaging apparatus and an image processing method that can improve the accuracy of distance measurement to a subject.

  Conventionally, a stereo base depth measuring device has been proposed and put into practical use (see Patent Document 1). In stereo-based depth measurement, the distance error at 1-pixel disparity increases as the object moves away, and the distance measurement accuracy tends to decrease.

JP 2000-354257 A

So, trying to adjust the baseline,
1. When the baseline is narrow, disparity tends to be less likely to occur as the distance increases.
2. When the baseline is wide, the improvement is made for 1. However, the appearance of the subject between the viewpoints changes at short distances, matching becomes difficult, and the case where Depth cannot be obtained correctly tends to occur.
It was a trade-off relationship.

  Further, even when the pixel pitch is narrowed for the purpose of increasing the accuracy per pixel of the disparity, the cell area is narrowed and the pixel value becomes noisy, and the ranging accuracy may be similarly lowered.

  The present disclosure has been made in view of such a situation, and can improve the accuracy of ranging to a subject.

  An imaging device according to an aspect of the present technology includes a filter array in which filters having the same characteristics or different characteristics are arranged in a two-dimensional manner, and a plurality of pixels into which light from the filter is incident are arranged in a two-dimensional manner A plurality of imaging units that generate an image obtained from the pixel array, and a distance calculation unit that calculates a distance to a subject by matching between images generated by the plurality of imaging units. In the filter array of at least one of the imaging units, filters having the same characteristics are arranged in the search direction in the matching.

  When the plurality of imaging units are configured by two, the filter arrays of the two imaging units are arranged side by side with filters having the same characteristics in the search direction in the matching.

  When the plurality of imaging units are configured by three, the filter array of the reference imaging unit is arranged in Bayer, and the filter arrays of the two imaging units all have the same characteristics in the search direction in the matching. The filters are arranged side by side.

  When the plurality of imaging units are configured by three, the filter array of the reference imaging unit is arranged by changing the Bayer arrangement so that more filters having the same characteristics in the search direction in the matching are arranged. The filter arrays of the two imaging units are arranged side by side with filters having the same characteristics in the search direction in the matching.

  A filter array in which filters having the same characteristics are all arranged side by side in the search direction in the matching and a pixel array corresponding to the filter array are configured with high density only in the search direction in the matching.

  A filter array in which filters having all the same characteristics are arranged side by side in the search direction in the matching and a pixel array corresponding to the filter array are configured with low density in a direction orthogonal to the search direction in the matching.

  A filter array in which filters having the same characteristics are all arranged in the search direction in the matching and a pixel array corresponding to the filter array are arranged by rotating an anamorphic lens by 90 degrees with respect to a normal use form By doing so, it is possible to perform imaging with high-density spatial resolution only in the search direction in the matching.

  A filter array in which filters having all the same characteristics are arranged side by side in the search direction in the matching and a pixel array corresponding to the filter array are arranged in a staircase offset in a direction orthogonal to the search direction in the matching By doing so, it is possible to perform imaging with high-density spatial resolution only in the search direction in the matching.

  The filter array is a color filter array in which color filters having the same color characteristics or different color characteristics are two-dimensionally arranged.

  An image processing method according to one aspect of the present technology includes a filter array in which filters having the same characteristic or different characteristics are arranged in a two-dimensional manner, and a plurality of pixels into which light from the filter is incident are arranged in a two-dimensional manner. A plurality of imaging units that generate images obtained from the plurality of pixels, and at least one filter array of the imaging units is lined with filters having the same characteristics in the search direction in the matching. The distance calculation unit of the imaging devices arranged in (1) calculates the distance to the subject by matching between the images generated by the plurality of imaging units.

  In one aspect of the present technology, a filter array in which filters having the same characteristic or different characteristics are two-dimensionally arranged, and a pixel array in which a plurality of pixels to which light from the filter is incident are two-dimensionally arranged A plurality of imaging units that generate images obtained from the plurality of pixels, and at least one filter array of the imaging units is arranged side by side with filters having the same characteristics in the search direction in the matching Has been. Then, the distance to the subject is calculated by matching between the images generated by the plurality of imaging units.

  According to the present technology, it is possible to improve the accuracy of ranging to the subject.

  In addition, the effect described in this specification is an illustration to the last, and the effect of this technique is not limited to the effect described in this specification, and there may be an additional effect.

It is a block diagram showing an example of composition of an imaging device to which this art is applied. It is a figure which shows the arrangement | positioning form of the camera in stereo matching. It is a block diagram which shows the structural example of a calculating part. It is a flowchart explaining the ranging process of an imaging device. It is a figure which shows the arrangement | positioning form of the camera in binocular stereo matching. It is a figure which shows the color filter arrangement | sequence of the camera suitable for binocular stereo. It is a figure which shows the arrangement | positioning form in 3 eyes stereo matching. It is an example of the color filter arrangement | sequence (part) of the camera suitable for a three-eye stereo. It is another example of the color filter arrangement | sequence (part) of the camera suitable for a three-eye stereo. It is another example of the color filter arrangement | sequence (part) of the camera suitable for a three-eye stereo. It is a figure which shows the usage example of an Anamorphic lens. It is a figure which shows the usage example of an Anamorphic lens. It is a figure explaining the color filter arrangement | sequence which attached the offset and shifted the position. And FIG. 11 is a block diagram illustrating an example of a personal computer.

  Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described.

<Configuration example of imaging device>
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus to which the present technology is applied. In the imaging device 11 of FIG. 1, distance measurement is performed using stereo matching between images from a plurality of cameras.

  In the example of FIG. 1, the imaging device 11 is configured to include a camera image input unit 21, a calculation unit 22, and a storage unit 23.

  The camera image input unit 21 includes a plurality of cameras (sensors) such as an RGB camera and an infrared camera having an image sensor (sensor). The camera image input unit 21 images the subject M and inputs the captured image of the subject M to the calculation unit 22. When the camera image input unit 21 includes the camera 21L and the camera 21R, for example, the camera image input unit 21 is arranged as illustrated in FIG.

<Camera layout example>
In the example of FIG. 2, a camera arrangement form in basic stereo matching is shown. The camera 21L and the camera 21R are arranged so that the respective imaging surfaces for imaging the subject M are on the same plane. Baseline BL is the distance between the position where camera 21L is disposed and the position where camera 21R is disposed.

  The calculation unit 22 includes a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), and the like. As shown in FIG. 3, the calculation unit 22 includes a disparity calculation unit 31 and a depth value calculation unit 32. The disparity calculation unit 31 calculates disparity by performing stereo matching processing, that is, matching processing between images from the camera image input unit 21. The depth value calculation unit 32 calculates a depth value from the calculated disparity.

  The storage unit 23 includes a memory and a hard disk. The storage unit 23 stores various data by the calculation unit 22.

<Operation of imaging device>
Next, distance measurement processing of the imaging apparatus 11 will be described with reference to the flowchart of FIG. In step S <b> 11, the camera image input unit 21 captures the subject M, and inputs the captured image of the subject M to the Disparity calculation unit 31.

  In step S <b> 12, the disparity calculation unit 31 calculates disparity by performing matching processing between images from the camera image input unit 21. The disparity calculation unit 31 supplies the calculated disparity to the depth value calculation unit 32.

  In step S <b> 13, the depth value calculation unit 32 calculates the depth value from the disparity from the disparity calculation unit 31. In the imaging apparatus 11, the depth value obtained in this way is used for immediate distance, and the distance measurement process ends.

  Here, as shown in FIG. 5A and FIG. 5B, the calculation of the depth value in step S13 is based on the similarity of triangles using Depth, Disparity, Baseline BL, and Focal Length. It is obtained by 5A shows an example of the arrangement of the camera 21L and the camera 21R with respect to the subject M. In the example of FIG. 5B, the camera 21L and the camera 21R viewed from the imaging surface side are shown. .

なお、Depthは、被写体Ｍのカメラ２１Ｌ（カメラ２１Ｒ）からの奥行き方向の距離[mm]である。Disparityは、カメラ２１Ｌで撮影された左画像５１Ｌ上の被写体Ｍの位置の、画像５１Ｌの中心からの水平方向の距離から、カメラ２１Ｒで撮影された右画像上の被写体Ｍの位置の、画像５１Ｒの中心からの水平方向の距離を減算した値[Pix]である。ベースラインBLは、カメラ２１Ｌが配置される位置とカメラ２１Ｒが配置される位置の水平方向の距離[mm]、すなわち、カメラ間距離である。Focal Lengthは、カメラ２１Ｌ（カメラ２１Ｒ）の焦点距離[Pix]である。

Depth is the distance [mm] in the depth direction of the subject M from the camera 21L (camera 21R). Disparity is the image 51R of the position of the subject M on the right image captured by the camera 21R from the horizontal distance from the center of the image 51L of the position of the subject M on the left image 51L captured by the camera 21L. This is a value [Pix] obtained by subtracting the horizontal distance from the center of the. The base line BL is a horizontal distance [mm] between the position where the camera 21L is disposed and the position where the camera 21R is disposed, that is, the inter-camera distance. Focal Length is the focal length [Pix] of the camera 21L (camera 21R).

  When disparity is considered in units of pixels (pixels), the depth value resulting from the difference in disparity of 1 Pixl increases as the disparity decreases. Distance resolution / accuracy tends to deteriorate.

  Therefore, as a countermeasure for the reduction in ranging resolution, the variables in the above-described equation (1), that is, the disparity, the baseline BL, and the Focal Length are changed by changing the configuration of the camera and the sensor. Each of these variable changes may have a trade-off relationship with respect to the resolution improvement. Hereinafter, the adverse effects caused by the measures for improving the ranging resolution for each variable will be described in order.

  With respect to the baseline BL, by increasing the baseline BL distance, the area that can be imaged overlapping between cameras is narrowed, and in particular, there is a possibility that the short distance may not be captured in both cameras to be matched. Even when the image is reflected, the appearance of the subject between the viewpoints changes greatly at a short distance, and the matching accuracy becomes difficult to obtain.

  Regarding Focal Length, increasing the Focal Length may narrow the viewing angle and narrow the image frame that can be taken.

  As for the disparity, disparity is likely to occur even at a long distance by narrowing the pixel pitch of the sensor, but the cell area becomes small and narrow, so that the pixel value becomes noisy and matching accuracy decreases. As a result, the Depth value of the output result also becomes noisy, and the reliability tends to decrease.

  On the other hand, in the present technology, the array of the sensor color filters in units of cells is in a state where RGB is arranged in one row. Thereby, ranging resolution can be improved. Hereinafter, the present technology will be specifically described.

<Color filter arrangement example>
6 shows an example of a part of a color filter (pixel) array of a camera suitable for a binocular stereo in which a camera 21L and a camera 21R are arranged on the left and right as shown in FIGS. Note that, in the example of FIG. 6, when viewed from the imaging surface side of the imaging element of the camera, a corresponding pixel array (array) is arranged under the color filter array (array). In the example of FIG. 6, the horizontal is the disparity search direction (stereo matching scan direction) in stereo matching. Further, FIG. 6A shows an example in which the color filter (pixel) array is a Bayer array for comparison with the color filter (pixel) array of the present technology described in FIGS. 6B and 6C.

  As shown in FIG. 6A, in the Bayer arrangement, the color filters are arranged in the first row from the top, R (Red), G (Green), R, G,..., The second row, G, B (Blue), G, B,..., 3rd stage, R, G,..., 4th stage, G, B,.

  On the other hand, as shown in FIG. 6B, the color filter of the present technology has the first stage, R, R, R, R..., The second stage, B, B, B,. , G, G,...

  In this way, by aligning the color filter array in the disparity search direction (by arranging color filters of the same color), when matching for each color, the color filter is compared with the conventional color filter array as shown in FIG. 6A. Each party resolution improves.

  In the example of FIG. 6, the example in which the first stage R, the second stage G, and the third stage B are described has been described. It is good.

  In the example of FIG. 6C, the arrangement of the color filters is the same as that of the color filter of the example of FIG. 6B, but the pixel density is increased (higher resolution) only in the disparity search direction. Thereby, the resolution can be further improved.

  Here, when the camera image input unit 21 includes the camera 21L, the camera 21R, and the camera 21T, for example, they are arranged as shown in FIG.

<Camera layout example>
In the example of FIG. 7, a camera arrangement form in the case of a three-lens stereo that performs stereo matching in the vertical direction in addition to the left and right is shown. For example, in this case, the camera image input unit 21 includes a camera 21L, a camera 21R, and a camera 21T, and the camera 21T is disposed on the camera 21L.

  In the example of FIG. 7A, an example of the arrangement of the camera 21L and the camera 21R with respect to the subject M and the camera 21T is shown. In the example of FIG. 7B, a camera 21L, a camera 21R, and a camera 21T viewed from the imaging surface side are shown. The camera 21T is disposed on the camera 21L with reference to the image of the camera 21L.

<Color filter arrangement example>
In the example of FIG. 8, the camera 21L and the camera 21R are arranged on the left and right as shown in FIG. 7, and the color filter (pixel) arrangement of the camera suitable for the three-eye stereo when the camera 21T is arranged on the camera 21L. Some are shown. Note that, in the vertical arrangement of the camera 21T and the camera 21L as viewed from the imaging surface side of the camera imaging element, the vertical is the disparity search direction of stereo matching, and in the horizontal arrangement of the camera 21L and the camera 21R, the horizontal Is the disparity search direction of stereo matching.

  In this case, the color filter array is a Bayer array only for the reference camera 21L, and the vertical matching target camera 21T and the horizontal matching target camera 21R are respectively high only in the corresponding vertical / horizontal disparity search direction. The resolution is an array in which RGB is arranged for each line along the Disparity search direction.

  That is, in the camera 21L, the color filters are in the first row from the top, R (Red), G (Green), R, G,..., The second row, G, B (Blue), G, B,. The rows are arranged in the order of R, G,..., 4th row, G, B,.

  In the camera 21R, the color filters are arranged in the first row from the top, R, R, R, R ..., the second row, B, B, B, ..., so that the same color filters are arranged in the horizontal disparity search direction. The third row, G, G, G,..., The fourth row, B, B, B,.

  In the camera 21T, the color filters are arranged in the first column from the left, R, R, R, R ..., the second column, B, B, B, ..., so that the same color filters are arranged in the vertical Disparity search direction. The third row, G, G, G,..., The fourth row, B, B, B,.

  By arranging in this way, the stereo matching accuracy can be improved, and at the same time, the reference camera (camera 21L) can output an RGB image with a normal picture quality.

  As another example, when the quality of photographic image quality is not so high, as shown in FIGS. 9 and 10, the reference camera (camera 21L) also has pixels of each color as much as possible in the RGB matching direction. By using an array with a large number, it is possible to further increase the resolution.

  In the example of FIG. 9, an example in which red is prioritized is shown. For the camera 21L in FIG. 9, G in the Bayer array of the camera 21L in FIG. 8 is replaced with R in the second and fourth pixels from the left in the first row. In the first pixel from the left in the second and fourth stages, G in the Bayer array of the camera 21L in FIG. In the third pixel from the left in the third row, R in the Bayer array of the camera 21L in FIG.

  In the example of FIG. 10, an example in which green is given priority from the example of FIG. 9 is shown. For the camera 21L in FIG. 10, R in the array of the camera 21L in FIG. 9 is replaced with G in the third pixel from the left in the first row. In the first pixel from the left in the third row, R in the array of the cameras 21L in FIG.

  Furthermore, it is possible to improve the pixel resolution in the disparity search direction using an Anamorphic lens.

<Usage example of Anamorphic lens>
As shown in FIG. 11, the Anamorphic lens 101 originally displays image frame information 111 that is assumed to be wide (= horizontal) screen irradiation in movie shooting, and is narrow (in a normal size such as 4: 3 or 35 mm). This is an optical system designed for photographing and storing on a film (sensor) 112.

  Here, the resolution in the disparity search direction of stereo matching is increased by utilizing the point that the resolution in the spatial direction can be fluctuated vertically and horizontally. Specifically, as shown in FIG. 12, by placing the Anamorphic lens 101 rotated 90 degrees, an image 122 captured by the Anamorphic lens 101 is vertically compared to an image 121 captured by a normal lens. The direction is input in an optically compressed state. As a result, even in a normal rectangular sensor array, it is possible to capture an image with a high resolution only in the horizontal direction, which is the disparity search direction.

  In addition, even when the Anamorphic lens is not used, there is a method in which the color filters (sensors) are arranged with an offset to shift the spatial resolution in the vertical direction as shown in FIG.

  In the example of FIG. 13, in order from the top, the first stage is offset by R, R, R,..., The second stage is offset by about 1/5 of the horizontal size of the pixel, and R, R, R,. , The third row is further offset by about 1/5 of the horizontal size, R, R, R, ..., the fourth row is G, G, G, ..., the second row is 1 / of the horizontal size G, G, G,..., The third stage is further offset by about 1/5 of the lateral size, and is arranged as G, G, G,.

  In this way, the matching resolution in the disparity search direction (horizontal direction) can also be increased by providing an offset in the vertical direction and shifting the position.

  In the above description, the arrangement of RGB pixels has been described as an example. However, in the case of invisible light such as IR, a wide variety of spectral filters, and a single-color gray scale, it can also be applied to cases including a multi-directional deflection filter. Is possible.

  In the above description, the example in which the color filter array is associated with the pixel array has been described. However, as long as the color is the same in one column, one filter may be used for a plurality of pixels in the same column.

  As described above, according to the present technology, it is possible to improve Depth accuracy required for a stereo base. In particular, when the baseline is narrow and disparity is unlikely to occur, the accuracy of distance measurement can be improved when the object is at a long distance and the disparity of one pixel has a large effect on the distance measurement result.

<Personal computer>
The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes a computer incorporated in dedicated hardware, a general-purpose personal computer capable of executing various functions by installing various programs, and the like.

  FIG. 14 is a block diagram showing a hardware configuration example of a personal computer that executes the above-described series of processing by a program.

  In the personal computer 500, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503 are connected to each other by a bus 504.

  An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a storage unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

  The input unit 506 includes a keyboard, a mouse, a microphone, and the like. The output unit 507 includes a display, a speaker, and the like. The storage unit 508 includes a hard disk, a nonvolatile memory, and the like. The communication unit 509 includes a network interface or the like. The drive 510 drives a removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  In the personal computer 500 configured as described above, the CPU 501 loads, for example, a program stored in the storage unit 508 to the RAM 503 via the input / output interface 505 and the bus 504 and executes the program. Thereby, the series of processes described above are performed.

  The program executed by the computer (CPU 501) can be provided by being recorded on the removable medium 511. The removable medium 511 is a package made of, for example, a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.), a magneto-optical disk, or a semiconductor memory. Media. Alternatively, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  In the computer, the program can be installed in the storage unit 508 via the input / output interface 505 by attaching the removable medium 511 to the drive 510. Further, the program can be received by the communication unit 509 via a wired or wireless transmission medium and installed in the storage unit 508. In addition, the program can be installed in the ROM 502 or the storage unit 508 in advance.

  Note that the program executed by the computer may be a program that is processed in time series in the order described in this specification, or in a necessary stage such as in parallel or when a call is made. It may be a program for processing.

  Further, in the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus composed of a plurality of devices (apparatuses).

  For example, the present disclosure may take a cloud computing configuration in which one function is shared and processed by a plurality of devices via a network.

  In addition, in the above description, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configurations described above as a plurality of devices (or processing units) may be combined into a single device (or processing unit). Of course, a configuration other than that described above may be added to the configuration of each device (or each processing unit). Furthermore, if the configuration and operation of the entire system are substantially the same, a part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or other processing unit). . That is, the present technology is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present technology.

  The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present disclosure belongs can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present disclosure.

In addition, this technique can also take the following structures.
(1) a filter array in which filters having the same characteristic or different characteristics are two-dimensionally arranged, and a pixel array in which a plurality of pixels into which light from the filter is incident are two-dimensionally arranged, A plurality of imaging units for generating an image obtained from a plurality of pixels;
A distance calculation unit that calculates a distance to a subject by matching between images generated by the plurality of imaging units;
The filter array of at least one of the imaging units includes an array of filters having the same characteristics in the search direction in the matching.
(2) When the plurality of imaging units are configured by two,
The imaging device according to (1), wherein the filter arrays of the two imaging units are arranged side by side with filters having the same characteristics in the search direction in the matching.
(3) When the plurality of imaging units are configured by three,
The filter array of the reference imaging unit is arranged in Bayer,
The imaging device according to (1), wherein the filter arrays of the two imaging units are arranged side by side with filters having the same characteristics in the search direction in the matching.
(4) When the plurality of imaging units are configured by three,
The filter array of the imaging unit serving as a reference is arranged by changing the Bayer arrangement so that there are many filters having the same characteristics in the search direction in the matching,
The imaging device according to (1), wherein the filter arrays of the two imaging units are arranged side by side with filters having the same characteristics in the search direction in the matching.
(5) A filter array in which filters having all the same characteristics are arranged side by side in the search direction in the matching and a pixel array corresponding to the filter array are configured with high density only in the search direction in the matching. The imaging device according to any one of (1) to (4).
(6) A filter array in which filters having all the same characteristics are arranged side by side in the search direction in the matching and a pixel array corresponding to the filter array are configured with low density in a direction orthogonal to the search direction in the matching. The imaging apparatus according to (5).
(7) A filter array in which filters having all the same characteristics are arranged in the search direction in the matching and a pixel array corresponding to the filter array rotate the anamorphic lens by 90 degrees with respect to a normal use form The imaging apparatus according to any one of (1) to (6), wherein imaging is performed with high-density spatial resolution only in the search direction in the matching by adopting the arrangement described above.
(8) A filter array in which filters having all the same characteristics are arranged side by side in the search direction in the matching and a pixel array corresponding to the filter array are offset stepwise in a direction perpendicular to the search direction in the matching. The imaging apparatus according to any one of (1) to (6), wherein imaging is performed with high-density spatial resolution only in the search direction in the matching.
(9) The imaging device according to any one of (1) to (8), wherein the filter array is a color filter array in which color filters having the same color characteristics or different color characteristics are two-dimensionally arranged. .
(10) A filter array in which filters having the same characteristic or different characteristics are two-dimensionally arranged, and a pixel array in which a plurality of pixels into which light from the filter is incident are two-dimensionally arranged, An imaging apparatus comprising a plurality of imaging units that generate images obtained from a plurality of pixels, wherein at least one filter array of the imaging units is arranged side by side with filters having the same characteristics in the search direction in the matching An image processing method in which a distance calculation unit calculates a distance to a subject by matching between images generated by the plurality of imaging units.

  DESCRIPTION OF SYMBOLS 11 Imaging device, 21 Camera image input part, 21L, 21R, 21T Camera, 22 Calculation part, 23 Storage part, 31 Disparity calculation part, 32 Depth value calculation part, 51, 51R, 51L image, 101 Anamorphic lens, 111 Image frame Information, 112 films, 121, 122 images

Claims (10)

A plurality of pixels each having a filter array in which filters having the same characteristics or different characteristics are two-dimensionally arranged, and a pixel array in which a plurality of pixels to which light from the filter is incident are two-dimensionally arranged A plurality of imaging units for generating images obtained from
A distance calculation unit that calculates a distance to a subject by matching between images generated by the plurality of imaging units;
The filter array of at least one of the imaging units includes an array of filters having the same characteristics in the search direction in the matching. When the plurality of imaging units are configured by two,
The imaging apparatus according to claim 1, wherein the filter arrays of the two imaging units are arranged side by side with filters having the same characteristics in the search direction in the matching. When the plurality of imaging units are configured by three,
The filter array of the reference imaging unit is arranged in Bayer,
The imaging apparatus according to claim 1, wherein the filter arrays of the two imaging units are arranged side by side with filters having the same characteristics in the search direction in the matching. When the plurality of imaging units are configured by three,
The filter array of the imaging unit serving as a reference is arranged by changing the Bayer arrangement so that there are many filters having the same characteristics in the search direction in the matching,
The imaging apparatus according to claim 1, wherein the filter arrays of the two imaging units are arranged side by side with filters having the same characteristics in the search direction in the matching. The filter array in which filters having the same characteristics are all arranged side by side in the search direction in the matching and the pixel array corresponding to the filter array are configured with high density only in the search direction in the matching. The imaging device described. A filter array in which filters having all the same characteristics are arranged side by side in the search direction in the matching and a pixel array corresponding to the filter array are configured with low density in a direction orthogonal to the search direction in the matching. Item 6. The imaging device according to Item 5. A filter array in which filters having the same characteristics are all arranged in the search direction in the matching and a pixel array corresponding to the filter array are arranged by rotating an anamorphic lens by 90 degrees with respect to a normal use form The imaging apparatus according to claim 1, wherein imaging with high-density spatial resolution is performed only in a search direction in the matching. A filter array in which filters having all the same characteristics are arranged side by side in the search direction in the matching and a pixel array corresponding to the filter array are arranged in a staircase offset in a direction orthogonal to the search direction in the matching The imaging apparatus according to claim 1, wherein imaging with high-density spatial resolution is performed only in a search direction in the matching. The imaging apparatus according to claim 1, wherein the filter array is a color filter array in which color filters having the same color characteristics or color characteristics are two-dimensionally arranged. A plurality of pixels each having a filter array in which filters having the same characteristics or different characteristics are two-dimensionally arranged, and a pixel array in which a plurality of pixels to which light from the filter is incident are two-dimensionally arranged A distance calculation unit of an imaging apparatus including a plurality of imaging units that generate an image obtained from the filter, wherein at least one filter array of the imaging unit is arranged side by side with filters having the same characteristics in the search direction in the matching An image processing method for calculating a distance to a subject by matching between images generated by the plurality of imaging units.

Images