JP5896090B1 - Imaging apparatus and colorimetric method - Google Patents

Abstract

The imaging device (10) includes a first imaging unit (1) that images a subject, a second imaging unit (2) that is located outside the first imaging unit (1) and images a subject, and a measurement unit. A distance unit (3) and a colorimetric unit (4) are provided. The distance measuring unit (3) is configured to receive a monocular image of the subject imaged by the imaging device (23) of the second imaging unit (2) and a reference filter of the first imaging unit (1). The distance to the subject is measured using the single-eye image of the subject imaged in 13). The color measurement unit (4) includes a distance measured by the distance measurement unit (3), the single-eye image captured by the image sensor (13) via a reference filter, and a filter other than the reference filter. A colorimetric value of the subject is obtained for a wavelength that passes through a filter other than the reference filter, using another single-eye image captured via the.

Description

  The present invention relates to an imaging device including an imaging unit that images a subject via a plurality of imaging optical systems arranged side by side in a direction perpendicular to an optical axis, and a subject colorimetry method using the imaging device. Is.

  In a compound eye imaging device that images a subject using a lens array in which a minute optical lens is two-dimensionally arranged and a solid-state imaging device, an image of the subject (each individual-eye image is formed on the light-receiving surface of the solid-state imaging device by each optical lens. ) Is formed. Since each optical lens is two-dimensionally arranged, each single-eye image is an image (an image having parallax) viewed from a slightly different angle with respect to the subject. Therefore, the compound eye image obtained by collecting each single-eye image includes information on the three-dimensional shape of the subject and information for increasing the resolution. Therefore, by using a large number of individual images slightly different from each other, it is possible to generate a reconstructed image based on a larger amount of information, and it is possible to obtain a reconstructed image having a higher definition than the resolution of each individual eye image. .

  In particular, in the imaging device of Patent Document 1, an optical filter having a predetermined wavelength transmission characteristic is disposed corresponding to at least a part of the micro optical lens of the lens array, and the wavelength is set in any two types of optical filters. It has a correlation. In addition, said correlation means that the wavelength or wavelength band which permeate | transmits in common exists. In this way, by giving correlation to any two types of optical filters, at least a part common to the single-eye images captured through the two types of optical filters appears. For this reason, the correlation between the single-eye images is increased, and it is possible to perform alignment by image matching with high accuracy, and it is possible to estimate a distance closer to the true value as the distance to the subject. ing.

JP, 2011-182237, A (refer to claim 1, paragraphs [0003], [0005], [0016] to [0018], [0077] to [0079], FIG. 1, FIG. 2, FIG. 19, FIG. 20, etc.) )

  However, in Patent Document 1, distance measurement is performed using images captured through two optical lenses constituting each individual eye of the lens array. In this case, since the distance (base line length) between the two optical lenses of the lens array is short, there arises a problem that the ranging accuracy is lowered. The distance between the two optical lenses of the lens array is determined in advance so as to be as short as possible from the viewpoint of miniaturization of the apparatus. Increasing the distance between the optical lenses in order to improve distance measurement accuracy and the like is not desirable because the design of the lens array needs to be changed and the cost increases. Further, if the distance between the two optical lenses is increased, the entire lens array becomes large, and the apparatus cannot be reduced in size. On the other hand, if the distance measurement accuracy decreases because the distance between the two optical lenses is short, the error in the measured distance increases. As a result, the accuracy of colorimetry at any point on the subject located at a distance measured from the imaging device also decreases.

  The present invention has been made in order to solve the above-described problems, and its purpose is to measure distance accuracy without changing the distance between a plurality of lenses (imaging optical systems) for forming individual eye images. Therefore, an object of the present invention is to provide an imaging apparatus capable of improving the accuracy of colorimetry, and a method for measuring the color of a subject using the imaging apparatus.

  An imaging apparatus according to an aspect of the present invention is imaged by a first imaging unit that images a subject, a second imaging unit that images the subject, the first imaging unit, and the second imaging unit. A distance measuring unit that measures the distance to the subject using each of the images of the subject and a color measuring unit that obtains a colorimetric value of the subject using the distance measured by the distance measuring unit. The first imaging unit is arranged side by side in a direction perpendicular to the optical axis, and corresponds to a plurality of first imaging optical systems having the same field of view, and the plurality of first imaging optical systems And a plurality of first optical filters having a plurality of spectral characteristics and a plurality of the first imaging optical system and the plurality of first imaging filters that are in a corresponding positional relationship. A first imaging element having an imaging region, and the second imaging unit includes: The second imaging optical system and the second optical filter, and the second imaging element that images the subject through the second imaging optical system and the second optical filter. The imaging unit is located outside the first imaging unit, and the second optical filter uses one of the plurality of first optical filters as a reference filter and has substantially the same spectrum as the reference filter. The distance measuring unit includes a monocular image of the subject imaged by the second image sensor and the subject imaged by the first image sensor via the reference filter. The distance to the subject is measured using the single-eye image, and the color measurement unit uses the distance measured by the distance measurement unit and the reference filter using the first image sensor. The single-eye image of the subject imaged through Using the other single-eye image of the subject imaged through the first optical filter other than the reference filter, color measurement of the subject with respect to a wavelength that passes through the first optical filter other than the reference filter Find the value.

  A subject colorimetric method according to another aspect of the present invention is a first imaging unit that images a desired subject, and is arranged side by side in a direction perpendicular to the optical axis, and has the same field of view, A plurality of first imaging optical systems that respectively form images, a plurality of first optical filters that are arranged corresponding to the plurality of first imaging optical systems and have a plurality of spectral characteristics, and corresponding positional relationships The first imaging optical system and the first imaging device having a plurality of imaging regions that respectively capture images of the subject formed through the first optical filter. A colorimetric method for obtaining a colorimetric value of a subject using an imaging device comprising a unit, wherein the imaging device is further a second imaging unit located outside the first imaging unit, A second imaging optical system for forming an image of the subject; And a second imaging element that images the subject image formed through the second imaging optical system and the second optical filter, and the second optical filter includes: The second imaging unit having substantially the same spectral characteristics as the reference filter, using any one of the plurality of first optical filters as a reference filter. The distance to the subject is measured using a monocular image of the subject imaged by the imaging device and a single-eye image of the subject imaged by the first imaging device via the reference filter. The measured distance, the single image of the subject imaged by the first image sensor via the reference filter, and the first optical filter other than the reference filter. Imaged by By using the other ommatidium images Utsushitai, the wavelengths transmitted through the first optical filter other than the reference filter and a step of obtaining a colorimetric value of the subject.

  The accuracy of distance measurement and color measurement can be improved without changing the distance between the imaging optical systems of the first imaging unit.

It is explanatory drawing which shows typically the structure of the outline of the imaging device which concerns on one Embodiment of this invention. It is explanatory drawing which shows the simple structure of the said imaging device. It is a graph which shows an example of the spectral characteristic of each filter used for the 1st image pick-up part of the above-mentioned image pick-up device. FIG. 3 is an explanatory diagram schematically showing a single-eye image (standard image) of a subject imaged by the first imaging unit and a monocular image (reference image) of a subject imaged by the second imaging unit 2; is there. It is explanatory drawing which shows the positional relationship of the attention point in space, the principal point of the lens of one camera, and the principal point of the lens of the other camera. It is explanatory drawing which shows the relationship between the position on an epipolar line, and a coincidence degree. It is explanatory drawing which shows the method of calculating | requiring a corresponding point from the approximation by a quadratic function. It is explanatory drawing which shows typically the positional relationship of a to-be-photographed object and each part of the said imaging device. It is a flowchart which shows the flow of operation | movement in the said imaging device. It is a graph which shows the other example of the spectral characteristic of each filter of a said 1st imaging part. It is explanatory drawing which shows the structure of the imaging device which concerns on other embodiment of this invention. It is a graph which shows the spectral characteristic of each filter used for the above-mentioned imaging device.

[Embodiment 1]
An embodiment of the present invention will be described below with reference to the drawings.

(Schematic configuration of the imaging device)
FIG. 1 is an explanatory diagram schematically illustrating a schematic configuration of an imaging apparatus 10 according to the present embodiment. The imaging device 10 includes a first imaging unit 1, a second imaging unit 2, a distance measuring unit 3, and a color measuring unit 4. The first imaging unit 1 and the second imaging unit 2 are for imaging a subject. The second imaging unit 2 is located outside the first imaging unit 1. In the present embodiment, the second imaging unit 2 is located away from the first imaging unit 1, but may be in contact with the first imaging unit 1.

  In the first imaging unit 1, for example, filters that transmit light of each color of red (R), green (G), blue (B), yellow (Y), magenta (M), and cyan (C) are arranged. An object is imaged through a filter and an imaging optical system (for example, a lens) in each area. The type of filter is not limited to the above type. The second imaging unit 2 has a region where a filter that transmits G light, for example, is arranged, and images a subject via this filter and the imaging optical system.

  The distance measuring unit 3 captures an image of a subject captured through a reference filter (for example, a G filter) of the first image capturing unit 1 and a filter of the second image capturing unit 2 (for example, a G filter). The distance to the subject is measured (calculated) using the image of the subject. The color measuring unit 4 uses the distance to the subject measured by the distance measuring unit 3 and each image of the subject imaged through each filter of the first imaging unit 1 to obtain a colorimetric value of the subject. Ask. The distance measuring unit 3 and the color measuring unit 4 can be configured by an arithmetic circuit that performs the predetermined processing (calculation) described above.

  FIG. 2 is an explanatory diagram illustrating a simple configuration of the imaging apparatus 10. In the figure, the first imaging unit 1 is simply configured with three individual eyes (cameras). The imaging device 10 further includes an overall control unit 5, a storage unit 6, and an output unit 7 in addition to the first imaging unit 1, the second imaging unit 2, the distance measuring unit 3, and the color measuring unit 4 described above. Yes.

  The overall control unit 5 controls the operation of each unit of the imaging apparatus 10, and includes, for example, a CPU (Central Processing Unit) and its peripheral circuits. In FIG. 2, the control signal of the overall control unit 5 is indicated by a dashed arrow, and the flow of various data is indicated by a solid arrow (the same applies to other drawings).

  The storage unit 6 stores the distance to the subject measured by the distance measuring unit 3, the colorimetric value Z of the subject obtained by the colorimetric unit 4, and the color image of the subject obtained from the colorimetric value (R / G / B) is a memory for storing information (data), and is composed of, for example, a volatile storage element such as a RAM (Random Access Memory) or a large-capacity storage device such as a hard disk.

  The output unit 7 outputs information (distance to the subject, colorimetric values) stored in the storage unit 6 and a color image of the subject obtained based on the colorimetric values. For example, the CRT display, LCD, organic It comprises a display device such as an EL display and a printing device such as a printer. Note that the output unit 7 may be provided outside the imaging device 10. That is, the information stored in the storage unit 6 may be transmitted to the output unit 7 outside the imaging device 10 by wire or wirelessly, and a color image or the like of the subject may be displayed or printed there.

  Next, details of main parts of the imaging apparatus 10 will be described.

(First imaging unit)
The first imaging unit 1 includes a plurality of single-lens lenses 11 (first imaging optical system), a plurality of filters 12 (first optical filters), and an imaging element 13 (first imaging element). doing. In FIG. 2, the number of the individual lenses 11 and the filters 12 is three, but may be two or four or more. The individual lenses 11 are arranged side by side in a direction perpendicular to the optical axis, but may be arranged in an array (in a matrix).

  One single lens 11 is composed of one or a plurality of optical lenses arranged along the optical axis. The individual lenses 11 have the same field of view, and are arranged side by side in a direction parallel to the imaging region (light receiving surface) of the imaging device 13 so that the optical axes are substantially parallel to each other. For convenience of the following description, a single lens 11 in which a filter 12R (described later) is positioned on the optical axis is referred to as a single lens 11R, and a single lens 11 in which the filter 12G is positioned on the optical axis is referred to as a single lens. The single lens 11 that is referred to as 11G and in which the filter 12B is located on the optical axis is referred to as a single lens 11B.

  The plurality of filters 12 are arranged side by side corresponding to each individual lens 11 on the light incident side of the plurality of individual lenses 11 and have a plurality of spectral characteristics. The plurality of filters 12 may be disposed on the light emission side of the plurality of single-lens lenses 11.

  FIG. 3 shows an example of the spectral characteristics of each filter 12. Each filter 12 has a spectral characteristic having a transmission wavelength peak in any wavelength region of RGB. Here, for convenience, the filter 12 having a transmission wavelength peak in the R wavelength region is referred to as a filter 12R, and the filter 12 having a transmission wavelength peak in the G wavelength region is referred to as a filter 12G. The filter 12 having a transmission wavelength peak is referred to as a filter 12B.

  As shown in FIG. 3, the filters 12R, 12G, and 12B have discrete spectral characteristics. Here, “discrete spectral characteristics” means that the graphs showing the spectral characteristics of each other are completely separated, and even if there are overlapping wavelength ranges, the transmission in the overlapping wavelength range The rate is 30% or less (preferably 20% or less) when the transmittance peak is 100%.

  The imaging element 13 is configured by a monochrome image sensor having a plurality of imaging areas for imaging a subject via a single lens 11 and a filter 12 that are in a corresponding positional relationship. As the image sensor, an image sensor having a plurality of pixels such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) can be used.

Here, in the imaging device 13, an imaging region in which the subject is imaged through the single lens 11R and the filter 12R is defined as an imaging region 13R, and an imaging region in which the subject is imaged through the single lens 11G and the filter 12G is imaged. An imaging region where the subject is imaged via the single lens 11B and the filter 12B is defined as an imaging region 13B. Each of the imaging regions 13R, 13G, and 13B outputs an electrical signal corresponding to the amount of received light, but the output is A / D converted, and data of the captured image is, for example, 8 bits of 0 to 255 data. Obtained for each of RGB. In FIG. 2, the image data output from the image sensor 13 is denoted by I ₁ (R), I ₁ (G), and I ₁ (B).

  The single lens 11R, the filter 12R, and the imaging region 13R constitute one single eye (camera), and the single lens 11G, the filter 12G, and the imaging region 13G constitute one single eye. One eye is constituted by the eye lens 11B, the filter 12B, and the imaging region 13B. From each individual eye, image data of the same subject that is shifted by substantially parallax is output.

(Second imaging unit)
The second imaging unit 2 includes a monocular lens 21 (second imaging optical system), a filter 22 (second optical filter), and an imaging element 23 (second imaging element).

  One monocular lens 21 is composed of one or a plurality of optical lenses arranged along the optical axis. The optical axis of the monocular lens 21 is substantially parallel to the optical axes of the individual lenses 11 of the first imaging unit 1.

  The filter 22 is disposed on the light incident side of the monocular lens 21, but may be disposed on the light emitting side. The filter 22 has one of the filters 12 of the first imaging unit 1 as a reference filter and has substantially the same spectral characteristics as the reference filter. The “substantially the same spectral characteristics” includes the case where the spectral characteristics are completely the same, and even if the spectral characteristics are not completely the same, the shift amount of the spectral characteristics (for example, the shift amount of the transmission peak wavelength) is This includes cases where the thickness is 30 nm or less. Here, the filter 22 has the same spectral characteristics as the filter 12G with the filter 12G as a reference filter. The reason why the filter 12G is used as a reference filter is that G is a central wavelength of visibility. The filter 12R or the filter 12B may be used as a reference filter, and the spectral characteristics of the filter 22 may be combined with the spectral characteristics of the filter 12R or the filter 12B. Thus, by using a monocular image and a single-eye image captured through the filter 22 and the reference filter having substantially the same spectral characteristics, the distance to the subject can be accurately measured.

The imaging element 23 is configured by a monochrome image sensor having an imaging area for imaging a subject via a monocular lens 21 and a filter 22. As the image sensor, an image sensor having a plurality of pixels such as a CCD and a CMOS can be used as in the image sensor 13. An electrical signal corresponding to the amount of received light is output from the imaging region. The output is A / D converted, and data of the captured image is, for example, 8-bit data of 0 to 255 transmitted through the filter 22. It is obtained for the color of the wavelength (here, the same G color as the transmission wavelength of the reference filter). In FIG. 2, the image data output from the image sensor 23 is indicated by I ₂ (G).

  Here, the spatial resolution of the second imaging unit 2 is determined by the number of pixels of the imaging device 23 and the focal length of the monocular lens 21, and the spatial resolution of each individual eye of the first imaging unit 1 is the pixel of the imaging device 13. It is determined by the number and the focal length of each individual lens 11. In the present embodiment, by adjusting the pixel pitch of the image sensors 13 and 23 and the focal length of each lens (monocular lens 21 and single lens 11), the spatial resolution of the second imaging unit 2 and the first The spatial resolution for each individual eye of the imaging unit 1 is made substantially the same. If the first imaging unit 1 and the second imaging unit 2 have different spatial resolutions, the accuracy in the distance measurement by the distance measuring unit 3 and the color measurement by the color measuring unit 4, which will be described later, is on the side where the spatial resolution is coarse. Since the rate is limited, it is possible to avoid a decrease in distance measurement accuracy and color measurement accuracy due to a difference in spatial resolution by aligning the spatial resolution as described above.

  In the present embodiment, the internal parameters in the first imaging unit 1 and the second imaging unit 2 are slightly different from the design, and thus are obtained in advance by a known method. The internal parameters include, for example, the focal length of the lens (single-lens 11 and monocular lens 21), and the center of the captured image (intersection with the lens optical axis) in the image sensors 13 and 23.

  In addition, in the second image pickup unit 2 and at least one single eye of the first image pickup unit 1 (particularly, the reference single eye that picks up the subject through the reference filter), the external parameters slightly deviate from the design. Obtain in advance by a known method. The external parameters include, for example, the posture (the direction of the optical axis and the position in the optical axis direction) between the second imaging unit 2 and the reference single eye of the first imaging unit 1. Furthermore, in the first imaging unit 1, the external parameters (the direction of the optical axis and the position in the optical axis direction) are obtained in advance by a known method even between the reference single eye and the other single eye.

  The calculation method of the internal parameter and the external parameter described above is described in, for example, “A Flexible New Technique for Camera Calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, VOL.22, No.11, 1330-1334, November 2000”. The method can be adopted.

  Further, in the present embodiment, the overall control unit 5 described above is captured by the second imaging unit 2 through the subject image (monocular image) and the first imaging unit 1 through the reference filter. It functions as an exposure control unit that controls exposure in the second imaging unit 2 and the first imaging unit 1 so that the image of the subject (single-eye image) has substantially the same brightness. More specifically, the overall control unit 5 integrates the image sensors 13 and 23 so that the average pixel value of each image is about 128 with 8 bits in the monocular image and the monocular image. Control the time.

  When the monocular image of the subject imaged by the second imaging unit 2 and the single-eye image of the subject imaged by the first imaging unit 1 have different brightness, distance measurement and colorimetry using these images Therefore, it is possible to avoid a decrease in distance measurement accuracy due to brightness.

(Ranging unit)
Next, details of the distance measuring unit 3 of the imaging apparatus 10 will be described. The distance measuring unit 3 captures an image with the imaging element 13 via the monocular image of the subject imaged by the imaging element 23 of the second imaging unit 2 and the reference filter (for example, the filter 12G) of the first imaging unit 1. The distance to the subject is measured using the single eye image of the subject. This will be described in more detail below.

<Corresponding point search>
FIG. 4 shows a single-eye image (reference image) of a subject imaged by the image sensor 13 through a reference filter, and a monocular image (reference image) of the subject imaged by the image sensor 23 of the second imaging unit 2. ) Schematically. The distance measuring unit 3 first searches for a corresponding point on the reference image and a corresponding point on the reference image. That is, by scanning the epipolar line in the reference image and calculating the degree of coincidence with the attention point in the base image, the corresponding point corresponding to the attention point is obtained on the reference image. Such a corresponding point search is performed while shifting the point of interest on the reference image.

  More specifically, a window (template T0) having a predetermined size around a predetermined pixel position (attention point P) for which a corresponding point is to be searched is set on the reference image. Similarly, a plurality of windows W0 having the same size on the reference image are set on the epipolar line E0. Then, the degree of coincidence (correlation degree) of each window W0 on the reference image with respect to the template T0 on the standard image is obtained (template matching), and the pixel at the center position in the window W0 having the highest degree of coincidence is set as the corresponding point Q.

Here, the description of the epipolar line E0 will be supplemented. Figure 5 illustrates the positional relationship between the target point X in the space, a main point O _L of one camera lens, and the principal point O _R of the other camera lens. The straight line connecting the principal point O _L and the principal point O _R is a point of intersection with the reference image captured by one camera and e _L, the point of intersection with the reference image captured by the other camera and e _R. Straight line connecting the target point X and the principal point O _L is projected as a straight line on the reference image. This straight line is called an epipolar line E. That is, the principal point _{O L} a plurality of points on the straight line connecting the point of interest _{X X 1, X 2, X} 3, ··· X L is arranged on the epipolar line E of the reference image. Therefore, by scanning the epipolar line E in the reference image and calculating the degree of coincidence with the attention point in the standard image, it is possible to obtain the corresponding point corresponding to the attention point on the reference image.

  In addition, as an epipolar line calculation method, there are a method of actually measuring an epipolar line, a method of calculating mathematically from a point on an image, a method of calculating by a camera parameter, etc., all of which are known methods. Therefore, detailed description thereof is omitted here.

  The degree of coincidence described above can be obtained by a known method using NCC (Normalized Cross Correlation), SAD (Sum of Absolute Difference), SSD (Sum of Squared Difference), or the like. For example, when the degree of coincidence is obtained using NCC, it can be obtained by the following equation (1). T (i, j) is the luminance value of the pixel of the template T0, and I (i, j) is the luminance value of the pixel of the window W0 of the reference image. Coordinates (i, j) are, when the template T0 has a width of M pixels and a height of N pixels, the upper left coordinates of the template are (0, 0) and the lower right are (M-1, N-1). It is a coordinate when doing.

  FIG. 6 schematically shows the relationship between the position (coordinates) on the epipolar line E0 and the degree of coincidence. In obtaining the point (corresponding point) having the highest degree of coincidence, the position corresponding to the vertex of the curve in FIG. 6 may be set as the position of the corresponding point. At this time, as shown in FIG. 7, three points near the vertex in the curve may be extracted and approximated by a quadratic function, and the position corresponding to the vertex may be set as the position of the corresponding point. In the latter case, the corresponding points can be obtained with higher accuracy.

<3D position measurement>
After performing the above-described corresponding point search, the distance measuring unit 3 measures the three-dimensional position of each point on the subject using the attention point and each position (coordinate) of the corresponding point, and the internal parameter and the external parameter described above. To do.

FIG. 8 schematically shows the positional relationship between the subject Ob and each part of the first imaging unit 1 and the second imaging unit 2 on the ZX plane in the XYZ orthogonal coordinate system. Here, the distance (baseline length) between the _single lens 11G and the monocular lens 21 in the positional relationship corresponding to the reference filter, that is, the principal point O _{0 of the single} lens 11G and the principal point O ₁ of the monocular lens 21. Let b be the distance. Further, each focal length of the single-eye lens 11G and the monocular lens 21 is set to f, and a distance from the optical axis AX1 of the single-eye lens 11G to the pixel (target pixel) corresponding to the target point in the image sensor 13 is set to u. The distance from the optical axis AX2 of 21 to the pixel (corresponding pixel) corresponding to the corresponding point in the image sensor 23 is represented by v. It is assumed that the optical axes AX1 and AX2 are parallel.

When the principal point O _{0 of the} single lens 11G of the reference single eye is the origin of the three-dimensional coordinates and the coordinates of the points on the subject Ob are (x, y, z), the following relational expression is obtained from FIG. It is done.
z: x = f: u (1)
z: (b−x) = f: v (2)

The above two formulas can be modified as follows.
z = (f / u) · x (1 ′)
z = (f / v) · (b−x) (2 ′)

  Since each parameter (f, u, v, b) is known in advance, z and x can be obtained by solving the simultaneous equations. By considering the YZ plane in the same way, the three-dimensional coordinates (x, y, z) of the point on the subject Ob can be obtained. By performing such processing using a plurality of pixels of the reference single eye as a target pixel, it is possible to obtain the three-dimensional coordinates of a plurality of points on the subject Ob, thereby obtaining the three-dimensional shape of the subject Ob. Can do.

(Colorimetric part)
Next, details of the color measurement unit 4 of the imaging apparatus 10 will be described. The colorimetric unit 4 includes the distance to the subject measured by the distance measuring unit 3, the single-eye image of the subject imaged by the imaging device 13 via the reference filter (filter 12G), and a reference filter other than the reference filter. Using the other single-eye images of the subject imaged through the filters (filters 12R and 12B), a colorimetric value of the subject is obtained for a wavelength (for example, R or B) that passes through a filter other than the reference filter. Note that as the colorimetric value of the subject for the wavelength (for example, G) transmitted through the reference filter, the pixel value when the subject is imaged by the image sensor 13 (imaging region 13G) via the reference filter may be used as it is. Hereinafter, as an example, a method for obtaining the colorimetric value of B of the subject will be described.

  First, the color measurement unit 4 is imaged by the image sensor 13 (imaging region 13G) from the three-dimensional position of each point on the subject measured by the distance measurement unit 3 through the filter 12G as a reference filter. A point of interest in the single-eye image of the subject and a corresponding point on the single-eye image of the subject captured through the filter 12B are detected.

  That is, in FIG. 8, a single-eye image of the subject Ob captured by the image sensor 13 through the filter 12G is used as a reference image, and a single-eye image of the subject Ob captured by the image sensor 13 through the filter 12B is used. A reference image is used. Since the distance measurement unit 3 knows the coordinates (x, y, z) of the point on the subject Ob, the color measurement unit 4 uses the above formulas (1 ′) and (2 ′) to draw attention in the reference image. The position of the corresponding point (corresponding pixel) corresponding to the point (target pixel) on the reference image can be obtained (deviation amount v from the optical axis AX2 of the corresponding point).

When the corresponding point is obtained, the colorimetric unit 4 sets the pixel value of the obtained corresponding point as a colorimetric value for the wavelength (B) transmitted through the filter 12B. That is, in FIG. 8, the pixel value of a point on the extension of the line connecting the point and the principal point O ₁ on the object Ob, the colorimetric values B.

  The colorimetric value of R can be obtained in the same manner as described above. That is, by performing the same processing as described above with the standard image as it is (a single-eye image captured through the filter 12G) and the single-eye image of the subject imaged through the filter 12R as a reference image, A colorimetric value of R can be obtained.

  As described above, in the present embodiment, the distance measuring unit 3 includes the monocular image of the subject imaged by the second imaging unit 2 and the single-eye image of the subject imaged by the first imaging unit 1. The distance to the subject is measured using the filter 22G of the second imaging unit 2 and the single-eye image captured through the filter 12G having the same spectral characteristic. Since the second imaging unit 2 is located outside the first imaging unit, the distance (baseline length) between the monocular lens 11G corresponding to the filter 12G and the monocular lens 21 of the second imaging unit 2 Can be easily made longer than the distance between the adjacent single-lens lenses 11G and 11R in the first imaging unit 1. By increasing the base line length, the accuracy of ranging accuracy can be improved. Further, distance measurement can be performed without changing the design of the lens array, such as shortening the distance between the individual lenses 11G and 11R, thereby avoiding an increase in cost and an increase in size of the apparatus due to a change in the design of the lens array. You can also

  In particular, as shown in FIG. 2, the second imaging unit 2 is located away from the first imaging unit 1, so that the above-described baseline length can be reliably increased, so that the ranging accuracy is ensured. Can be improved.

  The color measurement unit 4 also measures the distance to the subject measured by the distance measurement unit 3, the single-eye image captured through the reference filter (filter 12G), and the filter 12 other than the reference filter (filter 12R • 12B) is used to obtain the colorimetric value of the subject for the transmission wavelength of the filter 12 other than the reference filter, using the other single-eye image captured through 12B). By improving the distance measurement accuracy described above, the distance measurement error is reduced, so that it is possible to improve the accuracy of B and R colorimetry at any point on the subject located at the measured distance.

  In particular, the distance measuring unit 3 searches for the corresponding point on the monocular image corresponding to the attention point in the single-eye image captured through the reference filter while shifting the attention point, Since the three-dimensional position of each point is measured, the three-dimensional shape of the entire subject can be obtained.

  In addition, the colorimetric unit 4 detects a corresponding point on the single-eye image captured through the reference filter and a corresponding point on the other single-eye image from the three-dimensional position of each point on the subject. Since the pixel value of the point is a colorimetric value, the colorimetric value for the wavelength transmitted through the filter 12 other than the reference filter can be obtained with high accuracy.

  Further, the colorimetric values (R and B) obtained by the colorimetric unit 4 and the G image data obtained by the imaging device 13 are output to the output unit 7 via the storage unit 6, and the output unit 7 outputs RGB data. A color image is displayed. At this time, the RGB images need to be overlaid so that pixels representing the same point on the subject match in RGB. In other words, since the RGB images captured by the imaging regions 13R, 13G, and 13B of the imaging device 13 have parallax, it is necessary to correct the parallax and superimpose the RGB images. In the present embodiment, the error in the distance obtained for G is small due to the improvement in distance measurement accuracy, and the R and B pixel positions corresponding to the target pixel of G using the obtained distance (at the time of color measurement in FIG. 8). V) can be obtained with high accuracy, so that RGB images can be accurately superimposed so that pixels representing the same point on the subject coincide with each other in RGB, thereby improving the accuracy of parallax correction. That is, an increase in the base line length (corresponding to b in FIG. 8) leads not only to improvement in distance measurement accuracy but also to improvement in parallax correction accuracy.

  Further, as shown in FIG. 3, the filters 12R, 12G, and 12B of the first imaging unit 1 have discrete spectral characteristics, and there is almost no correlation between the spectral characteristics (in terms of transmission wavelength). There are few common parts). When correlation is given to each spectral characteristic, an overlap occurs in each spectral characteristic when performing colorimetry in a certain wavelength range (400 nm to 700 nm wavelength range), so the number (type) of filters 12 to be used is increased. Although the spectral characteristics are discrete, the number (types) of the filters 12 to be used can be reduced. As a result, the number (type) of the corresponding single-eye lenses 11 can be reduced, and the apparatus can be reduced in size and cost. In addition, since distance measurement can be performed with a small number of eyes, the resolution can be improved by increasing the number of pixels per eye.

  Further, when correlation is given to each spectral characteristic, for example, the G filter transmits B light in addition to G, and the accuracy of G colorimetry is lowered, so that an image of the subject is displayed. Sometimes color reproducibility deteriorates. However, in this embodiment, since each spectral characteristic is discrete, it is possible to suppress a decrease in color reproducibility due to a decrease in colorimetric accuracy of each color.

  In the present embodiment, the wavelength band in which colorimetry can be performed is the visible range (400 nm to 700 nm), but is not limited to this range, and the image sensor imaging range such as the ultraviolet range to the near infrared range. Can be set arbitrarily.

<Operation>
FIG. 9 is a flowchart showing a flow of operations including distance measurement processing and color measurement processing in the imaging apparatus 10. With reference to FIG. 9 and FIG. 2, the operation of the imaging apparatus 10 of the present embodiment will be described.

When the process is started by an instruction from the overall control unit 5, the subject Ob is imaged by each individual eye of the first imaging unit 1 and the monocular of the second imaging unit (step 10), and a captured image is output (step 10). 11). In step 11, specifically, each image data (I ₁ (R) in FIG. 2) captured from each imaging region 13R, 13G, 13B of the first imaging unit 1 and the imaging element 23 of the second imaging unit. , I ₁ (G), I ₁ (R)) are output to the distance measuring unit 3.

  The distance measuring unit 3 is a single-eye image (reference image of the subject imaged by the image sensor 13 through the reference filter of the first imaging unit 1. For example, an image imaged through the filter 12G in FIG. ) And the corresponding points in the monocular image (reference image, which is imaged through the filter 22 in FIG. 2) of the subject imaged by the imaging device 23 of the second imaging unit 2. This is performed while shifting the point of interest on the reference image (step 12).

  After performing the corresponding point search, the distance measuring unit 3 measures the three-dimensional position of each point on the subject Ob (step 13). Thereafter, the colorimetric unit 3 uses the three-dimensional position of each point on the subject Ob as described above, a point of interest in a single-eye image of the subject imaged by the imaging device 13 via the reference filter, and a filter other than the reference filter. Corresponding corresponding points are detected on another single-eye image imaged through (filter 12B in FIG. 2) (step 14), and the pixel values of the obtained corresponding points are transmitted through wavelengths other than the reference filter (FIG. 2 is a colorimetric value for wavelength B) (step 15).

(Modification)
FIG. 10 shows another example of the spectral characteristics of each filter 12 of the first imaging unit 1. The transmission wavelength of each filter 12 is not limited to RGB. For example, as shown by the dashed curve in FIG. 10, each filter 12 includes a filter a1 having a transmission wavelength peak shorter than the B peak, and a transmission wavelength peak having a B peak and a G peak. A filter a2 between them and a filter a3 having a transmission wavelength peak in the R wavelength region may be used in combination. Further, as the filter 22 of the second imaging unit 2, a filter having the same spectral characteristics as any of the filters a1, a2, and a3 may be used. For example, a filter a4 having a transmission wavelength peak in the G wavelength range. May be used. In this case, the filter a2 having the closest spectral characteristic to the filter a4 may be used as the reference filter. Furthermore, in addition to each filter 12, an IR cut filter a5 that absorbs infrared light may be provided. Thus, the filters 12 and 22 are not necessarily limited to RGB filters. For example, a special-purpose filter can be used as the filter 12, and any of general-purpose and low-cost RGB filters can be used as the filter 22, and ranging accuracy and measurement equivalent to those in the first embodiment can be used. Color accuracy can be obtained.

[Embodiment 2]
The following will describe another embodiment of the present invention with reference to the drawings.

  FIG. 11 is an explanatory diagram illustrating a configuration of the imaging apparatus 10 according to the present embodiment. In the present embodiment, filters 12X, 12Y, and 12Z are used as the filters 12 of the first imaging unit 1 of the imaging apparatus 10, and a filter 22a is used as the filter 22 of the second imaging unit 2. Note that the distance measurement by the distance measurement unit 3 and the color measurement method by the color measurement unit 4 are exactly the same as those in the first embodiment.

FIG. 12 shows the spectral characteristics of the filters 12X, 12Y, and 12Z and the filter 22a. The filter 22a is a color filter with a Bayer arrangement, in which 4 pixels of 2 × 2 are arranged such that R is 1 pixel, G is 2 pixels, and B is 1 pixel, and these are arranged as a set regularly. Is. Therefore, the image pickup device 23, by imaging a subject through the filter 22a, and outputs the RGB image data as _{I 2.} In addition to the above R / G / G / B, the color filter colors may be C (cyan) / M (magenta) / Y (yellow) / G or W (white) / Y / R / Ir (infrared), etc. may be sufficient. On the other hand, the imaging device 13 outputs XYZ image data as I ₁ (X), I ₁ (Y), and I ₁ (Z) by imaging the subject through the filters 12X, 12Y, and 12Z.

  As described above, the filter 22a has a spectral characteristic that allows RGB light to pass therethrough. From FIG. 12, the B spectral characteristic of the filter 22a is substantially the same as the spectral characteristic of the filter 12Z. Therefore, in this embodiment, the filter 12Z is used as the reference filter among the filters 12X, 12Y, and 12Z.

  Even when the filter 22a of the second imaging unit 2 transmits RGB light as in the present embodiment, the spectral characteristics for any one of the wavelengths (B in the above example) are the first imaging unit. The spectral characteristic of the reference filter (filter 12Z) of the first filter 12 is almost the same. Thus, high-precision ranging and colorimetry can be performed using the single-eye image captured through the filter 12Z and the monocular image captured through the filter 22a, as in the first embodiment. It becomes possible.

  In addition to the configuration of the first embodiment, the imaging device 10 of the present embodiment further includes an image generation unit 8. The image generation unit 8 interpolates pixels for the transmission wavelengths of the filter 22a (RGB wavelengths in this embodiment) from the image of the subject imaged by the imaging device 23 via the filter 22a of the second imaging unit 2. The image processing unit generates a color image for display. Such an image generation unit 8 can be configured by an image processing circuit that performs the image processing described above. The color image data generated by the image generation unit 8 is output to the storage unit 6 described in the first embodiment, and is output (for example, displayed) by the output unit 7 as necessary.

  If the image generation unit 8 generates a display color image, it is not necessary to generate a display color image from the image captured by the first imaging unit 1. Therefore, an XYZ filter (filters 12X, 12Y, and 12Z) can be used as the filter 12 used in the first imaging unit 1 as in the present embodiment. By using the XYZ filter, XYZ colorimetric values common in the field of colorimetry can be obtained, which is convenient.

  Note that colorimetric values for each pixel of XYZ are stored in the storage unit 6. Accordingly, the color image of the subject is displayed on the output unit 7 based on the RGB data, and when the cursor is set to an arbitrary point on the image, the XYZ colorimetric values at that point are displayed. You can also.

  As the filter 12, an NB (narrow band) filter other than RGB can be used instead of the XYZ filter. However, since the display in the output unit 7 is performed based on the RGB data, when the color image for display is not generated in the image generation unit 8, the RGB image data is obtained from the image captured through the NB filter. Need to get. In this case, it is necessary to prepare several types of NB filters having different spectral characteristics, and to add RGB data to generate RGB data, resulting in an enormous amount of calculation. Further, when the transmission wavelength band of the NB filter is narrow and less than the RGB band for display, the color reproducibility of the display image may deteriorate. However, in the present embodiment, since an image for display (RGB image) is generated by the image generation unit 8, even if an NB filter other than RGB is used as the filter 12 of the first imaging unit 1, The inconvenience does not occur. That is, an NB filter can be used as the filter 12.

  The imaging device described in each of the above embodiments can be expressed as follows, and thereby has the following effects.

  The imaging apparatus includes a first imaging unit that images a subject, a second imaging unit that images the subject, the first imaging unit, and the second imaging unit. A distance measuring unit that measures the distance to the subject using each image, and a color measuring unit that obtains a colorimetric value of the subject using the distance measured by the distance measuring unit, The first imaging unit is arranged side by side in a direction perpendicular to the optical axis, and is arranged corresponding to the plurality of first imaging optical systems having the same field of view, and the plurality of first imaging optical systems, A plurality of first optical filters having a plurality of spectral characteristics, a first imaging optical system having a corresponding positional relationship, and a plurality of imaging regions for imaging the subject through the first optical filter. And the second imaging unit includes a second imaging optical unit. And a second optical filter, a second imaging element that images the subject via the second imaging optical system and the second optical filter, and the second imaging unit includes: The second optical filter is located outside the first imaging unit, and has one of the plurality of first optical filters as a reference filter and has substantially the same spectral characteristics as the reference filter. And the ranging unit includes a monocular image of the subject imaged by the second image sensor, and a single-eye image of the subject imaged by the first image sensor via the reference filter. The distance to the subject is measured using the colorimetric unit, and the colorimetric unit captures the distance measured by the distance measuring unit and the first image sensor through the reference filter. The single eye image of the subject and the reference film A colorimetric value of the subject is obtained for a wavelength that passes through the first optical filter other than the reference filter, using another single-eye image of the subject imaged through a first optical filter other than .

  The distance measuring unit applies an image of a subject (monocular image) captured by the second image sensor via the second imaging optical system and the second optical filter, and the first image sensor via the reference filter. The distance to the subject is measured using the image of the subject (single-eye image) captured in this manner, that is, using each image captured through optical filters having substantially the same spectral characteristics. At this time, since the second imaging unit is located outside the first imaging unit, the first imaging optical system corresponding to the reference filter of the first imaging unit, and the second of the second imaging unit. It is possible to ensure a distance (baseline length) to the imaging optical system longer than the distance between the first imaging optical systems in the first imaging unit, for example. Thereby, ranging accuracy can be improved. Further, since it is possible to perform distance measurement without changing (for example, shortening) the distance between the first imaging optical systems, the cost increases due to the design change (change in the arrangement position of each first imaging optical system). Can also be avoided. Further, since the distance measurement accuracy can be improved without increasing the distance between the two first imaging optical systems, the apparatus can be downsized.

  In addition, the color measurement unit includes the distance to the subject measured by the distance measurement unit, the single-eye image of the subject imaged by the first image sensor via the reference filter, and the first other than the reference filter. A colorimetric value of the subject is obtained for a wavelength that passes through the first optical filter other than the reference filter, using another single-eye image of the subject imaged by the first image sensor via the optical filter. Since the error in the measured distance is reduced due to the improvement in the distance measurement accuracy described above, the accuracy of the color measurement of the subject performed using the distance and a plurality of single-eye images can be improved.

  The distance measuring unit searches the attention point in the monocular image of the subject captured by the first imaging element via the reference filter and the corresponding point corresponding to the monocular image, and uses the attention point as a search point. The distance between the first imaging optical system and the second imaging optical system, which are performed while being shifted and have a positional relationship corresponding to the reference filter, the first imaging optical system, and the second imaging optical system , The distance from the optical axis of the first imaging optical system to the pixel corresponding to the point of interest in the first imaging element, and the second from the optical axis of the second imaging optical system. The three-dimensional position of each point on the subject may be measured based on the distance to the pixel corresponding to the corresponding point in the image sensor.

  Since the three-dimensional position of each point on the subject is obtained by the distance measuring unit, not only the distance from the imaging device to each point on the subject but also the three-dimensional shape of the subject can be measured.

  The colorimetric unit is a monocular image of the subject imaged by the first image sensor via the reference filter from a three-dimensional position of each point on the subject measured by the distance measuring unit. And a corresponding point corresponding to the other single-eye image is detected, and the pixel value of the corresponding point is used as a colorimetric value for a wavelength that passes through the first optical filter other than the reference filter. Good.

  In this case, a colorimetric value for a wavelength that passes through the first optical filter other than the reference filter can be reliably obtained from each single-eye image obtained by the first image sensor.

  It is desirable that the plurality of first optical filters have discrete spectral characteristics. A certain wavelength range (for example, a wavelength range of 400 nm to 700 nm) is obtained by making the spectral characteristics of each first optical filter have little correlation, that is, making the transmission wavelength have little commonality. In the color measurement, the number of first optical filters to be used can be reduced. As a result, the number of first imaging optical systems to be used can be reduced, and the apparatus can be miniaturized.

  The second optical filter has a spectral characteristic that transmits light of red, green, and blue wavelengths, and the spectral characteristic for any one of the wavelengths is substantially the same as the spectral characteristic of the reference filter. There may be.

  Even when the second optical filter transmits RGB light, the spectral characteristics of any one of the wavelengths are almost the same as the spectral characteristics of the reference filter of the first optical filter. Color accuracy can be improved.

  The image pickup apparatus generates a color image for display by interpolating pixels for each wavelength from the image of the subject picked up by the second image pickup element via the second optical filter. You may further provide the production | generation part.

  In this case, since the color image for display is generated from the image captured by the second image sensor in the image generation unit, the color image for display is generated from the image captured by the first image capture unit. It does not have to be generated. For this reason, a filter having spectral characteristics different from those of the second optical filter can be used as the first optical filter used in the first imaging unit. Therefore, for example, if an XYZ filter is used as the first optical filter, XYZ colorimetric values common in the field of colorimetry can be obtained.

  The spatial resolution of the second imaging unit determined by the number of pixels of the second imaging element and the focal length of the second imaging optical system is the number of pixels of the first imaging element and the first imaging optics. It is desirable that the spatial resolution of each individual eye of the first imaging unit determined by the focal length of the system is substantially the same.

  If the first imaging unit and the second imaging unit have different spatial resolutions, the accuracy of distance measurement and colorimetry is rate-determined to the side where the spatial resolution is coarse, so that the accuracy of distance measurement and the like decreases due to the difference in spatial resolution. Can be avoided.

  In the imaging apparatus, the monocular image of the subject imaged by the second imaging unit and the monocular image of the subject imaged by the first imaging unit via the reference filter are approximately You may further provide the exposure control part which controls the exposure in a said 2nd imaging part and a said 1st imaging part so that it may become the same brightness.

  If the brightness of the monocular image of the subject imaged by the second imaging unit is different from that of the monocular image of the subject imaged by the first imaging unit, the accuracy of distance measurement and colorimetry using these images Therefore, it is possible to avoid a decrease in distance measurement accuracy and the like due to brightness.

  The second imaging unit may be located away from the first imaging unit. In this case, the distance (baseline length) between the first imaging optical system corresponding to the reference filter of the first imaging unit and the second imaging optical system of the second imaging unit is surely increased to measure the distance. Accuracy and the like can be improved reliably.

  In addition, the subject colorimetry method described in each of the above embodiments can be expressed as follows, and thereby, the same effect as that obtained by the imaging apparatus described above can be obtained.

  The subject colorimetric method described above is a first imaging unit that captures a desired subject, and is arranged side by side in a direction perpendicular to the optical axis, and forms an image of the subject with the same field of view. A plurality of first imaging optical systems, and a plurality of first optical filters arranged corresponding to the plurality of first imaging optical systems and having a plurality of spectral characteristics, and in a corresponding positional relationship. And a first image pickup device having a plurality of image pickup areas for picking up images of the subject formed through the first image pickup optical system and the first optical filter. A colorimetric method for obtaining a colorimetric value of a subject using an imaging device, wherein the imaging device is further a second imaging unit located outside the first imaging unit, the image of the subject A second imaging optical system and a second optical film And a second imaging element that captures an image of the subject formed through the second imaging optical system and the second optical filter, and the second optical filter includes: One of the plurality of first optical filters is used as a reference filter, and the second imaging unit having substantially the same spectral characteristics as the reference filter is provided, and the color measurement method includes the second color filter. A distance to the subject is measured using a monocular image of the subject imaged by the imaging device and a single-eye image of the subject imaged by the first imaging device via the reference filter. A step (steps 12 and 13), the measured distance, the single-eye image of the subject imaged through the reference filter by the first image sensor, and a number other than the reference filter; 1 optical filter A step of obtaining a colorimetric value of the subject with respect to a wavelength that passes through the first optical filter other than the reference filter using another single-eye image of the subject imaged via the step (steps 14 and 15). And have.

  In the step of measuring the distance to the subject, a search for a corresponding point on the monocular image of the subject imaged by the first imaging element via the reference filter and a corresponding point corresponding to the monocular image is performed. , While shifting the point of interest (step 12), the distance between the first imaging optical system and the second imaging optical system in a positional relationship corresponding to the reference filter, and the first imaging optical System, the focal length of the second imaging optical system, the distance from the optical axis of the first imaging optical system to the pixel corresponding to the point of interest in the first imaging device, and the second imaging The distance to the subject is determined by measuring the three-dimensional position of each point on the subject based on the distance from the optical axis of the optical system to the pixel corresponding to the corresponding point in the second image sensor. May be measured -Up 13).

  In the step of obtaining the colorimetric value of the subject, the image picked up by the first image sensor through the reference filter from the three-dimensional position of each point on the subject measured by the distance measuring unit. A point of interest in the single eye image of the subject and a corresponding point corresponding to the other single eye image are detected (step 14), and the pixel value of the corresponding point is transmitted through the first optical filter other than the reference filter. A colorimetric value for the wavelength to be used may be used (step 15).

  The imaging apparatus of the present invention can be used for, for example, measuring a three-dimensional shape of a subject, and spectral imaging in the beauty and medical fields.

DESCRIPTION OF SYMBOLS 1 1st imaging part 2 2nd imaging part 3 Distance measuring part 4 Color measuring part 5 Overall control part (exposure control part)
8 Image generation unit 10 Imaging device 11, 11R, 11G, 11B Single lens (first imaging optical system)
12, 12R, 12G, 12B, 12X, 12Y, 12Z filter (first optical filter)
13 Image sensor (first image sensor)
13R, 13G, 13B Imaging region 21 Monocular lens (second imaging optical system)
22, 22a filter (second optical filter)
23 Image sensor (second image sensor)

Claims (12)

A first imaging unit that images a subject;
A second imaging unit that images the subject;
A distance measuring unit that measures a distance to the subject using each image of the subject imaged by the first imaging unit and the second imaging unit;
A colorimetric unit that obtains a colorimetric value of the subject using the distance measured by the distance measuring unit;
The first imaging unit includes:
A plurality of first imaging optical systems arranged in a direction perpendicular to the optical axis and having the same field of view;
A plurality of first optical filters arranged corresponding to the plurality of first imaging optical systems and having a plurality of spectral characteristics;
A first imaging device having a plurality of imaging regions for imaging the subject via the first imaging optical system and the first optical filter that are in a corresponding positional relationship,
The second imaging unit includes:
A second imaging optical system and a second optical filter;
A second imaging element that images the subject through the second imaging optical system and the second optical filter;
The second imaging unit is located outside the first imaging unit,
The second optical filter has one of the plurality of first optical filters as a reference filter and has substantially the same spectral characteristics as the reference filter,
The distance measuring unit uses a monocular image of the subject imaged by the second image sensor and a single-eye image of the subject imaged by the first image sensor via the reference filter. Measure the distance to the subject,
The color measurement unit includes the distance measured by the distance measurement unit, the single-eye image of the subject imaged by the first image sensor via the reference filter, and other than the reference filter. A colorimetric value of the subject is obtained for a wavelength that passes through the first optical filter other than the reference filter, using another single-eye image of the subject imaged through the first optical filter. An imaging apparatus characterized by the above.   The distance measuring unit searches the attention point in the monocular image of the subject captured by the first imaging element via the reference filter and the corresponding point corresponding to the monocular image, and uses the attention point as a search point. The distance between the first imaging optical system and the second imaging optical system, which are performed while being shifted and have a positional relationship corresponding to the reference filter, the first imaging optical system, and the second imaging optical system , The distance from the optical axis of the first imaging optical system to the pixel corresponding to the point of interest in the first imaging element, and the second from the optical axis of the second imaging optical system. The imaging apparatus according to claim 1, wherein a three-dimensional position of each point on the subject is measured based on a distance to a pixel corresponding to the corresponding point in the imaging element.   The colorimetric unit is a monocular image of the subject imaged by the first image sensor via the reference filter from a three-dimensional position of each point on the subject measured by the distance measuring unit. And a corresponding point corresponding to the other single-eye image is detected, and a pixel value of the corresponding point is set as a colorimetric value for a wavelength that passes through the first optical filter other than the reference filter. The imaging apparatus according to claim 2.   The imaging apparatus according to claim 1, wherein the plurality of first optical filters have discrete spectral characteristics.   The second optical filter has a spectral characteristic that transmits light of red, green, and blue wavelengths, and the spectral characteristic for any one of the wavelengths is substantially the same as the spectral characteristic of the reference filter. The imaging apparatus according to claim 1, wherein the imaging apparatus is provided.   An image generation unit that generates a color image for display by interpolating pixels for each wavelength from the image of the subject imaged by the second imaging element via the second optical filter; The imaging apparatus according to claim 5, wherein   The spatial resolution of the second imaging unit determined by the number of pixels of the second imaging element and the focal length of the second imaging optical system is the number of pixels of the first imaging element and the first imaging optics. The imaging apparatus according to claim 1, wherein the imaging apparatus has substantially the same spatial resolution as each individual eye of the first imaging unit determined by a focal length of the system.   The monocular image of the subject imaged by the second imaging unit and the monocular image of the subject imaged by the first imaging unit via the reference filter have substantially the same brightness. The imaging apparatus according to claim 1, further comprising an exposure control unit that controls exposure in the second imaging unit and the first imaging unit.   The imaging apparatus according to claim 1, wherein the second imaging unit is located away from the first imaging unit. A first imaging unit that images a desired subject,
A plurality of first imaging optical systems that are arranged side by side in a direction perpendicular to the optical axis, each having the same field of view and forming an image of the subject;
A plurality of first optical filters arranged corresponding to the plurality of first imaging optical systems and having a plurality of spectral characteristics;
A first imaging device having a plurality of imaging regions that respectively capture images of the subject formed through the first imaging optical system and the first optical filter in a corresponding positional relationship, A colorimetry method for obtaining a colorimetric value of a subject using an image pickup apparatus including a first image pickup unit,
The imaging device further includes
A second imaging unit located outside the first imaging unit,
A second imaging optical system and a second optical filter for forming an image of the subject;
A second imaging device that captures an image of the subject formed through the second imaging optical system and the second optical filter, and the second optical filter includes the plurality of Including any one of the first optical filters as a reference filter, the second imaging unit having substantially the same spectral characteristics as the reference filter;
The colorimetric method is:
Using the monocular image of the subject imaged by the second image sensor and the single-eye image of the subject imaged by the first image sensor via the reference filter, Measuring the distance;
The measured distance, the single-eye image of the subject imaged through the reference filter, and the first image sensor, imaged through the first optical filter other than the reference filter. A step of obtaining a colorimetric value of the subject for a wavelength that passes through the first optical filter other than the reference filter using another single-eye image of the subject. The colorimetric method of the subject. In the step of measuring the distance to the subject,
Searching for the corresponding point on the monocular image of the subject imaged by the first image sensor through the reference filter and corresponding points on the monocular image while shifting the target point, A distance between the first imaging optical system and the second imaging optical system that are in a positional relationship corresponding to a reference filter; and focal lengths of the first imaging optical system and the second imaging optical system; The distance from the optical axis of the first imaging optical system to the pixel corresponding to the point of interest in the first imaging element, and the correspondence in the second imaging element from the optical axis of the second imaging optical system The subject colorimetric method according to claim 10, wherein the distance to the subject is measured by measuring a three-dimensional position of each point on the subject based on the distance to the pixel corresponding to the point. In the step of obtaining the colorimetric value of the subject,
Attention in the single-eye image of the subject imaged by the first image sensor through the reference filter from the three-dimensional position of each point on the subject measured in the step of measuring the distance to the subject A point and a corresponding point corresponding to the other single-eye image are detected, and a pixel value of the corresponding point is set as a colorimetric value for a wavelength that passes through a first optical filter other than the reference filter. Item 12. A colorimetric method for a subject according to Item 11.

Images