JP5674130B2 - Imaging device - Google Patents

Description

The present invention relates to an image with respect to one or via two or more polarizing filter or a color separation filter periodically arranged the optical filter the light from the imaging region received light by the light receiving element array, an image signal obtained by this The present invention relates to an imaging apparatus that performs processing to extract different optical components from each other and output image information using the extracted plural types of optical components.

  As this type of imaging apparatus, a predetermined imaging area is imaged using an imaging means such as a digital camera, and image information obtained by extracting only a desired wavelength band or only a desired polarization component is extracted from the image signal. Those that generate image information and the like are known. Such an imaging device, for example, receives light from an imaging region incident through an optical filter using a light receiving element array in which a large number of light receiving elements are two-dimensionally arranged, and according to the amount of light received by each light receiving element. Image information of the imaging region is generated from the image signal output from the light receiving element array. Patent Document 1 describes an imaging device using a color filter in which, for example, R (Red), G (Green), and B (Blue) color separation filters are arranged according to a predetermined arrangement pattern as an optical filter. Yes. Patent Document 2 describes an imaging device that receives light from an imaging region that is incident through an optical filter in which two types of polarizing filters (longitudinal polarization component and lateral polarization component) are periodically arranged, by a light receiving element array. Has been. According to such an imaging apparatus, image information obtained by extracting each of a plurality of different optical components in one imaging operation (for example, image information of R, G, and B, vertical polarization component, and horizontal polarization component, respectively) Image information).

  Such an optical component extraction image generated from a plurality of different optical components is used for object identification processing for identifying an object in an imaging region, and thus is image information (monochrome image information) consisting of only luminance information. This is useful in that object identification can be performed with higher accuracy than when object identification processing is performed. Therefore, in recent years, such an imaging apparatus has been used as, for example, an in-vehicle camera, and performs object detection processing based on a captured image obtained by imaging a forward area (imaging area) in the traveling direction of the host vehicle, and the preceding vehicle It is used for a driver support system that detects an object such as a vehicle and reduces the driving load on the driver of the own vehicle. In addition, it is expected to be widely used in other fields such as an imaging device for an object identification device used for robot control.

  The inventors of the present invention have developed an imaging apparatus that can output such an optical component extracted image at a frame rate of 30 fps in real time, and faced the following problems in developing the imaging apparatus. In the following description, an imaging device that obtains optical component extraction images of the longitudinal polarization component and the lateral polarization component will be described. However, the same applies to an imaging device that obtains image information having different wavelength components such as R, G, and B. It is.

  In developing such an imaging apparatus, real-time image processing is performed at a frame rate of 30 fps by extracting an optical component extracted image by extracting a longitudinal polarization component and a lateral polarization component from an image signal output from a light receiving element array. It is required to complete within a short period of time enabling video output. At present, in order to enable image processing that satisfies this requirement, in reality, a predetermined image processing program is executed by a general-purpose CPU, and software processing that realizes the image processing is performed instead of FPGA (Field Programmable Gate Array). It is necessary to employ hardware processing that realizes the image processing using dedicated hardware such as

Here, an example of image processing for generating an optical component extraction image from an image signal output from the light receiving element array will be described.
FIG. 30 is an explanatory diagram showing a correspondence relationship between the polarization filter of the optical filter and the light receiving elements of the light receiving element array.
In this figure, each square arranged vertically and horizontally indicates a light receiving element, a hatching area extending obliquely with respect to the direction in which the light receiving elements are arranged indicates an area of a vertical polarizing filter that cuts a vertical polarization component, and areas other than the hatching area indicate The area | region of the horizontal polarization filter which cuts a horizontal polarization component is shown. The optical filter of the example shown in the figure has a band of longitudinal polarization filters and a band of longitudinal polarization filters arranged alternately and the longitudinal direction of the bands is oblique with respect to the arrangement direction of the light receiving elements. And a lateral polarization filter are arranged.

The vertical polarization component V and the horizontal polarization component H of the target pixel (pixel number 5 in the figure) corresponding to one light receiving element on the light receiving element array are, for example, the image signal value I5 corresponding to the amount of light received by the target pixel 5. In addition to the image signal values I1 to I4 and I6 to I9 corresponding to the received light amounts of the eight peripheral pixels 1 to 4 and 6 to 9 that surround the periphery of the target pixel 5, the following equation 1 is used. More demanded. In the equation shown in the following equation 1, a coefficient matrix (calculation parameters) having elements A1 to A18 for calculating the vertical polarization component V and the horizontal polarization component H of the pixel of interest 5 from the image signal values I1 to I9. Hereinafter, they are referred to as pseudo inverse matrix parameters.

This pseudo inverse matrix parameter can be obtained in advance as follows.
First, the entire light receiving element array is irradiated with 100% longitudinally polarized sample light with a substantially uniform light intensity V through the optical filter shown in FIG. 30, and each light receiving element (the pixel number of the light receiving element is set to i). ) To obtain the sample longitudinal polarization intensity avi according to the amount of light received. Similarly, the entire light-receiving element array is irradiated with 100% laterally polarized sample light with an almost uniform light intensity H through an optical filter, and the sample laterally polarized light corresponding to the amount of light received by each light-receiving element at that time. Get strength ahi. It can be considered that the sample longitudinal polarization intensity avi and the sample transverse polarization intensity ahi of each light receiving element thus obtained are proportional to the sensitivities of the longitudinal polarization component and the lateral polarization component received by the light receiving element, respectively. it can. Therefore, if the intensity of light transmitted through the optical filter and received by each light receiving element i is Ii, Ii = avi × V + ahi × H is established. Then, it is assumed that the incident light is the same with respect to a total of nine pixels 1 to 9 of 3 × 3 pixels centered on the target pixel 5, and the determinant represented by the following equation 2 is set for these pixels 1 to 9. Get. In this determinant, an inverse matrix is obtained for sensitivity matrix parameters having av1 to 9 and ah1 to 9 as elements. This inverse matrix becomes the pseudo inverse matrix parameter described above. However, in order to optimize for hardware processing, the pseudo inverse matrix parameter obtained in this way may be slightly modified.

  In this example, the optical filter in which the vertical polarization filter and the horizontal polarization filter are alternately arranged has been described as an example. However, the optical filter in which the vertical polarization filter and no filter (area in which light is transmitted as it is) is alternately arranged. Even in the case of a filter or an optical filter in which a lateral polarization filter and no filter are alternately arranged, the pseudo inverse matrix parameter can be obtained by the same method as the above-described procedure. However, when such an optical filter is used, at the time of actual imaging, the light receiving element array does not receive either one of the horizontally polarized light component and the vertically polarized light component. For this reason, it is necessary to obtain a polarization component that does not receive light by using an image signal value of a polarization component that is received through the polarization filter and a non-polarization image signal value that is received through the absence of the filter. In this case, for example, the pseudo inverse matrix parameter may be generated using the fact that the amount of light received by non-polarized light approximates the sum of the amount of light received by the longitudinally polarized light component and the amount of light received by the horizontally polarized light component.

  Conventionally, the pseudo inverse matrix parameters generated in this way are individually set for each light receiving element. In this case, the total storage capacity required to store the pseudo inverse matrix parameter is, for example, the number of pixels of the light receiving element array (the number of light receiving elements) is 480 × 752, and the number of elements of one pseudo inverse matrix parameter is 18 If the data amount per element is 4 bytes, it is about 26 Mbytes. From the formula shown in the above equation 1, the vertical polarization component V and the lateral polarization component H of each light receiving element are used in the equation shown in the above equation 1 using 3 × 3 pixel image signal values I1 to I9 including the peripheral pixels. In the case where the image processing to be obtained is performed by hardware processing, a method of storing pseudo inverse matrix parameters in a ROM such as an SPI flash memory in advance can be considered. According to this method, since the pseudo inverse matrix parameter has been calculated in advance, it is not necessary to calculate the pseudo inverse matrix parameter every time the imaging apparatus is activated, and the activation time of the imaging apparatus can be shortened.

  However, this method requires a large-capacity memory of 26 Mbytes or more as a ROM for storing pseudo inverse matrix parameters, which causes a problem in terms of cost and power consumption. In particular, when image processing based on the formula shown in the above equation 1 is executed by programmable dedicated hardware such as FPGA, in general, not only pseudo inverse matrix parameters but also circuit data (for example, when the imaging apparatus is activated) It is also necessary to load other initial data into the RAM in the dedicated hardware. In this case, since it is necessary to store circuit data and initial data in the ROM that stores the pseudo inverse matrix parameters, the capacity required for the ROM further increases. Therefore, in order to reduce the capacity of the ROM that stores the pseudo inverse matrix parameters from the viewpoint of cost and power consumption, it is strongly desired to reduce the storage capacity of the pseudo inverse matrix parameters.

  Instead of storing the pseudo inverse matrix parameter in the ROM, for example, the sample longitudinal polarization intensity avi and the sample transverse polarization intensity ahi of each light receiving element i acquired for obtaining the pseudo inverse matrix parameter are stored in the ROM. A method of calculating the pseudo inverse matrix parameter of each light receiving element i in the imaging apparatus is also conceivable. In the case of this method, the total storage capacity of the sample longitudinal polarization intensity avi and the sample transverse polarization intensity ahi stored in the ROM is 480 × 752 in the number of pixels of the light receiving element array (the number of light receiving elements), and the data amount per pixel Is 4 bytes, it is about 2.9 Mbytes, which can be significantly smaller than the total capacity of the pseudo inverse matrix parameters (about 26 Mbytes). However, in this method, it is necessary to calculate pseudo inverse matrix parameters for all the light receiving elements in the imaging apparatus. Such inverse matrix calculation requires special calculation processing, and it is expensive to provide a dedicated device for performing the calculation processing, and the calculation processing load is large, which is disadvantageous in terms of power consumption. There is a fear.

  The present invention has been made in view of the above background, and an object of the present invention is to perform image processing on a captured image captured through an optical filter to extract different optical components and extract an optical component extracted image. It is an object of the present invention to provide an imaging apparatus capable of reducing the amount of calculation parameter data required to generate the memory and reducing the capacity of a memory for storing the calculation parameter.

In order to achieve the above object, the invention of claim 1 is an optical filter in which a plurality of types of selection filter regions that selectively transmit different optical components are periodically arranged, or one or more types of selections. The light from the imaging area is received by the light receiving element array in which the light receiving elements are two-dimensionally arranged through the optical filter in which the filter area and the non-selected filter area that transmits the incident light as they are periodically arranged. An image signal corresponding to the amount of light received for each unit pixel corresponding to one light receiving element or two or more light receiving element groups is output , and each light transmitted through a plurality of types of filter regions included in the optical filter is output . imaging means configured to receiving area ratio is biased in the unit pixel, based on the image signal outputted from the image pickup means, a unit pixel and its surrounding pixels From the light-receiving amount by using the operation parameter corresponding to the unit pixel, and an image processing means for executing image processing for calculating the different optical components to each other contained in the light received by the unit pixel, corresponding to all the unit pixels An imaging device including a parameter storage unit that stores calculation parameters to be received, wherein the imaging unit has a light receiving area ratio in each unit pixel of each light transmitted through a plurality of types of filter regions included in the optical filter. One type or two or more types of common unit pixels are present periodically, and the image processing means includes four or more adjacent unit pixels including a plurality of the same type of common unit pixels. It is divided into two or more configured unit processing areas, and the same type of common unit pixel in the same unit processing area is described above using the same calculation parameter. It is characterized in executing the image processing.
According to a second aspect of the present invention, in the image pickup apparatus according to the first aspect, the optical filter includes a plurality of types of filter regions arranged in a stripe shape.
According to a third aspect of the present invention, in the imaging apparatus according to the second aspect , the light receiving element array includes light receiving elements arranged at equal intervals in a vertical direction and a horizontal direction orthogonal to each other. The plurality of types of filter regions are configured such that the longitudinal direction thereof is inclined in both the vertical direction and the horizontal direction of the light receiving element array.
According to a fourth aspect of the present invention, in the imaging apparatus of the third aspect , when the light receiving element interval in the vertical direction of the light receiving element array is n and the light receiving element interval in the horizontal direction is m, the optical filter is The vertical length N of the plurality of types of filter regions satisfies the following formula (1), and the horizontal length M of the plurality of types of filter regions satisfies the following formula (2). It is a feature.
N = A × n (1)
M = B × m (2)
However, A and B are natural numbers and satisfy A ≠ B.
According to a fifth aspect of the present invention, in the imaging apparatus of the fourth aspect , the ratio of B in the formula (2) to A in the formula (1) is 1/2 or 2. It is.
According to a sixth aspect of the present invention, in the imaging apparatus according to any one of the first to fifth aspects, the selection filter region is configured by a polarization filter that selectively transmits a predetermined polarization component. It is characterized by.

In the present invention, one type or two or more types of common unit pixels having the same light-receiving area ratio in each unit pixel of each light flux that has passed through a plurality of types of filter regions provided in the optical filter of the imaging means periodically exist. . The “filter region” here means not only a selected filter region that selectively transmits a predetermined optical component but also a non-selected filter region that transmits incident light as it is. Further, here, the same unit pixels of the same type are used as those having the same light receiving area ratio.
As a result of intensive research by the present inventors, if the same type of common unit pixel having the same light receiving area ratio is adjacent to each other on the light receiving element array, the pixel value of the output image information is obtained from the image signal. It was found that the calculation parameters used in the image processing to be calculated are close to each other. The present invention has been made based on this finding. That is, the unit pixel on the light receiving element array is divided into two or more unit processing regions including four or more adjacent unit pixels including a plurality of common unit pixels of the same type. The unit processing area thus divided includes at least two common unit pixels of the same type that are close to each other on the light receiving element array. Therefore, the operation parameters used in the image processing for these common unit pixels are approximate to each other. Therefore, even if the same calculation parameter is repeatedly used as the calculation parameter for these common unit pixels, the calculation error can be within an allowable range. As a result, the number of calculation parameters stored in the parameter storage means can be reduced by the amount of reuse compared to the case where calculation parameters are stored in the parameter storage means for all unit pixels.

  As described above, according to the present invention, the amount of calculation parameter data required when performing image processing on a captured image captured through an optical filter to extract different optical components to generate an optical component extracted image is reduced. Since the number can be reduced, an excellent effect can be obtained in that the capacity of the memory (parameter storage means) for storing the operation parameters can be reduced.

It is a functional block diagram of the driver assistance system concerning an embodiment. It is a flowchart which shows the outline | summary of the vehicle detection process in the driver | operator assistance system. It is a flowchart which shows the flow of the process for discriminating a road surface and a solid object. It shows a differential polarization degree image obtained by imaging with a polarization camera mounted on a vehicle traveling on a highway. The image after performing the discrimination | determination process of a road surface and a solid object about the same differential polarization degree image is shown. It is a flowchart for determining a vehicle candidate area | region from the area | region determined to be a solid object area | region. It is explanatory drawing which shows schematic structure of the polarization camera which can be used for the driver | operator assistance system. It is a cross-sectional enlarged view which shows the optical filter and image sensor of the polarization camera. It is explanatory drawing which shows the correspondence of the polarization filter of the same optical filter, and the pixel on an image sensor. It is explanatory drawing for demonstrating the method of classifying the light receiving element on the image sensor for every unit processing area. It is explanatory drawing which shows distribution of four types of common unit pixels in one unit processing area. (A) is explanatory drawing which shows the common unit pixel distribution in 3x3 pixel used when performing an image process by making a 1st common unit pixel into an attention pixel. FIG. 6B is an explanatory diagram illustrating a common unit pixel distribution within 3 × 3 pixels used when image processing is performed using the second common unit pixel as a target pixel. (C) is an explanatory diagram showing a common unit pixel distribution within 3 × 3 pixels used when image processing is performed using the third common unit pixel as a target pixel. (D) is an explanatory diagram showing a common unit pixel distribution in 3 × 3 pixels used when image processing is performed with a fourth common unit pixel as a target pixel. It is explanatory drawing which shows the correspondence of the optical filter which concerns on the modification 1, and the pixel on an image sensor. FIG. 10 is an explanatory diagram showing a distribution of six types of common unit pixels existing in one unit processing region in Modification 1. It is explanatory drawing which shows the correspondence of the optical filter which concerns on the modification 2, and the pixel on an image sensor. FIG. 12 is an explanatory diagram showing a distribution of 12 types of common unit pixels existing in one unit processing area in Modification 2. It is a block diagram which shows schematic structure of the polarization camera which concerns on an Example. It is a block diagram which shows the internal structure of the difference polarization degree image generation part provided in the same polarization camera. It is an output timing chart of the BRAM address calculation part provided in the difference polarization degree image generation part. It is an internal block diagram of the coefficient generation part provided in the difference polarization degree image generation part. It is explanatory drawing which shows the division of the unit processing area | region in an Example. It is an address map of a BRAM for storing pseudo inverse matrix parameters in which pseudo inverse matrix parameters are stored. It is an internal block diagram of the polarization separation process part provided in the same differential polarization degree image generation part. It is an internal block diagram of the difference polarization degree calculation part provided in the same difference polarization degree image generation part. It is an internal block diagram of the difference polarization degree image calculation part provided in the difference polarization degree image generation part. It is an address map of the SPI flash memory in which pseudo inverse matrix parameters are stored. It is an address map of the pseudo inverse matrix parameter stored in the sector 24 of the SPI flash memory. It is an address map of the pseudo inverse matrix parameter stored in the sector 25 of the SPI flash memory. It is an address map of pseudo inverse matrix parameters stored in the sector 26 of the SPI flash memory. It is explanatory drawing which shows the correspondence of the polarizing filter of an optical filter, and the light receiving element of a light receiving element array.

Hereinafter, an embodiment in which an imaging device according to the present invention is applied to a driver support system for reducing the driving load of a driver of the own vehicle will be described.
First, a schematic configuration and an operation outline in the driver assistance system will be described.
FIG. 1 is a functional block diagram of the driver assistance system according to the present embodiment.
FIG. 2 is a flowchart showing an outline of vehicle detection processing in the driver assistance system of the present embodiment.
A landscape around the host vehicle including a road surface (moving surface) on which the vehicle travels is photographed by a polarization camera 10 as an imaging device mounted on a vehicle (not shown) (S1). Thereby, the vertical polarization intensity (hereinafter referred to as “vertical polarization intensity”) and horizontal polarization intensity (hereinafter referred to as “lateral polarization”) for each unit pixel (hereinafter simply referred to as “pixel”) corresponding to one light receiving element of the polarization camera 10. Strength ”) is obtained (S2). In the present embodiment, the differential polarization degree is calculated for each unit pixel from the vertical polarization intensity and the horizontal polarization intensity of each pixel obtained in this way (S3). From this differential polarization degree, a differential polarization degree image with the differential polarization degree of each pixel as the pixel value can be generated.

The differential polarization degree is obtained from the calculation formula shown in the following formula (3). That is, the differential polarization degree is the ratio of the difference value (luminance difference value) between the lateral polarization intensity and the longitudinal polarization intensity to the total value (luminance total value) of the transverse polarization intensity and the longitudinal polarization intensity. In addition, the differential polarization degree can be paraphrased as a difference value between the ratio of the P deflection intensity with respect to the total luminance value (lateral differential polarization degree) and the ratio of the S deflection intensity with respect to the total luminance value (vertical differential polarization degree). . In this embodiment, the case where the vertical polarization intensity is subtracted from the horizontal polarization intensity is described. However, the horizontal polarization intensity may be subtracted from the vertical polarization intensity.
Differential polarization degree = (transverse polarization intensity−longitudinal polarization intensity) / (transverse polarization intensity + longitudinal polarization intensity) (3)

In the present embodiment, differential polarization degree image data is output from the polarization camera 10, and the differential polarization degree image data is input to the road surface / three-dimensional object determination unit 21. The road surface / three-dimensional object discriminating unit 21 discriminates between an image area showing a road surface and an image area showing a three-dimensional object from the differential polarization degree image output from the polarization camera 10 (S4).
FIG. 3 is a flowchart showing a flow of processing for discriminating a road surface and a three-dimensional object.
When the road surface / three-dimensional object discriminating unit 21 receives the differential polarization degree image data from the polarization camera 10, first, a threshold value for binarizing the differential polarization degree image is set (S 41), and the difference is calculated using the threshold value. The polarization degree image is binarized (S42). Specifically, a binary image is created by assigning “1” to pixels having a differential polarization degree equal to or greater than a predetermined threshold and assigning “0” to pixels that do not. Thereafter, in the binarized image, when pixels assigned with “1” are close to each other, a labeling process for recognizing them as one image area is performed (S43). As a result, a set of a plurality of adjacent pixels having a high difference polarization degree is extracted as one high difference polarization degree region. Each high differential polarization degree region extracted in this way is compared with road surface feature data stored in a storage unit (not shown) as road surface feature data storage means, and the high difference polarization degree region that matches the road surface feature data. Is extracted as a road surface area (S44). Then, the shape of the road surface area extracted in this way is estimated by comparing with the road surface shape pattern (S45), and both ends of the road surface area, that is, road edge lines are specified. Thereafter, the remaining image area other than the extracted road surface area is extracted as a three-dimensional object area in which a three-dimensional object is projected (S46).

FIG. 4 shows a differential polarization degree image obtained by imaging with a polarization camera 10 mounted on a vehicle traveling on a highway.
FIG. 5 shows an image after the road surface and the three-dimensional object are subjected to the differential polarization degree image.
The image area of the black part in the image shown in FIG. 5 is a location extracted as a road surface area. Moreover, the line shown with the dotted line in the image shown in FIG. 5 is a road edge line.

The vehicle candidate area determination unit 22 determines a vehicle candidate area from the areas determined to be a three-dimensional object area in S46 using the characteristics of the differential polarization degree of light from the window glass of the vehicle. (S5).
FIG. 6 is a flowchart for determining a vehicle candidate area from among the areas determined to be solid objects areas.
The vehicle candidate area determination unit 22 determines whether or not the differential polarization degree is equal to or greater than a predetermined threshold for each solid object area determined by the road surface / three-dimensional object determination unit 21 (is within a predetermined numerical range). No) is detected, and a three-dimensional object region having a differential polarization degree equal to or greater than a predetermined threshold is detected (S51). Next, it is checked whether or not the area of the three-dimensional object region detected in this way is within the area range corresponding to the window glass of the vehicle (S52). Then, it is checked whether or not the three-dimensional object region that has passed this check has a shape corresponding to the window glass of the vehicle, and the three-dimensional object region that has passed this check is determined as the vehicle rear region (S53).

  The vehicle identification unit 23 identifies whether or not the vehicle candidate region determined by the vehicle candidate region determination unit 22 is a vehicle region by a pattern matching method using the vehicle feature amount pattern stored in the vehicle feature amount pattern storage unit 24. To do. As this pattern matching method, known methods can be widely used.

  In the present embodiment, for example, a monochrome luminance image (front) generated by using the luminance data calculated by the monochrome image processing unit is displayed on the display unit (display) 25 which is an in-vehicle information notification unit composed of CRT, liquid crystal, or the like. (View image) is displayed, and information indicating an area in which the other vehicle is displayed is displayed as information useful for the driver in a display form that is easy for the driver to see. According to this, for example, even in a situation where it is difficult for the driver to visually recognize the other vehicle, the driver can view the relative position between the host vehicle and the other vehicle by looking at the front view image on the display unit. The positional relationship can be grasped, and it is easy to travel safely without colliding with another vehicle.

  In addition, the vehicle control unit 26 of the present embodiment performs a process of grasping the relative positional relationship between the own vehicle and the other vehicle from the position information of the vehicle area identified by the vehicle identification unit 23, for example. It is determined whether or not the vehicle is approaching the vehicle, and processing for generating an alarm sound or the like when the vehicle approaches another vehicle is performed. Or when approaching another vehicle, you may perform the process which performs an automatic brake function and reduces the traveling speed of the own vehicle.

Hereinafter, the configuration and operation of the polarization camera 10 which is a characteristic part of the present invention will be described in detail.
FIG. 7 is an explanatory diagram showing a schematic configuration of the polarization camera 10 in the present embodiment.
The polarization camera 10 is mainly output from the image pickup lens 1, the optical filter 2, a sensor substrate 3 on which an image sensor 4 having a light receiving element array in which light receiving elements are two-dimensionally arranged, and a sensor substrate 3. A / D converter 5 that converts an analog electrical signal (the amount of light received by each light receiving element on image sensor 4) into a digital electrical signal, and a digital electrical signal (pixel data) output from A / D converter 5 ), And a signal processing unit 6 that generates and outputs differential polarization degree image data. Light from the imaging region including the subject (test object) passes through the imaging lens 1, passes through the optical filter 2, and is converted into an electrical signal corresponding to the light intensity by the image sensor 4. When an electrical signal (analog signal) output from the image sensor 4 is input to the A / D conversion unit 5, a digital signal indicating the amount of light received by each pixel on the image sensor 4 as pixel data from the electrical signal. Are output to the signal processing unit 6 together with the vertical synchronizing signal and horizontal synchronizing signal of the image.

FIG. 8 is an enlarged cross-sectional view showing the optical filter 2 and the image sensor 4.
The image sensor 4 is a CCD (Charge Coupled Device) or CMOS (Complementary Metal).
Oxide Semiconductor) and the like, and a photodiode 4a is used as a light receiving element thereof. The photodiodes 4a are two-dimensionally arranged for each pixel, and a microlens 4b is provided on the incident side of each photodiode 4a in order to increase the light collection efficiency of the photodiodes 4a. The image sensor 4 is bonded to a PWB (printed wiring board) by a technique such as wire bonding to form a sensor substrate 3. Since the wavelength range of the image to be captured is substantially visible light, an image sensor 4 having sensitivity in the visible light range may be selected. The image sensor 4 of the present embodiment has a configuration in which light receiving elements (photodiodes 4a) of 480 × 752 arranged at equal intervals in the vertical and horizontal directions orthogonal to each other are effective pixels, and one photodiode 4a. Is a unit pixel.

The optical filter 2 is disposed close to the surface of the image sensor 4 on the microlens 4b side. On the surface of the optical filter 2 on the image sensor 4 side, as shown in FIG. 9, there are a transmission region 2a that is a non-selection filter region that transmits light as it is, and a selection filter region that selectively transmits only the lateral polarization component. The horizontal polarization filter region 2b has stripe-shaped region division patterns alternately arranged so as to incline in both the vertical direction (y direction in the drawing) and the horizontal direction (x direction in the drawing) of the image sensor 4. doing. Specifically, when the pixel interval (distance between the centers of the photodiodes 4a) in the vertical direction of the image sensor 4 is n and the pixel interval in the horizontal direction is m, the vertical direction of the transmission region 2a and the horizontal polarization filter region 2b. The optical filter 2 is configured such that the length N satisfies the following formula (1) and the lateral length M of the transmission region 2a and the lateral polarization filter region 2b satisfies the following formula (2). As shown in FIG. 9, the optical filter 2 of the present embodiment is an example of A = 1 and B = 2 (that is, an example of B / A = 2).
N = A × n (1)
M = B × m (2)

In the present embodiment, as shown in FIG. 9, the ratio of the light receiving area in one pixel of each light transmitted through the two types of filter regions (the transmission region 2a and the lateral polarization filter region 2b) provided in the optical filter 2 is as follows. Four types of common unit pixels that are equal to each other are configured to periodically exist.
More specifically, the first common unit pixel receives the horizontally polarized light component transmitted through the horizontally polarized filter region 2b in the small region at the upper right corner of the pixel, like the pixel of pixel number 1 shown in FIG. These are pixels having a pattern for receiving the non-polarized light transmitted through the transmission region 2a in the remaining large region.
Like the pixel number 2 shown in FIG. 9, the second common unit pixel receives non-polarized light transmitted through the transmission region 2a in a small region at the lower left corner of the pixel, and horizontally in the remaining large region. It is a pixel having a pattern for receiving a laterally polarized light component transmitted through the polarizing filter region 2b.
The third common unit pixel receives the non-polarized light transmitted through the transmission region 2a in the small region at the upper right corner of the pixel, as in the pixel of pixel number 3 shown in FIG. It is a pixel having a pattern for receiving a laterally polarized light component transmitted through the polarizing filter region 2b.
The fourth common unit pixel, like the pixel of pixel number 5 shown in FIG. 9, receives the horizontally polarized light component transmitted through the horizontally polarized filter region 2b in the small region at the lower left corner of the pixel, and in the remaining large region. It is a pixel of the pattern which receives the non-polarized light which permeate | transmitted the transmission region 2a.

In the signal processing unit 6 of the polarization camera 10 according to the present embodiment, the vertical polarization of the pixel of interest (pixel number 5 in FIG. 9) using the pixel data output from the A / D conversion unit 5 and the pseudo inverse matrix parameter. The component V and the lateral polarization component H are calculated from the formula shown in the following equation 3. That is, in the present embodiment, the vertical polarization component V and the horizontal polarization component H of the target pixel 5 are not only the pixel data I5 of the target pixel 5 but also the eight peripheral pixels 1 to 4 surrounding the target pixel 5. Calculations are also made using 6 to 9 pixel data I1 to I4 and I6 to I9. As the pseudo inverse matrix parameter necessary for this calculation, a parameter obtained in advance and stored in an SPI flash memory as a parameter storage means described later is used.

  In the image processing executed by the signal processing unit 6 of the present embodiment, the effective pixels (480 × 752) of the image sensor 4 are divided into 30 equal parts and 47 equal parts as shown in FIG. , A processing process is performed in which the vertical polarization component V and the horizontal polarization component H of the pixel of interest 5 are calculated by the equation shown in the above equation 3 by dividing the pixel processing area into 1410 unit processing regions each composed of 16 × 16 pixels. When divided in this way, each unit processing region has two or more common unit pixels of the same type for each of the four types of common unit pixels described above.

FIG. 11 is an explanatory diagram showing the distribution of four types of common unit pixels in one unit processing region.
FIGS. 12A to 12D are explanatory diagrams showing the distribution of four types of common unit pixels in the 3 × 3 pixels used for image processing when each of the four types of common unit pixels is a target pixel. It is.
For the four types of common unit pixels, the first common unit pixel is indicated by a circled number 1, the second common unit pixel is indicated by a circled number 2, the third common unit pixel is indicated by a circled number 3, and the fourth common When the unit pixel is indicated by a circled numeral 4, in the case of the optical filter 2 configured as shown in FIG. 9, the common unit pixel distribution in each unit processing region is as shown in FIG. In such a distribution, the distribution of the four types of common unit pixels in the 3 × 3 pixels used for image processing using the first common unit pixel as the target pixel is which first common unit pixel in the unit processing region. Is also as shown in FIG. That is, when the first common unit pixel is the target pixel, the 3 × 3 common unit pixel distribution is uniquely determined.

Similarly, when the second common unit pixel is the target pixel, the 3 × 3 common unit pixel distribution is uniquely determined as shown in FIG.
When the third common unit pixel is the target pixel, the common unit pixel distribution of 3 × 3 pixels is uniquely determined as shown in FIG.
Further, when the fourth common unit pixel is the target pixel, the 3 × 3 common unit pixel distribution is uniquely determined as shown in FIG.

  As a result of intensive research by the present inventor, regarding the same type of common unit pixel having the same light receiving area ratio, if the pixels are close to each other on the image sensor 4, pseudo inverse matrix parameters ( We have found that the calculation parameters are close to each other. In particular, if not only the light receiving area ratio but also the same light receiving position in the pixel of each light transmitted through the transmission region 2a and the lateral polarization filter region 2b is a common unit pixel of the same type, the pseudo inverse matrix parameter The closeness of is further increased. Therefore, even if the same pseudo inverse matrix parameter is used repeatedly as the pseudo inverse matrix parameter of the same type of common unit pixel located close to each other, the calculation error can be kept within an allowable range.

  Therefore, in the present embodiment, the effective pixels of 480 × 752 on the image sensor 4 are divided into 1410 unit processing regions composed of 16 × 16 pixels adjacent to each other. The number of divisions is appropriately set according to the allowable calculation error. In other words, even with the same type of common unit pixel, the closer the distance between each other, the lower the approximation, so that the range of the same type of common unit pixel that uses the same pseudo inverse matrix parameter is allowed to have a calculation error. The number of divisions is set so as to be limited to a range located at a distance at which the closeness within the range can be obtained.

As the number of divisions increases, the approximation of the original pseudo inverse matrix parameter corresponding to the same type of common unit pixel that uses the same pseudo inverse matrix parameter is higher, so a smaller calculation error is required. The number of pseudo inverse matrix parameters (data amount) increases.
Specifically, in the case of this embodiment, since the number of elements of one pseudo inverse matrix parameter is 18 and the data amount per element is 4 bytes, the data amount per pseudo inverse matrix parameter Is 72 bytes. When dividing into 1410 unit processing areas as in the present embodiment, four pseudo inverse matrix parameters are reused in each unit processing area, so that 4 × 1410 = 5640 pseudo inverse matrix parameters in total. Is required. Therefore, the total data amount of the pseudo inverse matrix parameter is 406080 bytes (about 0.4 Mbytes). On the other hand, if one unit processing area is divided into 8 × 8 pixels, the number of divisions is 5640, and a total of 4 × 5640 = 22560 pseudo inverse matrix parameters are required. In this case, the total data amount of the pseudo inverse matrix parameter is 1624320 bytes (about 1.6 Mbytes).
On the other hand, if there is a margin for calculation error, the number of segments is reduced, the number of common unit pixels of the same type that reuse the same pseudo inverse matrix parameter is increased, and the number of pseudo inverse matrix parameters (data amount) May be further reduced.

  When preparing the pseudo inverse matrix parameters individually for all the pixels, the data amount of all the pseudo inverse matrix parameters is about 26 Mbytes, and the memory for storing the pseudo inverse matrix parameters is a large memory of 26 Mbytes or more. Where a capacity memory is required, according to the present embodiment, the data amount of all pseudo inverse matrix parameters can be reduced to about 0.4 Mbytes, so that an inexpensive small capacity memory with low power consumption is sufficient.

In the present embodiment, the data amount per element of the pseudo inverse matrix parameter may be reduced to, for example, 1 byte. In this case, the total data amount of the pseudo inverse matrix parameter can be further reduced to 101520 bytes (about 0.1 Mbyte).
In particular, in this embodiment, the four types of pseudo inverse matrix parameters that are reused in each unit processing area are the sum of the pseudo inverse matrix parameters obtained for all of the 16 × 16 pixels that constitute the unit processing area, You may normalize so that the absolute value of the element with the largest absolute value may become 127. With this normalization, the data amount per element of the pseudo inverse matrix parameter is 1 byte. Even if the data amount is reduced in this way, a differential polarization degree with good calculation accuracy can be calculated.

  In the present embodiment, the arrangement pattern (area division pattern) of the filter regions 2a and 2b of the optical filter 2 is 2 horizontally with respect to the image sensor 4 composed of 480 pixels vertically and 752 pixels horizontally. Although an example of a stripe pattern having an inclination of one pixel in the vertical direction has been described, the same applies to a stripe pattern having an inclination of one pixel in the horizontal direction and two pixels in the vertical direction. . This inclination is appropriately set. For example, the inclination may be set to 1.

[Modification 1]
FIG. 13 is an explanatory diagram showing a correspondence relationship between the optical filter 2 according to the modification example 1 and the pixels of the image sensor 4 configured such that the inclinations of the filter regions 2a and 2b are 1/3.
FIG. 14 is an explanatory diagram showing the distribution of six types of common unit pixels existing in one unit processing area in the first modification.
When the filter regions 2a and 2b are configured to have an inclination of 1/3 as in the optical filter of the first modification, there are six types of common unit pixels in total, and the distribution is The one shown in FIG. 14 is repeated. In the first modification, in consideration of the six types of common unit pixels, one unit processing region is configured by 6 × 6 pixels. In this case, since the number of divisions is approximately 10080, 6 × 10080 = 60480 pseudo inverse matrix parameters are required as a whole. Therefore, if the data amount per pseudo inverse matrix parameter is 18 bytes, the total data amount of the pseudo inverse matrix parameter is 1088640 bytes (about 1.1 Mbytes). The same applies when the inclination is 3.

[Modification 2]
FIG. 15 is an explanatory diagram showing a correspondence relationship between the optical filter 2 according to the modified example 2 configured so that the inclination of each of the filter regions 2a and 2b is 2/3 and the pixels of the image sensor 4. FIG.
FIG. 16 is an explanatory diagram showing the distribution of 12 types of common unit pixels existing in one unit processing area in the second modification.
When the filter regions 2a and 2b are configured to have an inclination of 2/3 as in the optical filter of the second modification, there are 12 types of common unit pixels as a whole. In the second modification, in consideration of the 12 types of common unit pixels, one unit processing region is configured by 8 × 8 pixels. In this case, since the number of divisions is 5640, 12 × 5640 = 67680 pseudo inverse matrix parameters are required as a whole. Therefore, if the data amount per pseudo inverse matrix parameter is 18 bytes, the total data amount of the pseudo inverse matrix parameter is 1218240 bytes (approximately 1.2 Mbytes).

In the present embodiment (including each modification example, the same applies hereinafter), a plurality of types of filter regions provided in the optical filter 2 selectively transmit only the transmission region 2a that transmits light as it is and the lateral polarization component. Although it is a case where it is with the polarizing filter area | region 2b, it is not restricted to this. For example, a transmission region that transmits light as it is and a vertical polarization filter region that selectively transmits only the vertical polarization component, or a vertical polarization filter region and horizontal polarization component that selectively transmits only the vertical polarization component. Even in the case of the lateral polarization filter region that allows selective transmission, a differential polarization degree image can be obtained with the same calculation accuracy.
In the present embodiment, the plurality of types of filter regions provided in the optical filter 2 are configured to be inclined in both the vertical direction and the horizontal direction of the image sensor 4. It is good also as a structure which is parallel to any one of a vertical direction and a horizontal direction. However, in this case, both light beams transmitted through each filter region are received within one pixel.

〔Example〕
Examples of the present invention will be described below.
FIG. 17 is a block diagram illustrating a schematic configuration of the polarization camera 10 according to the present embodiment.
An analog electrical signal corresponding to the amount of light received by each pixel output from the image sensor 4 is converted into pixel data, which is a digital signal, by the A / D converter 5, and the pixel data is a signal along with a vertical synchronization signal and a horizontal synchronization signal. The data is sequentially output to the processing unit 6. The vertical synchronization signal, horizontal synchronization signal, and pixel data sequentially input to the signal processing unit 6 are temporarily held in the line buffer unit 31 in the signal processing unit 6. The line buffer unit 31 holds pixel data for at least the previous two lines. Then, the line buffer unit 31 performs image processing from the pixel data for three lines including the pixel data for the previous two lines and the latest pixel data for one line input to the signal processing unit 6. 3 × 3 pixel data I1 to I9 consisting of the target pixel and its peripheral pixels are selected and output to the differential polarization degree image generation unit 50. The differential polarization degree image generation unit 50 then inputs the pixel data I1 to I9 for 3 × 3 pixels and the pseudo inverse matrix corresponding to the pixel of interest stored in the SPI flash memory 32 as parameter storage means. Using the parameters, the vertical polarization intensity V and the horizontal polarization intensity H of the light received by the target pixel are calculated from the equation shown in Equation 3 above, and the differential polarization degree for the target pixel is calculated from the calculation result. calculate. The calculated differential polarization degree is sequentially output to the subsequent apparatus together with the horizontal / vertical synchronization signal as data (pixel value) of the differential polarization degree image. By performing such image processing for all the effective pixels of the image sensor 4, a differential polarization degree image with the differential polarization degree for each pixel as a pixel value is generated.

Here, the configuration and operation of the differential polarization degree image generation unit 50 in the present embodiment will be described.
FIG. 18 is a block diagram illustrating an internal configuration of the differential polarization degree image generation unit 50.
This differential polarization degree image generation unit 50 is composed of an FPGA, calculates the vertical polarization intensity V and the horizontal polarization intensity H for the pixel from the pixel data I1 to I9 and the pseudo inverse matrix parameter, and the calculation result Thus, image processing for calculating the degree of differential polarization for the pixel is realized by hardware processing. Note that all pseudo inverse matrix parameters used in this image processing are loaded in advance into a pseudo inverse matrix parameter storage BRAM (Block-RAM) provided in the FPGA.

  The BRAM address calculation unit 51 of the differential polarization degree image generation unit 50 receives the vertical synchronization signal and the horizontal synchronization signal, and calculates a BRAM address value in which a pseudo inverse matrix parameter used for the image processing is stored. The output timing chart of the BRAM address calculation unit 51 in this embodiment is as shown in FIG.

  When the coefficient generation unit 52 of the differential polarization degree image generation unit 50 receives the BRAM address value output from the BRAM address calculation unit 51, the pseudo inverse matrix parameter stored in the BRAM address value is transmitted to the polarization separation processing unit 53. Output. FIG. 20 shows an internal configuration diagram of the coefficient generation unit 52 in the present embodiment. In addition, FIG. 21 shows divisions of unit processing areas in the present embodiment. FIG. 22 shows an address map of the pseudo-inverse matrix parameter storage BRAM in which pseudo-inverse matrix parameters are stored.

  The polarization separation processing unit 53 of the differential polarization degree image generation unit 50 includes nine pieces of pixel data I1 to I9 output from the line buffer unit 31 and 18 elements A1 to A18 output from the coefficient generation unit 52. The pseudo inverse matrix parameter is received, and the lateral polarization component H and the longitudinal polarization component V are calculated from the equation shown in Equation 3 above. FIG. 23 shows an internal configuration of the polarization separation processing unit 53 in this embodiment.

  When the differential polarization degree calculation unit 54 of the differential polarization degree image generation unit 50 receives the horizontal polarization component H and the vertical polarization component V output from the polarization separation processing unit 53, the differential polarization degree P = (HV) / ( H + V) is calculated. Theoretically, the differential polarization degree P takes a value in the range of −1 to 1, and therefore, the differential polarization degree calculation unit 54 outputs a bit representation of the decimal part. FIG. 24 shows an internal configuration diagram of the differential polarization degree calculation unit 54 in the present embodiment.

  When the differential polarization degree image calculation unit 55 of the differential polarization degree image generation unit 50 receives the differential polarization degree P output from the differential polarization degree calculation unit 54, the differential polarization degree P that represents the decimal part in bits is represented by the differential polarization degree P. A process of converting the pixel value of the image is performed. FIG. 25 shows an internal configuration diagram of the differential polarization degree image calculation unit 55 in the present embodiment.

Next, the configuration and operation of the parameter control unit 40 in the present embodiment will be described.
In this embodiment, all pseudo inverse matrix parameters are calculated in advance and stored in the SPI flash memory 32. The address map of the SPI flash memory 32 in this embodiment is as shown in FIGS. As shown in FIGS. 17 and 18, the parameter control unit 40 includes a general-purpose CPU 41. When the power is turned on, the parameter control unit 40 reads the pseudo inverse matrix parameters from the SPI flash memory 32 through the SPI interface 42, and reads all the pseudo inverses. The matrix parameter is transferred to the pseudo inverse matrix parameter storage BRAM in the FPGA constituting the differential polarization degree image generation unit 50 via the CPU bus 43 and the User interface 44.

Next, a processing flow from when the power is turned on to when the differential polarization degree image is generated will be described.
When the power is turned on, first, the CPU 41 of the parameter control unit 40 loads the FPGA circuit data in the SPI flash memory 32 into the FPGA constituting the differential polarization degree image generation unit 50. Thereafter, the CPU 41 executes a pseudo inverse matrix parameter transfer process. In this pseudo inverse matrix parameter transfer process, first, a read command is issued from the CPU 41 to the SPI interface 42. As a result, the pseudo inverse matrix parameter in the SPI flash memory 32 is read into the CPU 41 via the SPI interface 42. Thereafter, the CPU 41 issues a write command to the user interface 44. As a result, the pseudo inverse matrix parameter is written into the pseudo inverse matrix parameter storage BRAM in the coefficient generation unit 52 via the User interface 44. Until all the pseudo inverse matrix parameters are transferred to the pseudo-inverse matrix parameter storage BRAM in the coefficient generation unit 52, the above-described processing from issuing the Read command to issuing the Write command is repeated.

  When all the pseudo inverse matrix parameters in the SPI flash memory 32 are transferred to the pseudo inverse matrix parameter storage BRAM in the coefficient generation unit 52 and the pseudo inverse matrix parameter transfer process is completed as described above, the differential polarization degree image generation is performed. Move to the process. In this differential polarization degree image generation step, when a vertical synchronization signal, a horizontal synchronization signal, and pixel data are transferred from the image sensor 4, first, coefficients are generated for each pixel with reference to the vertical synchronization signal and the horizontal synchronization signal. The address value of the pseudo-inverse matrix parameter storage BRAM in the unit 52 is calculated, and the pseudo-inverse matrix parameter element (144-bit bus signal) stored in the BRAM is read. Then, the read 144-bit bus signal is divided into 18 elements to obtain 18 elements A1 to A18 of pseudo inverse matrix parameters. Thereafter, the polarization separation processing unit 53 uses the 18-element pseudo inverse matrix parameter to convert the horizontal polarization component H and the vertical polarization component V of the pixel to be processed (target pixel) into the target pixel and its peripheral pixels. The pixel data I1 to I9 are calculated from the equation shown in the above equation (3). The lateral polarization component H and the longitudinal polarization component V calculated in this way are sent to the differential polarization degree calculation unit 54, and the differential polarization degree calculation unit 54 calculates the differential polarization degree P. Then, the differential polarization degree image calculation unit 55 performs a process of converting the differential polarization degree P into a pixel value (image data) of the differential polarization degree image. Thereby, image data for one pixel of the differential polarization degree image is obtained. A differential polarization degree image is generated by performing the above processing for all pixels.

As described above, the polarization camera 10 of the driver assistance system according to the present embodiment is configured to transmit the incident light as it is and the lateral polarization filter region 2b as a selection filter region that selectively transmits the lateral polarization component that is a desired optical component. Light from the imaging region is received by the image sensor 4 which is a light receiving element array in which light receiving elements made up of the photodiodes 4a are two-dimensionally arranged through the optical filter 2 in which transmission regions 2a as selection filter regions are periodically arranged. An imaging unit serving as an imaging unit that outputs pixel data that is an image signal corresponding to an amount of received light for each unit pixel (simply referred to as “pixel”) corresponding to one light receiving element on the image sensor 4; It is included in the light received by the pixel from the output pixel data using a pseudo inverse matrix parameter that is a calculation parameter according to each pixel. Signal processing unit 6 as image processing means for executing image processing for calculating different optical components (longitudinal polarization component and transverse polarization component), and parameter storage means for storing pseudo inverse matrix parameters corresponding to all pixels An imaging apparatus including an SPI flash memory 32. In this polarization camera 10, the imaging unit periodically includes four types of common unit pixels having the same light receiving area ratio in each pixel of each light transmitted through the plurality of types of filter regions 2 a and 2 b provided in the optical filter 2. It is configured to exist. In the signal processing unit 6, two or more unit processing regions each including four or more adjacent unit pixels including a plurality of the same type of common unit pixels, specifically, each type of common unit pixel. A total of 1410 unit processing regions are obtained by dividing the unit processing region into 64 × 16 × 16 pixels, and the same type of common unit pixel in the same unit processing region is the same. The image processing is executed using the pseudo inverse matrix parameter. With this configuration, image processing can be executed using the same pseudo inverse matrix parameter for the same type of common unit pixel in the same unit processing region. When preparing pseudo inverse matrix parameters individually for all pixels, the data amount of all pseudo inverse matrix parameters is about 26 Mbytes, but in the present embodiment, this is reduced to about 0.4 Mbytes. can do.
In the present embodiment, the imaging unit is configured such that the light receiving area ratio in one pixel of each light transmitted through the plurality of types of filter regions 2 a and 2 b provided in the optical filter 2 is biased. As the light receiving area ratio is biased in this way, the sensitivity of at least one optical component (optical component having a large light receiving area) among the optical components received by the pixel is equal to the light receiving area of each optical component. It can be larger than the case. In this embodiment, since such pixels periodically exist, a pixel having a large light receiving area for an optical component different from the target pixel is adjacent to the target pixel. Become. Since these pixels are positioned adjacent to each other in the minute region, it can be considered that light from the same location in the imaging region is substantially received. Therefore, based on the pixel data from these pixels, the light intensities of optical components different from each other can be detected with high sensitivity for light from the same location in the imaging region.
In the present embodiment, the optical filter 2 has a plurality of types of filter regions 2a and 2b arranged in a stripe pattern. Thereby, the optical filter 2 can be manufactured at a lower cost than when a plurality of types of filter regions 2a and 2b are arranged in a checkered pattern, for example.
In the present embodiment, the image sensor 4 includes light receiving elements arranged at equal intervals in the vertical direction and the horizontal direction orthogonal to each other, and the optical filter 2 includes the longitudinal direction of a plurality of types of filter regions 2a and 2b. Is configured to incline in both the vertical direction and the horizontal direction of the image sensor 4. With such a configuration, more common unit pixels of a type different from the target pixel are more easily arranged around the target pixel than in a configuration that is not inclined, and the calculation accuracy is easily improved.
In particular, when the distance between the light receiving elements in the vertical direction of the image sensor 4 is n and the distance between the light receiving elements in the horizontal direction is m, the vertical length N of the plurality of types of filter regions 2a and 2b satisfies the above formula (1). If the horizontal length M of the plurality of types of filter regions 2a and 2b satisfies the above formula (2), more common unit pixels of a type different from the target pixel are arranged around the target pixel. The calculation accuracy can be increased.

In the above description, the selection filter region of the optical filter 2 is configured with a polarization filter that selectively transmits a predetermined polarization component, and the case where a differential polarization degree image is generated from two different polarization components has been described. The present invention is similarly applied to a case where a selective filter region of the optical filter 2 is configured by a spectral filter that selectively transmits a specific wavelength component and, for example, a color separation image is generated from a plurality of different wavelength components. it can.
Further, the driver assistance system according to the present embodiment is mounted on the vehicle as a whole, but the entire system is not necessarily mounted on the vehicle. Therefore, for example, only the polarization camera 10 may be mounted on the own vehicle, and the remaining system components may be remotely arranged at a location different from the own vehicle. In this case, a system in which a person other than the driver objectively grasps the traveling state of the vehicle can be provided.
The imaging apparatus according to the present invention is not limited to the present embodiment, and can be applied to a wide range of fields.

DESCRIPTION OF SYMBOLS 1 Image pick-up lens 2 Optical filter 2a Transmission area 2b Lateral polarization filter area 4 Image sensor 4a Photodiode (light receiving element)
5 A / D conversion unit 6 Signal processing unit 10 Polarization camera 31 Line buffer unit 32 SPI flash memory 40 Parameter control unit 41 CPU
DESCRIPTION OF SYMBOLS 50 Differential polarization degree image generation part 51 Address calculation part 52 Coefficient generation part 53 Polarization separation process part 54 Differential polarization degree calculation part 55 Differential polarization degree image calculation part

JP 2007-086720 A Japanese Patent No. 3777054

Claims (6)

An optical filter in which a plurality of types of selection filter regions that selectively transmit different optical components are periodically arranged, or one or more types of selection filter regions and a non-selection filter region that transmits incident light as they are A unit corresponding to one light receiving element or two or more light receiving element groups on the light receiving element array by receiving light from the imaging region by a light receiving element array in which the light receiving elements are two-dimensionally arranged through an optical filter arranged in a line An image signal that outputs an image signal corresponding to the amount of light received for each pixel, and is configured so that a light receiving area ratio in each unit pixel of each light transmitted through a plurality of types of filter regions provided in the optical filter is biased Means,
Based on the image signal outputted from the image pickup means, from the unit pixel and a light receiving amount of the surrounding pixels, by using a calculation parameter corresponding to the unit pixel, different from each other are included in the light received by the unit pixel optical Image processing means for executing image processing for calculating components;
An image capturing apparatus including parameter storage means for storing calculation parameters corresponding to all unit pixels,
In the imaging means, one or two or more types of common unit pixels having the same light receiving area ratio in the unit pixel of each light transmitted through the plurality of types of filter regions provided in the optical filter periodically exist. Configured,
The image processing means divides into two or more unit processing areas composed of four or more adjacent unit pixels including a plurality of common unit pixels of the same type, and the same type of common unit pixels in the same unit processing area. For the unit pixel, the image processing is executed using the same calculation parameter. The imaging device according to claim 1 .
The above-mentioned optical filter is an image pickup apparatus in which a plurality of types of filter regions are arranged in a stripe shape. The imaging device according to claim 2 .
The light receiving element array is a structure in which light receiving elements are arranged at equal intervals in the vertical and horizontal directions orthogonal to each other.
The optical filter is configured such that the longitudinal direction of the plurality of types of filter regions is inclined in both the longitudinal direction and the lateral direction of the light receiving element array. The imaging device according to claim 3 .
When the light receiving element interval in the vertical direction of the light receiving element array is n and the light receiving element interval in the horizontal direction is m, in the optical filter, the vertical length N of the plurality of types of filter regions is expressed by the following formula (1) ), And the horizontal length M of the plurality of types of filter regions is configured to satisfy the following expression (2).
N = A × n (1)
M = B × m (2)
However, A and B are natural numbers and satisfy A ≠ B. The imaging device according to claim 4 .
The ratio of B in the said Formula (2) with respect to A in said Formula (1) is 1/2 or 2, The imaging device characterized by the above-mentioned. The imaging device according to any one of claims 1 to 5 ,
The image pickup apparatus, wherein the selection filter region includes a polarization filter that selectively transmits a predetermined polarization component.