JP4974543B2 - Polarization imaging device - Google Patents

Description

  The present invention relates to a polarization imaging apparatus, a photographing apparatus using the same, a surface shape measuring apparatus, and the like.

  In general, the imaging principle of an image is to detect only the intensity of light (luminance information) from the subject as information obtained from the subject, and reproduce the shadow based on the luminance information to form an image. In this simple image forming method, unnecessary light information is simultaneously detected in addition to light from the subject, so that an optimum image cannot be obtained.

  Therefore, for example, when shooting a show window in which surrounding scenery is reflected with a camera, an optimal shooting is performed using a polarizing filter. A polarizing filter is a filter that increases the contrast of a subject by cutting the irregular reflection of ambient light by utilizing the property that reflected light from a glossy surface such as glass is polarized. The polarizing filter prevents the surrounding scenery from being reflected in the glass of the show window, and the blue light of the forward light has the property of polarization, so it is also used when the blue sky is to be extracted vividly blue.

  However, a normal polarizing filter has an anisotropy that mainly transmits polarized light in only one direction, and when a subject to be photographed has a plurality of polarized lights, all the disturbing polarized lights cannot be cut off. As a method for dealing with this problem, there is a method in which a polarization filter is rotated to obtain polarization information in a plurality of directions, and a plurality of images based on each polarization information are optimized. However, this method is not suitable for high-speed photography because it requires rotational driving and the apparatus is not large or robust, and it takes time to rotate 360 degrees corresponding to the previous polarization direction. As for the rotational speed, when it is continuously rotated for the purpose of speeding up, it is difficult to obtain accurate information, and on the other hand, when the rotational drive is slowed down and performed stepwise (stepwise) However, it is difficult to realize high-speed processing.

  On the other hand, an ellipsometer that eliminates the rotation driving unit has been proposed (see International Publication WO 2004/008196 (Patent Document 1 below)). In this ellipsometer, by using a polarizer array having a plurality of polarization directions, polarized light in various directions included in a certain incident light is separately received by a light receiving element array provided at the subsequent stage of the polarizer array. be able to. As a result, even with a fixed polarizer array, it is possible to obtain the same information as if polarization information is obtained by rotating a polarization filter that mainly transmits polarized light in only one direction, eliminating the rotation drive unit. Can do.

International Publication WO 2004/008196

  However, the light from the subject obtained when the subject is photographed includes light in various polarization states, and the literature describing the ellipsometer describes the light in various polarization states. A technique for analyzing in detail and separating light in a necessary polarization state and light in an unnecessary polarization state or extracting light in a specific polarization state has not been proposed.

  First, the meaning of general terms for "polarized light" will be described, and then "polarized light component" and "non-polarized light component" used in this specification will be described.

In general, light can be distinguished between “polarized light” and “non-polarized light”.
The meaning of “polarized light” as a general term means “light in which the electric field of light (electromagnetic wave) is oscillating while being biased in any direction”. Accordingly, when focusing attention on the light passing through the polarizer, “polarized light” is light whose intensity changes as the polarizer is rotated.
Moreover, the meaning of “non-polarized light” as a general term means “light whose electric field of light (electromagnetic wave) is oscillating in any direction”. Actually, the polarization direction changes in a finite and extremely short time, and even when observed using a measuring instrument that measures the polarization state, only the average state is measured. Accordingly, when focusing attention on light that has passed through the polarizer, “non-polarized light” means light that is always constant without changing its intensity even when the polarizer is rotated.

Furthermore, “polarized light” can be classified into the following three: “linearly polarized light”, “circularly polarized light”, and “elliptical polarized light”.
“Linearly polarized light” is light in which an electric field vibrates only in a specific direction. There is no component oscillating in a direction perpendicular to the vibration direction, and therefore light does not pass through the polarizer so that the transmission axis of the polarizer is perpendicular to the polarization direction.
“Circularly polarized light” is light in which an electric field vibrates in a certain direction and a direction perpendicular thereto, and the phase difference between the vibrations is π / 2 + nπ (n is an integer). By combining these two vibrations, when the electric field vector is projected onto a plane perpendicular to the light traveling direction, the tip of the vector forms a circular orbit. Therefore, when the intensity in each direction is measured using a polarizer, the measured intensity is an average value within a certain time, and therefore, the same value is obtained in any direction.
The “elliptical polarization” is the light in the case where the amplitudes of the electric fields in the two directions to be synthesized are different from each other in the above “circular polarization”, or even if the amplitudes of the two electric fields to be synthesized are the same magnitude. This is light when the phase is other than π / 2 + nπ. When the amplitudes of the electric fields in the two directions to be combined are different, the tip of the combined vector draws an elliptical orbit in a plane perpendicular to the light traveling direction. Further, even if the amplitudes of the two electric fields to be combined are the same, if the phase is other than π / 2 + nπ, the tip of the combined vector also draws an elliptical orbit. Therefore, when the intensity in each direction is measured using a polarizer, the intensity of transmitted light changes depending on the rotation angle of the polarizer, but this intensity does not become zero. Note that “linearly polarized light” and “circularly polarized light” are special examples of “elliptical polarized light”.

Based on the meaning of the general terms described above, “polarization component” and “non-polarization component” used in this specification will be described.
In this specification, the “polarized light component” refers to a component whose intensity changes depending on the rotation angle of the polarizer when the light to be measured passes through the polarizer, that is, the “linearly polarized light” and the “elliptical polarized light”. Say.
In the present specification, the “non-polarized component” means a component whose intensity does not change depending on the rotation angle of the polarizer when the light to be measured passes through the polarizer, that is, the “non-polarized component” and the “circular component”. "Polarized light".

  A polarization imaging apparatus according to a preferred embodiment of the present invention and an imaging apparatus and a surface shape measurement apparatus using the polarization imaging apparatus separate the “polarized light component” and the “non-polarized light component” and remove or extract one of them. It is an apparatus that exhibits various effects, and does not separate the “polarized light” and the “non-polarized light” that are used as general terms. That is, the “non-polarized component” includes the “circularly polarized light” in addition to the “non-polarized light”, and these are not distinguished. However, in reality, it is very rare that the reflected light from the object is the above-mentioned “circularly polarized light”, so in practice, the “non-polarized light component” may mean the above “non-polarized light”. There will be many.

  That is, the polarization imaging apparatus or the like according to a preferred aspect of the present invention obtains luminance information and a plurality of polarization information at the same time, and further includes the “polarization component” and the “non-polarization component” that constitute light from the subject. It is an object of the present invention to provide a polarization imaging apparatus and the like in which the image processing accuracy is further improved.

  In the present invention, basically, a light receiving module having a polarizer array and a subsequent light receiving element array is used, and image processing is performed on luminance (light intensity) information for each polarization information from the light receiving module by an image processing unit. Thus, the present invention is based on the knowledge that an optimal image can be obtained or in-plane polarization information can be obtained.

[1] As a polarization imaging apparatus according to the present invention,
It is divided into three or more polarizer regions each having a different transmission axis, and transmits the non-polarized component of the input light in each region of the input light that is incident, and the polarization direction varies depending on each region. A polarizer array including one or a plurality of polarizer units that transmit the polarization component of the input light;
A light receiving element array that independently receives light transmitted through each of the regions;
An image processing unit for processing the polarization component and the non-polarization component from the light receiving element array;
A polarization imaging apparatus having

  A polarizer array is a combination of multiple units of polarizers with different transmission axes (polarizer units), and light with multiple polarization directions can be extracted. This improves the analysis accuracy. In this specification, the polarizer is also referred to as a “region”. This region transmits, for example, polarized light that is perpendicular or parallel to the direction of irregularities formed periodically by self-cloning and reflects polarized light orthogonal thereto. Have In addition, for example, a wire grid type polarizer has a characteristic of transmitting polarized light perpendicular to the direction of the fine metal wire formed in the region and reflecting polarized light parallel to the direction of the fine metal wire. By using this polarizer array as part of the light receiving module, it is possible to simultaneously acquire polarization information in addition to luminance information for each element (pixel) in the light receiving element array, and detect local polarization information in the acquired image. can do. Then, the data from each light receiving element is processed based on the polarization information, and the degree of polarization at a specific location can be measured in acquisition of an image with little reflection or observation with a microscope.

  When focusing on the polarization state of the reflected light when it reflects light, the subject is roughly divided into a subject having a smooth surface (photographed portion) and a subject having an uneven surface (photographed portion). Divided. Of course, a smooth surface and an uneven surface are not strictly distinguishable, and both are continuous so that if the size of the unevenness is reduced, the smooth surface is approached. As a result, in general, light from a subject is composed of the “polarized component” and the “non-polarized component”. The smoother the surface of the subject, the more the reflected light is polarized. On the other hand, the reflected light from the uneven surface is scattered on the surface and is in an unpolarized state.

  Therefore, in the polarization imaging apparatus, an equal amount of the “non-polarization component” in all the regions constituting the polarizer unit among the “polarization component” and the “non-polarization component” constituting the light from the subject. The incident light is separated into a plurality of components by transmitting the “polarized component” having a different polarization direction depending on the region. Then, in the image processing unit, the transmitted light is analyzed in detail and separated into necessary components and unnecessary components, or one of them is extracted.

  As a result, polarization information of light from different reflection locations and reflection targets can be acquired individually and simultaneously for each degree of polarization and polarization state, and polarization images composed of different polarization directions can be captured simultaneously. Unnecessary noise due to polarized light is removed by separating, selecting, and reconstructing a polarized image into the “polarized component” and the “non-polarized component”.

[2] As a polarization imaging apparatus according to the present invention,
The image processing unit applies a mathematical model represented by the following formula (1) to the intensity fm (i, j) of transmitted light transmitted through each region with respect to the angle of the transmission axis in each region,
The polarization imaging apparatus according to [1], in which the intensity of the transmitted light is separated into an intensity A (i, j) relating to the polarization component and an intensity B (i, j) relating to the non-polarization component.
Here, m is a number assigned to each region, i and j are coordinates of the polarizer unit in the polarizer array, and θm is 0 ° of the transmission axis in the reference region among the regions. Is the angle of the transmission axis in each other region, and θ (i, j) is the angle between the polarization direction of the polarization component input to the polarizer unit and the transmission axis in the reference region It is a difference.

  As a specific example of image processing, there is a method of applying the mathematical model described above to the intensity of transmitted light obtained from each region (polarizer) in the polarizer unit. The above formula (1) will be briefly described. The transmitted light has a different amount of transmission (intensity) from the non-polarized light component (second term relating to the intensity B) that transmits the same amount of light in each area of the polarizer unit. The first term regarding A) is combined.

  Since the polarization state of light incident on a certain polarizer unit changes with a period larger than that of one unit, the polarization state is uniform in all regions. In contrast, the transmission axis in each region is shifted by θm (m = 0 to 0) from the reference region (transmission axis direction = 0 °), so that the transmitted light transmitted through the polarizer unit is The intensity depends on the transmission axis direction of each region. That is, the strength is in accordance with the first term relating to the strength A in the formula (1).

  By applying the mathematical model described above to the intensity of transmitted light, the intensity A (i, j) relating to the polarization component and the intensity B (i, j) relating to the non-polarization component are obtained, and both can be separated and extracted. it can. A desired image can be obtained by freely reconstructing these separated components.

[3] As a polarization imaging apparatus according to the present invention,
The image processing unit calculates an average value by adding all the intensities fm (i, j) of transmitted light transmitted through the respective regions and dividing the sum by the number of regions, and for the angle of the transmission axis in each region, By applying a mathematical model represented by the following formula (2) to the intensity obtained by subtracting the average value from the intensity fm (i, j) of the transmitted light transmitted through each region,
The polarization imaging apparatus according to [1], in which the intensity A (i, j) related to the polarization component is extracted from the intensity of the transmitted light.
Here, m is a number assigned to each region, i and j are coordinates of the polarizer unit in the polarizer array, and θm is 0 ° of the transmission axis in the reference region among the regions. Is the angle of the transmission axis in each other region, and θ (i, j) is the angle between the polarization direction of the polarization component input to the polarizer unit and the transmission axis in the reference region It is a difference. Further, the average value of the intensity fm (i, j) is displayed with a symbol with f (i, j) overlined.

  The average value of the transmitted light intensity in each region obtained by adding all the intensities fm (i, j) of the transmitted light transmitted through each region and dividing by the number of regions is the intensity A for the polarization component and the unpolarized component Is equal to the sum of intensity B for Therefore, by subtracting the average value of the transmitted light intensity in each region from the function of the transmitted light intensity fm (i, j) expressed by the above formula (1), as shown in the above formula (2), the polarization component Can be simplified to

[4] As a polarization imaging apparatus according to the present invention,
The image processing unit further includes the polarization imaging apparatus according to [2] or [3], in which the θ (i, j) is calculated to determine the polarization direction of the polarization component.

[5] Furthermore, as a polarization imaging apparatus according to the present invention,
It is divided into three or more polarizer regions each having a different transmission axis, and transmits the non-polarized component of the input light in each region of the input light that is incident, and the polarization direction varies depending on each region. A polarizer array including one or a plurality of polarizer units that transmit the polarization component of the input light;
A light receiving element array that independently receives light transmitted through each of the regions;
With respect to the intensity of transmitted light transmitted through each of the regions, an image processing unit that separates a polarizer unit having a difference in transmitted light intensity between adjacent regions larger than a predetermined value and a polarizer unit smaller than a predetermined value;
A polarization imaging apparatus having

[6] As a polarization imaging apparatus according to the present invention,
The polarization imaging apparatus according to [1] or [5], in which the transmission axis in the polarizer unit is different at an angle of 45 ° or less for each region.

[7] As a polarization imaging apparatus according to the present invention,
The polarizer unit has four regions,
The polarization imaging apparatus according to [1] or [5], wherein the transmission axis in the region is a direction of 0 °, 45 °, 90 °, and 135 ° with respect to a transmission axis in a reference region among the regions. Is mentioned.

  There are various ways of providing each region in the polarizer unit, but it is preferable to provide three or more regions in one unit. Moreover, it is preferable that the transmission axes in each region are formed so as to be evenly shifted. For example, when three areas are provided in one unit, a free transmission axis direction can be designed for other areas with respect to the reference area (transmission axis direction = 0 °), but preferably 60 ° and 120 ° shifted. The transmission axis direction. In addition, when four regions are provided in one unit, a free transmission axis direction can be designed for other regions with respect to the reference region (transmission axis direction = 0 °), but preferably 45 °, 90 °, 135 The direction of the transmission axis is shifted. That is, forming the transmission axis direction evenly shifted means that when there are M areas in one unit, the transmission axis direction of each area is shifted by π / M. Also, the smaller this deviation angle is, the more it becomes possible to deal with various polarization angles that the input light can have, so a deviation of 45 ° or less, preferably 36 ° or less, more preferably 30 ° or less. The shift is good. Examples of the number of regions constituting one polarizer unit include 3-180, 3-90, 3-45, 3-30, etc., preferably 36, 25, 16, Nine and four.

[8] As a polarization imaging apparatus according to the present invention,
A light shielding portion is provided at a boundary portion of each region of the polarizer unit, or a region of the light receiving element array corresponding to a boundary portion of each region of the polarizer unit is shielded, and light is diffracted or scattered at the boundary portion. The polarization imaging apparatus of [1] or [5] that suppresses the influence of the above is mentioned.

[9] As a polarization imaging apparatus according to the present invention,
The polarizer unit is a multilayer structure in which two or more transparent materials are alternately stacked in the z direction on a single substrate parallel to the xy plane in an orthogonal coordinate system x, y, z. Each layer is divided into three or more polarizer regions, and each layer has a one-dimensional periodic uneven shape repeated in one direction in the xy plane determined for each region,
The input light is incident on the xy plane;
The polarization imaging apparatus according to any one of [1] to [8] is included.

[10] As a polarization imaging apparatus according to the present invention,
In the polarizer unit, each region is configured by a wire grid polarizer.
The polarization imaging apparatus according to any one of [1] to [8] is included.

  Each region (polarizer) constituting the polarizer unit is not particularly limited as long as it has a function as a polarizer. For example, a polarizer formed by a self-cloning technique, a wire grid polarizer, etc. Is mentioned.

[11] As a polarization imaging apparatus according to the present invention,
The polarization imaging apparatus according to any one of [1] to [10], wherein the light receiving element array is any one of a photodetector, a CCD, a C-MOS, and an imaging tube.
Regarding the correspondence between the polarizer (region) and the light receiving element in the polarizer unit, it is preferable that at least one light receiving element corresponds to one polarizer. Further, two or more light receiving elements may correspond to one polarizer. Adjacent light receiving elements do not necessarily have the same optical characteristics, and there are many individual differences in sensitivity and noise intensity. Therefore, by assigning two or more light receiving elements to one polarizer, it is possible to average the variation among the plurality of light receiving elements, and improve the S / N ratio when calculating the polarization intensity, the polarization angle, and the like. be able to.

[12] As an application example of the polarization imaging apparatus according to the present invention,
Comprising the polarization imaging apparatus of [1],
There is an imaging device in which the image processing unit separates the polarization component and the non-polarization component constituting the input light to obtain the non-polarization component.

  For example, when shooting a product in a show window, the reflected light from the show window is polarized, while the reflected light from the product in the window is often not polarized. By analyzing the polarization state, only the non-polarized component can be separated and extracted, and the reflected image from the glass (polarized component) can be removed to obtain an optimum image. In addition, when the show window is a curved glass or the like, the polarization angle of the reflected light and the like differ depending on the difference in the reflection part in the window. Even in such a case, by using a polarizer unit corresponding to a plurality of polarization angles and an array plate thereof, various polarized reflected lights from the window can be separated.

[13] As an application example of the polarization imaging apparatus according to the present invention,
Comprising the polarization imaging apparatus of [4],
A surface shape measuring device that further detects the surface shape of an object by mapping the angle of the polarization direction of the polarization component for each polarizer unit by the image processing unit.

[14] As an application example of the polarization imaging apparatus according to the present invention,
Comprising the polarization imaging apparatus of [4],
A surface property measurement apparatus that further detects the continuity of a substance constituting the surface of an object by mapping an angle of a polarization direction of the polarization component for each polarizer unit by the image processing unit.

  Since the input light input to a certain polarizer unit is input to each region with a uniform polarization angle, the polarized input light passes through any region (or has a distribution for each region). come. Therefore, by analyzing which region the input light has passed through, the polarization angle of the input light can be obtained, and a map of the polarization angle for each polarizer unit can be created. Reflected light from places where the surface shape is different from other parts (for example, scratches on a smooth surface) is often in a different polarization state compared to reflected light from other parts. In the case of an overview, the abnormal part can be detected immediately. In addition, the reflected light from a location where the substance constituting the surface of the object is different from other parts (for example, when the surface is constituted by multiple substances) is different from the reflected light from other locations. Therefore, when this map is overviewed, the location of the different substance can be detected immediately.

[15] As an application example of the polarization imaging apparatus according to the present invention,
Comprising the polarization imaging apparatus of [4],
The image processing unit further maps the angle of the polarization direction of the polarization component for each polarizer unit, and in the map, the predetermined polarization angle is uniform over a predetermined range or continuously. An imaging device that detects a changing portion is mentioned.

[16] As an application example of the polarization imaging apparatus according to the present invention,
[15] including a photographing device;
The image processing unit further specifies a location where a predetermined polarization angle is uniform over a predetermined range in the map or a location where the polarization angle changes continuously as a road surface, and is located at a location specified as the road surface. An obstacle detection device on a road surface that detects an object is mentioned.

[17] As an application example of the polarization imaging apparatus according to the present invention,
[15] including a photographing device;
The image processing unit further specifies a place where a predetermined polarization angle is uniform over a predetermined range in the map or a place where it continuously changes as a road surface,
Further, in the portion corresponding to the location specified as the road surface in the map by the image processing unit, a location where the intensity relating to the polarization component is greater than a predetermined value or a location where the intensity relating to the non-polarization component is less than a predetermined value is liquid. A road surface state detection device that determines a location where a component exists and determines a location where the intensity related to the polarized component is smaller than a predetermined value or a location where the intensity related to the non-polarized component is higher than a predetermined value as a location where a solid component exists. Can be mentioned.

[18] As an application example of the polarization imaging apparatus according to the present invention,
Comprising the polarization imaging device according to any one of [1] to [3],
The image processing unit separates the polarization component and the non-polarization component constituting the input light, and maps the intensity related to the polarization component or the intensity related to the non-polarization component for each polarizer unit. Is mentioned.

[19] As an application example of the polarization imaging apparatus according to the present invention,
[18] including a photographing apparatus;
The image processing unit further determines, in the map, a location where the intensity relating to the polarization component is greater than a predetermined value or a location where the intensity relating to the non-polarization component is less than a predetermined value as a location where a liquid component exists, and the polarization A weather observation imaging apparatus that determines a location where the intensity related to the component is smaller than a predetermined value or a location where the intensity related to the non-polarized component is greater than the predetermined value as a location where a solid component exists is cited.

[20] As an application example of the polarization imaging apparatus according to the present invention,
Comprising the polarization imaging apparatus of [5],
The image processing unit further detects, as a moving body, information from a location where a polarizer unit having a transmitted light intensity difference between adjacent regions larger than a predetermined value is adjacent and continuous over a predetermined range. An imaging device for detecting a moving object is exemplified.

  According to the present invention, in addition to simultaneously obtaining luminance information and a plurality of polarization information, the polarization component and the non-polarization component constituting the light from the subject are further separated, or a specific component is extracted. By doing so, a polarization imaging apparatus with improved image processing accuracy can be obtained.

  For example, when shooting a product in a show window, the reflected light from the show window is polarized, while the reflected light from the product in the window is often not polarized. By analyzing the polarization state, it is possible to separate and mainly extract reflected light from the product to obtain an optimum image. In addition, when the show window is a curved glass or the like, the polarization angle of the reflected light and the like differ depending on the difference in the reflection part in the window. Even in such a case, by using a polarizer unit corresponding to a plurality of polarization angles and an array plate thereof, various polarized reflected lights from the window can be separated. In addition, it is possible to create a map of the polarization angle for each polarizer unit, where the surface shape is different from other parts (for example, scratches on a smooth surface), and where the material constituting the surface of the object is different from other parts An abnormal location such as a case where the surface is composed of a plurality of substances can be detected.

  Further, by separating a polarizer unit having a transmitted light intensity difference between adjacent regions larger than a predetermined value and a polarizer unit smaller than a predetermined value, a portion where a polarization component is relatively contained in a captured image is relatively different. It is possible to distinguish from a portion that is included in a small amount. Thereby, for example, a vehicle (moving body) can be detected as a result of being able to identify a portion having a large amount of polarization components such as glass of the vehicle (moving body) from a landscape including a moving body such as a vehicle.

  Further, a map of the polarization angle for each polarizer unit can be created, for example, a location where the predetermined polarization angle is uniform over a predetermined range or a location where it continuously changes in the map, Furthermore, the detection location can be specified as a road surface, and an object located at the location specified as the road surface can be detected. In addition, by specifying the detection location as a road surface and analyzing the intensity of the polarized light component or non-polarized light component at the location specified as the road surface, there is a water surface or ice surface on the road surface based on the analysis result. It is possible to determine whether or not to do so. As a result, obstacles on the road surface, road surface conditions, and the like can be detected by only performing image processing on a part of the captured image, and it is not necessary to perform image processing on the entire captured image. Note that the detection of the road surface state uses a function for discriminating between a liquid component and a solid component, which will be described below, so that an algorithm for further improving the discrimination accuracy may be required as will be described later.

  Further, it is possible to create a map of intensity related to the polarization component (or non-polarization component) for each polarizer unit. For example, the liquid component is relatively large due to the magnitude of the intensity related to the polarization component (or non-polarization component) in the map. It is possible to distinguish between a location where the solid component exists and a location where a relatively large amount of the solid component exists. Furthermore, the location where the liquid component is present and the location where the solid component is present can be more clearly identified by adopting an algorithm for improving the identification accuracy, or considering other information, for example. . As a result, it is possible to determine components (for example, water vapor, sand, dust, etc.) constituting the cloud when the cloud is photographed.

<Outline of device configuration>
Hereinafter, a polarization imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic configuration diagram of a polarization imaging apparatus according to an embodiment of the present invention. As illustrated in FIG. 1, the polarization imaging apparatus 100 includes a light receiving module 103 including a polarizer array 101 and a light receiving element array 102 that receives light from the polarizer array, and information on light from the light receiving element array 102 (polarization). Component and non-polarized light component).

  The image processing unit 104 is based on a command received from a control program in the main memory 106 by a calculation unit 105 such as a CPU provided in the polarization imaging apparatus, storage information read from a storage unit such as a memory, and the like. The image processing described in detail below is performed. The obtained processed image is output from an output unit 107 such as a monitor or a printer in response to a command from the calculation unit 105.

  Next, each component for realizing the polarization imaging apparatus, that is, the image processing principle in the light receiving module 103 including the polarizer array 101 and the light receiving element array 102 and the image processing unit 104 will be described.

<Light receiving module>
FIG. 2 is a conceptual diagram of a polarizer made of a photonic crystal. First, a polarizer made of a photonic crystal will be described. A transparent high-refractive index medium 202 and a low-refractive index medium 203 are alternately stacked on a transparent material substrate 201 having a periodic groove array as shown in FIG. 2 while preserving the shape of the interface. Each layer has periodicity in the x direction, but may be uniform in the y direction, or may have a periodic or aperiodic structure having a length larger than that in the x-axis direction. Such a fine periodic structure (photonic crystal) can be produced with high reproducibility and high uniformity by using a method called a self-cloning technique (see Japanese Patent Laid-Open No. 10-335758).

  When non-polarized light or elliptically polarized light is incident on the periodic structure thus created perpendicularly or obliquely to the xy plane, the polarized wave parallel to the groove array, that is, the y-polarized light, and the x-polarized light orthogonal to the polarized wave, Each of TE mode light and TM mode light is excited inside the periodic structure. The propagation constant of the TE mode and the TM mode can be selected in a wide range depending on the refractive index of the material constituting the periodic structure, the period of the xy plane, and the stacking period.

FIG. 3 is a band diagram showing the propagation characteristics of the photonic crystal shown in FIG. This figure shows an example of a dispersion curve of a two-dimensional periodic structure when Si is used as the high refractive index material and SiO ₂ is used as the low refractive index material. The vertical axis is the value obtained by normalizing the reciprocal of the wavelength λ by the stacking period Lz, and the horizontal axis is the phase change amount k _z L _z (k _z is the propagation constant in the z direction) when propagating one period is normalized by π. Value. White circles indicate TE waves and black circles indicate TM waves. L _x represents a period in the in-plane direction, and here, L _z / L _x = 1.

  If the frequency of the incident light is within the band gap, the mode cannot propagate in the periodic structure and the incident light is reflected or diffracted. On the other hand, if the frequency of light is within the photonic band, light can pass through the periodic structure. In the frequency region 301, the TE wave is reflected as a band gap, and the TM wave is transmitted because it is a propagation region. Therefore, the polarization separation element (Japanese Patent Laid-Open Nos. 2001-83321 and 2000-056133 (Patent No. 3288976)) is transmitted. See). In the frequency domain 302, it operates as a polarizer that transmits TE waves and reflects TM waves. Features of the polarizer of this structure include a high extinction ratio of transmitted light, a thin and light weight, and can be formed on an arbitrary substrate. Up to now, numerical simulations and experiments have utilized a region of 301 on the high frequency side in particular, and a high extinction ratio of 50 dB has been realized with a small number of stacks of 10 cycles. On the other hand, in the frequency region 303, both the TE wave and the TM wave become a propagation region and are transmitted. However, in this case, since the two curves are deviated from each other, the propagation constants are different from each other, and it operates as a wave plate that gives a phase difference between the two modes.

For polarizers and wave plates made of photonic crystals, the operating wavelength range can be freely set by adjusting the refractive index, filling factor, groove row period L _x , and laminating direction period L _z of the constituent materials. Can do. For example, it is possible to design a wave plate that gives an arbitrary phase difference by appropriately designing the pattern of the substrate, the material used for film formation, the lamination period and the number of laminations, and the phase difference becomes π / 2, for example. By doing so, it can be operated as a quarter-wave plate. Furthermore, since the period and direction of the groove can be changed independently for each region in one substrate, the characteristics of the photonic crystal can be changed for each region. This is called a multi-pattern photonic crystal. For example, the polarizer can change the optical axis direction for each region, and the wavelength plate can change the optical axis direction and phase difference.

As a low refractive index medium constituting a photonic crystal, a material having SiO ₂ as a main component is the most common, a transparent wavelength region is wide, and it is chemically, thermally, and mechanically stable. Can also be done easily. Further, as the low refractive index medium, other optical glass, for example, a material having a lower refractive index such as MgF ₂ may be used. As the high refractive index material, a semiconductor such as Si or Ge, or an oxide or nitride such as Ta ₂ O ₅ , TiO ₂ , Nb ₂ O ₅ , HfO ₂ , or Si ₃ N ₄ can be used. Since the semiconductor material has a large refractive index, there is an advantage that a large band cap can be obtained, but the use wavelength range is limited to the near infrared. On the other hand, since oxides and nitrides have a wide transparent wavelength range, they can be used even in the visible light region.

  When a photonic crystal polarizer is manufactured by the self-cloning method, first, periodic grooves as shown in the substrate 201 of FIG. 2 are formed on the substrate by electron beam lithography and dry etching. For the formation of the groove pattern, other photolithography, interference exposure, or a stamping technique using a mold may be used. In FIG. 2, the cross-sectional shape of the groove is rectangular, but other shapes such as a triangle may be used. As the substrate, Si, quartz glass, other optical glass, or the like can be used. The pitch of the unevenness is about half of the wavelength of incident light, for example, about 0.4 μm for 0.8 μm light, and the depth of the groove is about 0.2 μm.

  The aperture area and transmission axis of the polarizer described above can be freely designed according to the size and direction of the groove pattern to be processed on the substrate first. Pattern formation can be performed by various methods such as electron beam lithography, photolithography, interference exposure, and nanoprinting. In any case, the direction of the groove can be determined with high accuracy for each minute region. Therefore, it is possible to form a polarizer unit in which micro-polarizers having different transmission axes are combined, and further a polarizer array in which a plurality of them are arranged. In addition, since only a specific region having a concavo-convex pattern operates as a polarizer, if the peripheral region is flat or isotropic concavo-convex pattern in the plane, light can be used as a medium having no polarization dependency. To Penetrate. Therefore, a polarizer can be formed only in a specific region.

An alternating multilayer film is laminated on the substrate 201 by using a target such as Ta ₂ O ₅ and SiO ₂ in combination with sputtering deposition and bias sputtering. At this time, it is important to appropriately set the bias condition so that the periodic uneven shape of each layer is preserved in the x-axis direction. This self-cloning technique (see Japanese Patent Application Laid-Open No. 10-335758 (Japanese Patent No. 3325825)) is an excellent technique for producing an industrially fine periodic structure (photonic crystal) with high reproducibility and uniformity. is there. The reason why the regular laminated structure is formed on the substrate is that (1) deposition by neutral incidence of neutral particles from the target, (2) sputter etching by perpendicular incidence of Ar ions, and (3) deposition particles This can be explained by the superposition of the three actions of redeposition.

As an example of the production conditions, for example, a gas pressure of 0.27 Pa (2 mTorr) is used for the Ta ₂ O ₅ layer film formation, a target applied high frequency power of 300 W, and a gas pressure of 0.80 Pa (6 mTorr) is formed for the SiO ₂ layer film formation. The target application high frequency power is 300 W, and the sputter etching is performed after the SiO ₂ layer is formed.

  Each region (polarizer) constituting the polarizer unit is not particularly limited as long as it has a function as a polarizer. Examples of the polarizer include, for example, a wire grid type polarizer in addition to the above-described photonic crystal formed by the self-cloning technique.

  A wire grid type polarizer is a polarizer formed by periodically arranging thin metal wires, and has been conventionally used in the millimeter wave region of electromagnetic waves. The structure of the wire grid polarizer has a structure in which fine metal wires that are sufficiently thin compared to the wavelength of the input light are arranged at intervals that are sufficiently short compared to the wavelength. It is already known that when light is incident on such a structure, polarized light parallel to the metal thin wire is reflected and polarized light orthogonal thereto is transmitted (see, for example, US Pat. No. 6,122,103). About the direction of a metal fine wire, since it can change and produce for every area | region within one board | substrate, the characteristic of a wire grid polarizer can be changed for every area | region. If this is utilized, it can be set as the structure which changed the direction of the transmission axis for every area | region like the polarizer unit in this embodiment, for example.

  As a method for manufacturing the wire grid, a thin metal can be left by forming a metal film on a substrate and performing patterning by lithography. As another manufacturing method, a groove is formed on the substrate by lithography, and a metal is deposited by vacuum deposition from a direction perpendicular to the direction of the groove and inclined from the normal line of the substrate (a direction oblique to the substrate surface). Can be produced. In vacuum deposition, particles flying from the deposition source hardly collide with other molecules or atoms in the middle of the deposition, and the particles travel linearly from the deposition source to the substrate. On the other hand, at the bottom part (concave part) of the groove, the film is shielded by the convex part and hardly formed. Therefore, by controlling the amount of film formation, a metal film can be formed only on the convex portion of the groove formed on the substrate, and a thin metal wire can be produced.

  Examples of other production methods include the following examples. In order to produce a substrate having irregularities on the surface, a resist is usually applied on the substrate, exposure and development are performed by lithography, and patterning is performed on the base substrate while leaving the resist by dry or wet etching. Do. Therefore, normally, the resist exists on the upper part of the convex portion on the substrate when the etching process is completed. In this state, when a metal film is formed from the vertical direction of the substrate by vacuum evaporation, the metal film is formed on the resist and on the bottom of the groove, but is hardly formed on the side surface (wall surface) of the groove. Next, by dissolving the resist with a solvent, the metal film on the resist can be peeled off, leaving the metal only at the bottom of the groove.

  As described above, a fine metal wire can be produced. In the interference exposure method, the fine metal wire can be produced by changing the transmission axis for each minute region, for example, by masking each region on the substrate. What is necessary is just to implement the said method. For example, when a polarizer array is produced by arranging a plurality of polarizer units having four polarizer regions in parallel, after forming a metal film on the substrate, the first region is opened and the second to second regions are formed. By using a mask that shields the region 4 to produce a fine metal wire in the first region, and repeating this sequentially, a fine metal wire having a different transmission axis can also be produced in the second to fourth regions. it can. Similarly, a method of forming a groove (unevenness) on a substrate can be manufactured by using a mask for each region.

  The wire metal used in the wire grid polarizer is preferably aluminum or silver, but the same phenomenon can be realized with other metals such as tungsten. In addition, as lithography, optical lithography, electron beam lithography, X-ray lithography, and the like can be given. However, when an operation with visible light is assumed, the interval between thin lines is about 100 nm, and thus electron beam lithography or X-ray lithography is more preferable. . Also, vacuum deposition is desirable for metal film formation. However, since the directionality of particles incident on the substrate is important, sputtering in a high vacuum atmosphere or collimation sputtering using a collimator is also possible.

  The aperture area and transmission axis of the polarizer described above can be freely designed according to the size and direction of the groove pattern to be processed on the substrate first. Pattern formation can be performed by various methods such as electron beam lithography, photolithography, interference exposure, and nanoprinting. In any case, the direction of the groove can be determined with high accuracy for each minute region. Therefore, it is possible to form a polarizer unit in which micro-polarizers having different transmission axes are combined, and further a polarizer array in which a plurality of them are arranged. In addition, since only a specific region having a concavo-convex pattern operates as a polarizer, if the peripheral region is flat or isotropic concavo-convex pattern in the plane, light can be used as a medium having no polarization dependency. To Penetrate. Therefore, a polarizer can be formed only in a specific region. One or more rows can be produced.

  In the following description, an example in which a photonic crystal is used as a region (polarizer) constituting the polarizer unit will be described. In addition, even if it is a wire grid type polarizer and another polarizer, it can implement by the same description.

  FIG. 4 is a schematic overview of a light receiving module including a polarizer array in which a plurality of polarizer units having four types of transmission axes are arranged, and a light receiving element array. As shown in FIG. 4, the light receiving module is configured by overlapping a polarizer array 401 and a light receiving element array 402. In FIG. 4, the respective arrays are illustrated separately for the sake of explanation.

First, the polarizer array 401 will be described. The polarizer has the structure shown in FIG. 2, and in the orthogonal coordinate system x, y, z, a multilayer structure in which two or more transparent materials are alternately stacked in the z direction on one substrate parallel to the xy plane. A body (for example, an alternating multilayer film of Ta ₂ O ₅ and SiO ₂ ). The polarizer unit 407 is divided into three or more regions in the xy plane, and in this embodiment, four regions 403 to 406. In each region, each film has an uneven shape, and this uneven shape is a region. It is formed by being repeated periodically in one direction within the xy plane determined every time.

  In the first region 403 (reference region), the groove direction is 0 ° with respect to the x-axis (the transmission axis in the reference region), the second region 404 is 45 °, the third region 405 is 90 °, A region 406 of 4 is 135 °. (It is not necessary to use the x axis as a reference, and it can be defined using another axis as a reference.) However, the arrangement order is not limited and can be freely arranged. It suffices if the sub unit 407 has regions in which the angles in the groove direction are 0 °, 45 °, 90 °, and 135 °. Each region operates as the photonic crystal polarizer described above. That is, from the input light incident on the xy plane, polarized components having different polarization directions are transmitted through each region, and equal amounts of unpolarized components are transmitted through the entire region. In the present embodiment, there are four types of transmission axes of the polarizer, but the axis directions of the unevenness may be three types of 0 °, 60 °, and 120 °, or four or more directions.

Here, the photonic crystal polarizer is used for camera photography, for example, the in-plane period is 0.44 μm, the stacking period is 0.44 μm, and the film thickness ratio of Ta ₂ O ₅ and SiO ₂ is 4: 6. An example is given in which the wavelength is designed to operate in a wavelength band of 0.8 μm, for example. When light is incident on the structure of FIG. 2 from the vertical direction, polarized light parallel to the groove is reflected and attenuated in the transmission direction, and the attenuation rate becomes about 30 dB in 15 periods. On the other hand, polarized light perpendicular to the grooves propagates, and the transmission loss becomes 0.1 dB or less in 10 cycles. However, there is flexibility in selecting materials and structural parameters. For example, Ta ₂ O ₅ , TiO ₂ , and Nb ₂ O ₅ may be used as the high refractive index material, and Pyrex (registered trademark) glass, MgF ₂ , and other optical glasses may be used as the low refractive index. If the operating wavelength is infrared, Ge, SiGe or the like can be used as a high refractive index material. Further, the dispersion relation of each polarization depends on the film thickness ratio, the in-plane period, the stacking period, and the angle of the slope, and the wavelength band that operates as a polarizer changes. Therefore, it can be designed and manufactured for any wavelength band from visible / ultraviolet to infrared.

  Here, the groove formation on the substrate surface can use electron beam lithography and reactive etching. In addition, photolithography can be formed by selecting a light wavelength suitable for the pitch. The size of each region constituting the polarizer array is 50 μm square, but may be larger (for example, 1000 μm square) or smaller (for example, 5 μm square). Moreover, not only a square pattern but also a triangle, a rectangle, a hexagon, etc. are arbitrary. In this way, it is possible to form a polarizer array having different transmission polarization directions for each region.

  The polarization state is measured by mounting the polarizer array 401 on the light receiving element array 402 in which the light receiving elements (pixels) are arranged in the same cycle. Regarding the correspondence between the polarizer (region) and the light receiving element in the polarizer unit, it is preferable that at least one light receiving element corresponds to one polarizer. Further, two or more light receiving elements may correspond to one polarizer. Adjacent light receiving elements do not necessarily have the same optical characteristics, and there are many individual differences in sensitivity and noise intensity. Therefore, by assigning two or more light receiving elements to one polarizer, it is possible to average the variation among the plurality of light receiving elements, and improve the S / N ratio when calculating the polarization intensity, the polarization angle, and the like. be able to. The light receiving element may be a CCD, a semiconductor photodiode, a C-MOS, or an image pickup tube. In the case of a CCD, since the size of one region (pixel) is several μm to several tens of μm, the polarization state of light can be observed as highly accurate image information by combining with a polarizer array. As a result, it is possible to observe the polarization state of the reflected light such as reflected light and transmitted light from the substance, the ground surface, and the water surface. For example, it is possible to transmit or reflect light through glass, an optical disk, or other structures, and measure the magnitude of change in the polarization state due to the birefringence induced by strain. It is also possible to perform micro polarization analysis by incorporating it into a microscope.

  FIG. 5 is a diagram showing a polarizer unit having four types of regions whose transmission axes are separated by non-transparent regions, and a light receiving element array combined therewith. Since the polarizer array used in this embodiment has a plurality of different regions arranged, the boundary between the regions is discontinuous, and light scattering and diffraction occur. Scattered light and diffracted light appear as noise in signal processing and cause deterioration in the accuracy of ellipsometry. For this reason, in order to realize a highly accurate apparatus, there is a technique in which a light shielding region is arranged in a polarizer array or a light receiving element array used in a polarization imaging apparatus so that scattered light and diffracted light are not received.

  As shown in FIG. 5, an opaque region 508 is arranged at the boundary between the regions 503 to 506 in the polarizer unit 507, and light incident on this portion receives light receiving elements (pixels) constituting the light receiving element array 502. ) 509 cannot be reached. As another example, scattered light and diffracted light can be removed by similarly providing a non-transparent region in a region where light from the boundary portion of the polarizer array is incident in the light receiving element array 502. It is important that the light detected by each light receiving element is light that has passed through the corresponding polarizer, that is, it is important that crosstalk is small. However, this problem can be solved by providing an opaque region as described above. can do,

<Principle of image processing in image processing unit>
Next, the principle of image processing in the image processing unit 104 will be described. FIG. 6 is a schematic configuration diagram of a polarizer array and a light receiving element array for explaining the image processing method according to the embodiment. As shown in FIG. 6, there are a plurality of polarizer units 607 constituting a polarizer array 601, and the polarizer unit 607 has four polarizer regions 603 to 606. On the other hand, the light receiving element array 602 has a plurality of light receiving elements, and the light receiving element 602-2 (area number m = 0) receives the transmitted light from the area 603, and the light receiving element 602-1 (area number m = 1). Receives the transmitted light from the region 604, the light receiving element 602-2 (region number m = 2) receives the transmitted light from the region 605, and the light receiving element 602-2 (region number m = 3) from the region 606. The transmitted light is received.

For convenience of explanation, the polarizer unit and the light receiving element unit corresponding thereto are represented by coordinates i and j, the coordinates of the polarizer unit 607 in the polarizer array, and the light receiving element unit corresponding to the polarizer unit 607 in the light receiving element array. Let both coordinates be coordinates (i, j). If the transmitted light intensity data obtained from the polarizer unit 607 at coordinates (i, j) is fm (i, j), the data f ₀ (i, j) regarding the four directions of each region is obtained from the polarizer unit 607. , F ₁ (i, j), f ₂ (i, j), and f ₃ (i, j) are obtained. The intensity fm (i, j) of the transmitted light is the intensity of the polarization component that is different for each region (the maximum intensity (vibration width) of the polarization component is 2A (i, j) and the amplitude is A (i, j). ) And the intensity B (i, j) of the non-polarized component that is uniform in the entire region. The intensity fm (i, j) of the transmitted light can be expressed by the following formula (3).

Here, m is a number assigned to each region, i and j are coordinates of the polarizer unit in the polarizer array, and θm is a repetitive direction (transmission axis) in a reference region among the regions. ) Is the angle of the repetitive direction (transmission axis) in each other region when 0 °, and θ (i, j) is the polarization direction of the polarization component input to the polarizer unit, and the reference It is an angle difference with the repetitive direction (transmission axis) in the area to be performed.

  In this embodiment, since each region is produced with a uniform direction shift, θm in the above equation (3) can be πm / M. This is represented by the following formula (4). Here, M is the number of regions in one polarizer unit. For example, when four regions are formed with a uniform direction shift, they are 0 °, 45 °, 90 °, and 135 °. For example, when M = 9, the angles are 0 °, 20 °, 40 °, 60 °, 80 °, 100 °, 120 °, 140 °, and 160 °.

  Here, the intensity A (i, j), the intensity B (i, j), and θ (i, j) change with a period larger than the size of one polarizer unit. It can be considered uniform within the unit. That is, in one polarizer unit (when i and j are fixed), the values other than the variable m are constant. FIG. 7A is a graph of the above formula (3) or the special formula (4), with the horizontal axis being m and the vertical axis being fm (i, j). As can be seen from FIG. 7A, the above formula (3) or the above formula (4) relates to a polarization component whose transmission intensity varies depending on the angle of the transmission axis for each region to a predetermined amount of intensity B (i, j). Intensity distribution is added.

  Therefore, in the image processing unit 104, the intensity of the transmitted light transmitted through each region fm (i, j) with respect to the angle of the concavo-convex shape formed in each region constituting the polarizer unit is expressed by the above formula ( 3) or by applying the mathematical model represented by the above formula (4), the intensity of the transmitted light is changed into the intensity A (i, j) relating to the polarization component and the intensity B (i, j) relating to the non-polarization component. To separate. As a result, a desired image can be obtained by freely reconstructing these separated components.

Next, an example of a method for applying the mathematical model will be described in detail.
In the image processing unit 104, first, all the intensities fm (i, j) of the transmitted light transmitted through the respective areas constituting the polarizer unit are added and divided by the number of areas, thereby obtaining the intensity fm (i, j of the transmitted light. ) Average value (indicated by a symbol with f (i, j) overlined). Here, as can be seen from FIG. 7A, the average value of the transmitted light intensity fm (i, j) is the sum of the intensity A (i, j) and the intensity B (i, j). Therefore, the average value of the transmitted light intensity fm (i, j) can be expressed by the following equation (5).

  The above equation (3) is a function of the intensity A (i, j) and the intensity B (i, j) and θ (i, j). By using the above equation (5), the intensity A (i, j ) And θ (i, j). That is, it can deform | transform into following formula (6).

Here, m is a number assigned to each of the regions, i and j are coordinates of the polarizer unit in the polarizer array, and θm is a repeat direction in a reference region of the regions of 0 °. And θ (i, j) is the angle between the polarization direction of the polarization component input to the polarizer unit and the repeat direction in the reference region. It is a difference.

In this embodiment, since each region is produced with a uniform direction shift, θm in the above equation (6) can be πm / M. This is represented by the following formula (7).

  In this way, the image processing unit 104 constitutes a polarizer unit using the above equations (6) and (7) simplified to the functions of the intensity A (i, j) and θ (i, j). The intensity obtained by subtracting the average value from the intensity fm (i, j) of the transmitted light transmitted through each area with respect to the angle of the concavo-convex shape formed in each area in the repeating direction is expressed by the above formula (6) or By applying the mathematical model represented by the above formula (7) (see FIG. 7B), the intensity A (i, j) relating to the polarization component is extracted from the intensity of the transmitted light. Further, an angle difference θ (i, j) between the polarization direction of the polarization component input to the polarizer unit and the repetition direction in the reference region is calculated. The predetermined mathematical model can be fitted by, for example, Fourier analysis or least square method. By extracting the intensity A (i, j), the remaining variable intensity B (i, j) can also be extracted. As a result, a desired image can be obtained by freely reconstructing these separated components.

<Photographing device using polarization imaging device>
Next, a photographing apparatus to which the polarization imaging apparatus is applied will be described. The configuration of the imaging apparatus is the same as that of the polarization imaging apparatus described above. In the image processing unit, the polarization component and the non-polarization component constituting the input light to the apparatus are separated to obtain the non-polarization component. Yes.

  FIG. 8 is an example of a captured image obtained from each region in the polarizer unit. FIG. 9 is an example of a processed image that is reconstructed by separating and extracting input light input to the polarization imaging apparatus according to the embodiment of the present invention. FIG. 9A is an example of an image in which a person inside the vehicle is extracted by separating and removing the polarization component from the input light, and FIG. 9B is an example of an image in which only the polarization component is extracted. (C) is an image example obtained by mapping the polarization angle for each polarizer unit.

As described above, the polarizer unit is composed of four regions, and the transmitted light intensity data f ₀ (i, j), f ₁ (i, i, j), f ₂ (i, j), f ₃ (i, j) are obtained. Further, since the polarizer array is configured by arranging i × j polarizer units, for example, the data f _{0 to} f ₃ constitute i × j map-like data. This map-like data is shown in FIGS. That is, each image shown in FIG. 8 is an image obtained by mapping the intensity of light transmitted through a polarizer having a certain transmission axis.

  In general, for example, when photographing the interior of a vehicle, in addition to the reflected light from the object inside the vehicle (the light beam that you want to detect), the reflected light from the windshield and side glass (the light beam that causes noise and reflections) is received. Detected by the element. Here, in addition to the problem that the degree of polarization of reflected light from an object inside the vehicle differs from the degree of polarization of reflected light from a windshield, etc., the reflected light to be removed also has a plurality of reflections having different polarizations. The problem is that it consists of light. That is, the front glass and the side glass have different surface orientations when viewed from the camera, the degree of polarization and the polarization state are different, and each glass often has a curved surface shape, and the reflected light from each part of the glass. Have different polarization degrees and polarization states.

  Referring to FIG. 8, in FIGS. 8A, 8 </ b> B, and 8 </ b> C, the surrounding background (right person or sunlight from the sky) is reflected on the side glass of the vehicle, The person image is unclear. This is because the reflected light from the side glass has reflected light having substantially the same polarization angle as the transmission axes in the regions m = 0, 1, and 2. On the other hand, since the reflected light from the person inside the vehicle is composed of a non-polarized component which is hardly polarized, any region is transmitted, and FIGS. 8 (A), (B), (C), and (D) ) In all images. Therefore, when it is desired to capture a person image inside the vehicle clearly, the transmitted light information from the area m = 3 shown in FIG. In this example, the surface shape of the side glass is a curved surface, so that the reflected light from the side glass does not have a uniform polarization angle, and is transmitted light information from the region m = 3. This is because there is a partial reflection. In such a case, an optimum image can be taken by selecting optimum information for each unit coordinate and performing image processing (see FIG. 9A).

  Further, in the present photographing apparatus, the description has been given as the information for obtaining the non-polarized component of the input light to the apparatus. However, for example, the polarization component that is the opposite component may be extracted, and the resulting image is obtained. As shown in FIG.

This photographing apparatus can be applied in various fields.
(1) A road surface wet by rain or the like is smoother than a road surface not wet and generates reflected light. This reflected light causes problems such as making it difficult to see the white lines and road signs on the road surface, which causes various obstacles for the driver. On the other hand, since the reflected light from the wet road surface is polarized, an image from which the reflected light from the road surface is removed can be taken in real time and provided to the driver.
(2) In recent years, iris identification has attracted attention. Iris identification is a system that recognizes and identifies a person using an iris. An iris is a cocoon that is formed in a chaotic shape outward from the pupil. The shape (pattern) of the cocoon is a pattern unique to that person, similar to fingerprints, and is therefore attracting attention as a means of personal recognition. Has been. As for this iris, the reflected light becomes an obstacle to identification when the eyes are wet by tears or the like. Even in such a case, the image capturing apparatus can capture an image from which the reflected light from the eye is removed, and can accurately identify the image.

<Surface shape measuring device using polarization imaging device>
Next, a surface shape measuring apparatus to which the polarization imaging apparatus is applied will be described. The configuration of the surface shape measurement device is the same as that of the polarization imaging device described above, and the image processing unit detects the surface shape of the object by mapping the polarization direction angle of the polarization component for each polarizer unit. To.

  Since the input light input to a certain polarizer unit is input to each region constituting the unit at a uniform polarization angle, the polarized input light has a distribution (or distribution for each region). ). For this reason, the polarization angle of the input light can be obtained by analyzing which region the input light has been transmitted through, what distribution is transmitted in each region, etc., and the polarization angle for each polarizer unit. Maps can be created. The resulting image is shown in FIG. Reflected light from places where the surface shape is different from other parts (for example, scratches on a smooth surface) is often in a different polarization state compared to reflected light from other parts. In the case of an overview, the abnormal part can be detected immediately.

  As a specific application example, there is a case where the quality of the fruit when the fruit is shipped is inspected. Since the value of a product is reduced if the surface of the fruit has scratches or the like, the presence or absence of scratches and the quality of the surface shape are quality control items. The presence or absence of this scratch and the quality of the surface shape appear as a difference in the polarization angle of the polarization component of the light from the fruit surface, so use this surface shape measurement device to create and identify the polarization angle map. Can do quality control.

  As another specific application example, there is a case where the quality of soldering is determined. A soldering operation is performed when an electronic component or the like is attached to the substrate. However, if the soldering is poor, there is a problem that an electric circuit does not operate. In general, the quality of soldering can be determined by the shape of the molten solder that connects the substrate and the electrical wiring extending substantially perpendicularly from the substrate over the periphery of the wiring. If it is (the solder surface is convex), it is considered as bad, and if it is Mt. Fuji (the solder surface is concave), it is good. This difference in the solder shape appears as a difference in the polarization angle of the polarization component of the light from the solder surface, so that a map of the polarization angle can be created and identified by using this surface shape measuring apparatus.

  When evaluating the surface shape of an object, there is a case where reflected light with high intensity is locally generated from the object surface (particularly reflected when light is applied to the surface of an apple or metal), and the dynamic range may be generated. Problem arises. Even in such a case, according to the surface shape measuring apparatus, information based on the reflected light with high intensity locally is deleted, and the surface shape is accurately obtained using only light from other locations. It can be measured.

<Surface texture measuring device using polarization imaging device>
Next, a surface property measuring apparatus using a polarization imaging apparatus will be described. The configuration of the surface texture measuring apparatus is the same as that of the polarization imaging apparatus described above, and the image processing unit detects the surface texture of the object by mapping the polarization direction angle of the polarization component for each polarizer unit. To.

  Since the input light input to a certain polarizer unit is input to each region constituting the unit at a uniform polarization angle, the polarized input light has a distribution (or distribution for each region). ). For this reason, the polarization angle of the input light can be obtained by analyzing which region the input light has been transmitted through, what distribution is transmitted in each region, etc., and the polarization angle for each polarizer unit. Maps can be created. Reflected light from a location where the material constituting the surface of the object is different from other portions (for example, when the surface is composed of multiple materials) is polarized differently than reflected light from other locations. Since the map is often in a state, when this map is overviewed, the location of the different substance can be detected immediately.

  As a result of ellipsometry, whether the reflected light is polarized due to the surface shape of the object (such as surface irregularities) or the surface properties of the object (such as changes in the materials constituting the surface). Since it is difficult to distinguish, it is preferable to make a determination in consideration of other information. For example, when discontinuity is observed in the map of the polarization state of the reflected light as a result of performing the above polarization analysis on the surface whose surface properties are known to be continuous from other information, It can be determined that this is due to the discontinuity of the surface shape. The reverse is also true.

<Photographing device applying polarization imaging device, obstacle detection device on road surface, road surface state detection device>
Next, an imaging device, a road surface obstacle detection device, and a road surface state detection device to which the polarization imaging device is applied will be described. The configuration of the imaging apparatus is the same as that of the polarization imaging apparatus described above. In the image processing unit, the angle of the polarization direction of the polarization component is mapped for each polarizer unit, and the predetermined polarization angle is predetermined in the map. A part that is uniform over a range or a part that continuously changes is detected. In the configuration of the obstacle detection device on the road surface, the image processing unit further specifies, as a road surface, a place where a predetermined polarization angle is uniform over a predetermined range or a place where it continuously changes in the map. The object located at the location specified as the road surface is detected.

  In addition, the configuration of the road surface state detection apparatus is further specified by the image processing unit as a road surface where a predetermined polarization angle is uniform over a predetermined range in the map or where the predetermined polarization angle changes continuously (first). In the portion corresponding to the location specified as the road surface on the map by the image processing unit, the portion where the intensity relating to the polarization component is greater than the predetermined value or the location where the intensity relating to the non-polarization component is less than the predetermined value is liquidated. It is determined that the component is present, and a portion where the intensity related to the polarization component is smaller than the predetermined value or a position where the intensity related to the non-polarized component is larger than the predetermined value is determined as the position where the solid component exists (second function). Yes.

  In recent years, vehicles have been installed with devices that detect obstacles and the like present on the road. With this device, for example, even an obstacle that can be visually recognized by the driver of the vehicle can be positively notified to the driver, or an obstacle that is not noticed by looking away can be notified. In addition, since the driver can be notified of obstacles that become the viewing angle, unexpected accidents can be prevented in advance. In such an apparatus used in an emergency, obstacle detection speed often becomes a problem. For example, when image processing is used for detection, it is preferable to increase the image processing speed.

  The obstacle detection device on the road surface detects an object on the road surface by first detecting a road surface where many obstacles are present from the photographed image and then image-processing only the portion of the road surface. In other words, since obstacles and the like can be detected only by performing image processing on a part of the captured image, it is not necessary to perform image processing on the entire captured image. As a result, the burden of image processing can be reduced, and the obstacle detection speed can be improved.

  The operating principle of this device is as follows. Since the entire road surface is a relatively flat surface, the polarization angle of the polarization component of the light from the road surface is uniform or continuously changing. Therefore, when the captured image captured by the present imaging apparatus is processed and the polarization angle of the polarization component is mapped, the entire road surface is a portion where the predetermined polarization angle is uniform over a predetermined range or continuously. It is expressed as a changed part. And the location which became uniform or the location which changed continuously can be specified as a road surface. On the other hand, for example, a road shoulder or the like existing at the end of the road surface is a discontinuous shape with the road surface, so that the polarization angle indicated by the road surface and the polarization angle indicated by the road shoulder are discontinuous. An appropriate value may be set for the “predetermined range”.

  Since the “polarization angle” element of the polarization component is an element defined by the relationship with the transmission axis of the polarizer, the polarization angle of the polarization component from the road surface varies depending on the angle of the photographing apparatus with respect to the road surface. However, in a road surface obstacle detection device (and a photographing device as an internal configuration thereof) mounted on a vehicle, the vehicle and the road surface are in a predetermined angular relationship (for example, the road surface and the vehicle roof are parallel). In connection with this, it has a known angle with respect to the road surface. That is, since the angle of the photographing apparatus with respect to the road surface is known, the polarization angle of the polarization component from the road surface can be predicted. In this embodiment, as a determination material for detecting a road surface, in addition to whether or not it is the “predetermined range” described above, whether or not it is a predicted polarization angle is used to improve detection accuracy. Can do. As a method for detecting an obstacle on the road surface after detecting the road surface, a known obstacle detection method may be used.

  Moreover, this road surface state detection apparatus detects a road surface state by the 1st function mentioned above and a 2nd function. The first function is a function of the above-described obstacle detection device on the road surface, and thereby, the road surface can be specified while reducing the burden of image processing. The second function is a function used in the weather observation imaging device described later (see the following description for details), and by analyzing the intensity related to the polarization component or the non-polarization component at the location specified as the road surface, Based on the analysis result, it can be determined whether there is a water surface or an ice surface on the road surface.

  For example, when there is a puddle and ice surface on a road surface such as asphalt, the smoothness of those surfaces is often the roughest asphalt, the smoothest puddle, and the ice surface is often between these . And the intensity | strength of a polarization component or a non-polarization component changes with the difference in these smoothness. Therefore, it is possible to determine whether there is a water surface or an ice surface on the road surface from the difference in strength. As will be described later, an algorithm or the like that further increases the accuracy of identifying the liquid component and the solid component may be required. For example, by further using information such as air temperature, the accuracy of detecting the road surface condition can be further increased. it can.

<Photographing device for weather observation using polarization imaging device>
Next, an imaging apparatus and a weather observation imaging apparatus to which the polarization imaging apparatus is applied will be described. The configuration of the imaging apparatus is the same as that of the polarization imaging apparatus described above. In the image processing unit, the polarization component and the non-polarization component constituting the input light are separated and the intensity related to the polarization component or the intensity related to the non-polarization component is set. Mapping is performed for each polarizer unit. Further, in the configuration of the imaging device for weather observation, the image processing unit further includes a liquid component in the map where the intensity relating to the polarization component is greater than the predetermined value or the intensity relating to the non-polarization component is less than the predetermined value. In addition, it is determined that a location where the intensity of the polarization component is smaller than a predetermined value or a location where the intensity of the non-polarization component is greater than a predetermined value is a location where the solid component exists.

  FIG. 10 is an example of a processed image obtained by separating and extracting the input light input to the polarization imaging apparatus according to the embodiment of the present invention into each component and reconstructing the input light. FIG. 10A is an example of a normal photographed image in which each component is not processed, and FIG. 10B is an image example in which only the polarization component is extracted and the intensity is mapped.

  As shown in FIG. 10, for example, a case where a cloud is photographed (FIG. 10A) will be described. The cloud is composed of, for example, water particles (water vapor), ice particles, sand and dust, and has a liquid component and a solid component as illustrated. Among these components, the liquid component typified by water grains corresponds to a subject having a smooth surface, while the solid component typified by ice grains, sand and dust is suitable for a subject having an uneven surface. Applicable. Therefore, as shown in FIG. 10B, when the captured image is processed and mapped with respect to the intensity of the polarization component, the cloud image can be represented by the intensity of the polarization component, and there are relatively many liquid components. And a place where a relatively large amount of solid components are present. However, weather phenomena and the like are phenomena in which various factors are intricately intertwined, and therefore, there are many cases where each component cannot be clearly separated and identified from one piece of information of intensity of polarization component. Therefore, in order to separate and identify each component more clearly, an algorithm that improves the identification accuracy of each component is further adopted (for example, “predetermined value” is set more accurately), or other information is used, for example. Is also preferably taken into account. Further, by analyzing other information such as observation information of yellow sand, for example, it is possible to perform a more detailed analysis such as whether the solid component is an ice grain or yellow sand or dust. An appropriate value may be set for the “predetermined value” for determining the liquid component and the solid component described above. Alternatively, a predetermined value for identifying the liquid component and a predetermined value for identifying the solid component may be set separately to identify a polarizer unit having an intensity between these predetermined values. Thereby, a more detailed weather observation can be performed. In addition, although mapping about the intensity | strength of a polarization component was demonstrated, even when it is set as the mapping about the intensity | strength of a non-polarization component, it can explain similarly.

<Polarization imaging apparatus according to another embodiment>
The above-described polarization imaging apparatus processes (for example, separation processing) a polarization component and a non-polarization component of input light and uses information obtained from each component. On the other hand, an embodiment in which useful information can be obtained without processing the polarization component and the non-polarization component of the input light will be described next.

  That is, a polarization imaging apparatus according to another embodiment of the present invention includes the polarizer array described above and the light receiving element array described above, and further, the intensity of transmitted light transmitted through each region is determined between adjacent regions. The polarization imaging apparatus includes an image processing unit that separates a polarizer unit having a transmitted light intensity difference larger than a predetermined value and a polarizer unit smaller than a predetermined value.

  As the polarizer array and the light receiving element array, the above-described various forms can be applied.

  Next, the operation of the image processing unit in the polarization imaging apparatus according to the present embodiment will be described with reference to FIG. As described above, the light from the subject obtained when the subject is photographed includes a non-polarization component and a polarization component. Here, when the light input to the polarizer unit 607 is light that contains a relatively large amount of non-polarized light components (for example, light from a subject having an uneven surface or a portion to be imaged), the polarizer unit 607 is configured. The intensity of light transmitted through each of the regions 603 to 606 is almost the same for each region. Therefore, the intensity of the transmitted light received by the corresponding light receiving elements 602-0 to 602-3 has almost no difference between the elements. On the other hand, when the light input to the polarizer unit 607 is light containing a relatively large amount of polarization components (for example, light from a subject having a smooth surface or a portion to be imaged), the polarizer unit 607 is configured. The intensity of light transmitted through each of the regions 603 to 606 varies greatly from region to region (see FIG. 7). Accordingly, the intensity of the transmitted light received by the corresponding light receiving elements 602-0 to 602-3 varies greatly from element to element.

  Using this, the intensity of the transmitted light transmitted through each region is extracted by extracting a polarizer unit in which the difference in transmitted light intensity between adjacent regions is larger than a predetermined value and a polarizer unit smaller than a predetermined value. By separating, a non-polarized component part (or a part containing a large amount of the component) and a polarized light component part (or a part containing a large amount of the component) in the captured image can be distinguished. An appropriate value may be set for this “predetermined value”. In addition, a predetermined value (upper predetermined value) for identifying a polarizer unit having a large intensity difference and a predetermined value (lower predetermined value) for identifying a polarizer unit having a small intensity difference are set separately, and the upper side A polarizer unit having an intensity difference between a predetermined value and a lower predetermined value may be identified.

  In addition, when two or more light receiving elements are installed for one polarizer, the intensity difference of transmitted light between “adjacent polarizers (regions)” even if light containing a large amount of polarization components is input. Although large, “adjacent light receiving elements” do not necessarily have a large intensity difference. This is because a plurality of light receiving elements corresponding to a certain polarizer all receive transmitted light having the same intensity. In this case, a plurality of light receiving element groups corresponding to one polarizer may be regarded as one unit, and the intensity difference may be calculated for the light receiving element groups adjacent to the light receiving element group.

<Moving Object Detection Imaging Device Applying Polarization Imaging Device According to Other Embodiment>
Next, a moving body detection imaging apparatus to which a polarization imaging apparatus according to another embodiment is applied will be described. The configuration of the moving body detection imaging apparatus is the same as that of the polarization imaging apparatus according to the other embodiments described above. In the image processing unit, the polarizer unit has a transmitted light intensity difference between adjacent areas larger than a predetermined value. Is detected as a moving object from a location that is adjacent and continuous over a predetermined range. The moving body detection imaging apparatus can be applied to a traffic monitoring system such as an N system.

  FIG. 11A is an example in which input light input to a polarization imaging apparatus according to another embodiment of the present invention is used as an image. FIG. 11B is an enlarged view of FIG. As shown in these figures, since the light from the windshield of a moving body such as a vehicle having a relatively smooth surface contains a large amount of polarization components, the difference in transmitted light intensity between adjacent polarizers (regions) is large. A mosaic image having clearly different intensities for each polarizer is obtained. And the polarizer unit which gives this mosaic-like image adjoins over a predetermined range so that it may correspond to the whole windshield. On the other hand, light from parts other than the vehicle having a relatively uneven surface contains a large amount of non-polarized light components, so that the difference in intensity of transmitted light between adjacent polarizers (regions) is small, and the intensity of each polarizer is almost the same. It becomes the same image. By processing the difference between these images, for example, a vehicle (moving object) can be identified from a landscape containing a moving object such as a vehicle, such as a glass (mobile object) with a large amount of polarization components. ) Can be detected. An appropriate value may be set for the “predetermined range”.

  The polarization imaging apparatus realized by the present invention can be used for a digital camera, a video camera, a surface shape measuring device, a surface property measuring device, and the like.

FIG. 1 is a schematic configuration diagram of a polarization imaging apparatus according to an embodiment of the present invention. FIG. 2 is a conceptual diagram of a polarizer made of a photonic crystal. FIG. 3 is a band diagram showing the propagation characteristics of the photonic crystal shown in FIG. FIG. 4 is a schematic overview of a light receiving module including a polarizer array in which a plurality of polarizer units having four types of transmission axes are arranged, and a light receiving element array. FIG. 5 is a diagram showing a polarizer unit having four types of regions whose transmission axes are separated by non-transparent regions, and a light receiving element array combined therewith. FIG. 6 is a schematic configuration diagram of a polarizer array and a light receiving element array for explaining the image processing method according to the embodiment. FIG. 7 is a diagram for explaining a method of applying a mathematical model in the image processing unit. FIG. 8 is an example of a captured image obtained from each region in the polarizer unit. FIG. 9 is an example of a processed image obtained by separating and extracting the input light input to the polarization imaging apparatus according to the embodiment of the present invention and extracting the components. FIG. 9A is an example of an image in which a person inside the vehicle is extracted by separating and removing the polarization component from the input light, and FIG. 9B is an example of an image in which only the polarization component is extracted. (C) is an image example obtained by mapping the polarization angle for each polarizer unit. FIG. 10 is an example of a processed image obtained by separating and extracting the input light input to the polarization imaging apparatus according to the embodiment of the present invention into each component and reconstructing the input light. FIG. 10A is an example of a normal photographed image in which each component is not processed, and FIG. 10B is an image example in which only the polarization component is extracted and the intensity is mapped. FIG. 11A is an example in which the input light input to the polarization imaging apparatus according to another embodiment of the present invention is an image. FIG. 11B is an enlarged view of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Polarization imaging apparatus 101 Polarizer array 102 Light receiving element array 103 Light receiving module 104 Image processing part 105 Operation part 106 Main memory 107 Output part 201 The transparent material board | substrate 202 which formed the periodic groove | channel row | line | column 202 High refractive index medium 203 Low refractive index Medium 401 Polarizer array 402 Light receiving element arrays 403 to 406 Polarizer region 407 Polarizer unit

Claims (19)

It is divided into three or more polarizer regions each having a different transmission axis, and transmits the non-polarized component of the input light in each region of the input light that is incident, and the polarization direction varies depending on each region. A polarizer array including one or a plurality of polarizer units that transmit the polarization component of the input light;
A light receiving element array that independently receives light transmitted through each of the regions;
An image processing unit for processing the polarization component and the non-polarization component from the light receiving element array;
Have
The image processing unit applies a mathematical model represented by the following formula (1) to the intensity fm (i, j) of transmitted light transmitted through each region with respect to the angle of the transmission axis in each region,
2. The polarization imaging apparatus according to claim 1, wherein the intensity of the transmitted light is separated into an intensity A (i, j) relating to the polarization component and an intensity B (i, j) relating to the non-polarization component.
Here, m is a number assigned to each region, i and j are coordinates of the polarizer unit in the polarizer array, and θm is 0 ° of the transmission axis in the reference region among the regions. Is the angle of the transmission axis in each other region, and θ (i, j) is the angle between the polarization direction of the polarization component input to the polarizer unit and the transmission axis in the reference region It is a difference.
It is divided into three or more polarizer regions each having a different transmission axis, and transmits the non-polarized component of the input light in each region of the input light that is incident, and the polarization direction varies depending on each region. A polarizer array including one or a plurality of polarizer units that transmit the polarization component of the input light;
A light receiving element array that independently receives light transmitted through each of the regions;
An image processing unit for processing the polarization component and the non-polarization component from the light receiving element array;
Have
The image processing unit calculates an average value by adding all the intensities fm (i, j) of transmitted light transmitted through the respective regions and dividing the sum by the number of regions, and for the angle of the transmission axis in each region, By applying a mathematical model represented by the following formula (2) to the intensity obtained by subtracting the average value from the intensity fm (i, j) of the transmitted light transmitted through each region,
A polarization imaging apparatus that extracts an intensity A (i, j) related to the polarization component from the intensity of the transmitted light.
Here, m is a number assigned to each region, i and j are coordinates of the polarizer unit in the polarizer array, and θm is 0 ° of the transmission axis in the reference region among the regions. Is the angle of the transmission axis in each other region, and θ (i, j) is the angle between the polarization direction of the polarization component input to the polarizer unit and the transmission axis in the reference region It is a difference. Further, the average value of the intensity fm (i, j) is displayed with a symbol with f (i, j) overlined.
The polarization imaging apparatus according to claim 1 , wherein the image processing unit further calculates θ (i, j) to obtain a polarization direction of the polarization component.
It is divided into three or more polarizer regions each having a different transmission axis, and transmits the non-polarized component of the input light in each region of the input light that is incident, and the polarization direction varies depending on each region. A polarizer array including one or a plurality of polarizer units that transmit the polarization component of the input light;
A light receiving element array that independently receives light transmitted through each of the regions;
With respect to the intensity of transmitted light transmitted through each of the regions, an image processing unit that separates a polarizer unit having a difference in transmitted light intensity between adjacent regions larger than a predetermined value and a polarizer unit smaller than a predetermined value;
A polarization imaging apparatus.
5. The polarization imaging apparatus according to claim 1 , wherein the transmission axis in the polarizer unit differs by an angle of 45 ° or less for each region.
The polarizer unit has four regions,
5. The polarization imaging apparatus according to claim 1 , wherein the transmission axis in the region is a direction of 0 °, 45 °, 90 °, and 135 ° with respect to a transmission axis in a reference region among the regions. .
A light shielding portion is provided at a boundary portion of each region of the polarizer unit, or a region of the light receiving element array corresponding to a boundary portion of each region of the polarizer unit is shielded, and light is diffracted or scattered at the boundary portion. The polarization imaging apparatus according to claim 1 or 4 , which suppresses the influence of.
The polarizer unit is a multilayer structure in which two or more transparent materials are alternately stacked in the z direction on a single substrate parallel to the xy plane in an orthogonal coordinate system x, y, z. It is divided into three or more polarizer regions, and each layer has a one-dimensional periodic uneven shape repeated in one direction in the xy plane determined for each region,
The input light is incident on the xy plane;
The polarization imaging apparatus according to claim 1 .
In the polarizer unit, each region is configured by a wire grid polarizer.
The polarization imaging apparatus according to claim 1 .
The polarization imaging apparatus according to claim 1 , wherein the light receiving element array is one of a photodetector, a CCD, a C-MOS, and an imaging tube.
Comprising the polarization imaging apparatus according to claim 1;
An imaging apparatus that obtains the non-polarized component by separating the polarized component and the non-polarized component constituting the input light by the image processing unit.
Comprising the polarization imaging apparatus according to claim 3 ;
The surface shape measuring device further detects the surface shape of the object by mapping the polarization direction angle of the polarization component for each polarizer unit by the image processing unit.
Comprising the polarization imaging apparatus according to claim 3 ;
A surface property measurement apparatus that further detects the continuity of a substance constituting the surface of an object by mapping an angle of a polarization direction of the polarization component for each polarizer unit by the image processing unit.
Comprising the polarization imaging apparatus according to claim 3 ;
The image processing unit further maps the angle of the polarization direction of the polarization component for each polarizer unit, and in the map, the predetermined polarization angle is uniform over a predetermined range or continuously. An imaging device that detects changes.
A photographing apparatus according to claim 14 is provided,
The image processing unit further specifies a location where a predetermined polarization angle is uniform over a predetermined range in the map or a location where the polarization angle changes continuously as a road surface, and is located at a location specified as the road surface. An obstacle detection device for detecting an object on the road surface.
A photographing apparatus according to claim 14 is provided,
The image processing unit further specifies a place where a predetermined polarization angle is uniform over a predetermined range in the map or a place where it continuously changes as a road surface,
Further, in the portion corresponding to the location specified as the road surface in the map by the image processing unit, a location where the intensity relating to the polarization component is greater than a predetermined value or a location where the intensity relating to the non-polarization component is less than a predetermined value is liquid. A road surface state detection device that determines a location where a component exists and determines a location where the intensity related to the polarized component is smaller than a predetermined value or a location where the intensity related to the non-polarized component is higher than a predetermined value as a location where a solid component exists.
Comprising the polarization imaging apparatus according to claim 1 ;
The image processing unit separates the polarization component and the non-polarization component constituting the input light, and maps the intensity related to the polarization component or the intensity related to the non-polarization component for each polarizer unit. .
A photographing apparatus according to claim 17 is provided,
The image processing unit further determines, in the map, a location where the intensity relating to the polarization component is greater than a predetermined value or a location where the intensity relating to the non-polarization component is less than a predetermined value as a location where a liquid component exists, and the polarization An imaging device for weather observation that determines a location where the intensity related to a component is smaller than a predetermined value or a location where the intensity related to the non-polarized component is higher than a predetermined value as a location where a solid component exists.
Comprising the polarization imaging apparatus according to claim 4 ;
The image processing unit further detects, as a moving body, information from a location where a polarizer unit having a transmitted light intensity difference between adjacent regions larger than a predetermined value is adjacent and continuous over a predetermined range. Moving body detection imaging device.