KR102656562B1 - Hyperspectral sensor, hyperspectral imaging system including the sensor, and hyperspectral imaging method using the system - Google Patents

Abstract

The present invention relates to a window, a first focusing unit provided on the rear of the window and including a plurality of lenses, and a first image sensor provided on the rear of the first focusing unit and having a front side parallel to the rear of the window. , a first mirror spaced apart from the first focusing unit and the first image sensor and having a front surface inclined with respect to the rear surface of the window, a first optical element spaced apart from the first mirror, the first optical element, and A second optical element spaced apart and having an internal periodic refractive index distribution, a second focusing unit spaced apart from the second optical element and including a plurality of lenses, and a second image provided on the rear surface of the second focusing unit. A hyperspectral sensor including a sensor, a hyperspectral imaging system including the same, and a hyperspectral imaging method using the same are provided.

Description

Hyperspectral sensor, hyperspectral imaging system including the same, and hyperspectral imaging method using the same {Hyperspectral sensor, hyperspectral imaging system including the sensor, and hyperspectral imaging method using the system}

The present invention relates to a hyperspectral sensor, and more specifically, a hyperspectral sensor including a general camera optical system and a hyperspectral optical system, a hyperspectral imaging system including the same, and a hyperspectral imaging method for determining the spectrum of an analysis target using the same. It's about.

Hyperspectral sensors that can determine the spectral characteristics of the object in addition to its spatial distribution are being used to detect hazardous substances, the growth status of agricultural crops, and green algae blooms in lakes in a wide area. These hyperspectral sensors are mounted on manned aircraft and small unmanned aerial vehicles. It has the advantage of greatly expanding its range of use when mounted on a small unmanned vehicle, so research is underway to realize this.

In particular, when a hyperspectral sensor includes many components with focal lengths, the overall size of the sensor can become large, and the structure becomes complex and manufacturing becomes difficult to compensate for aberrations occurring in each component with focal lengths. There is. Accordingly, many efforts are being made to reduce the size and weight of hyperspectral sensors.

The purpose of the present invention is to provide a miniaturized and simplified hyperspectral sensor, a hyperspectral imaging system including the same, and a hyperspectral imaging method using the same.

The problem to be solved by the present invention is not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below.

A hyperspectral sensor according to an embodiment of the present invention is provided on a window, a rear surface of the window, a first focusing unit including a plurality of lenses, and a rear surface of the first focusing unit, the rear surface of the window and A first image sensor having a parallel front surface, a first mirror spaced apart from the first focusing unit and the first image sensor and a front surface inclined with respect to the rear surface of the window, and a first light spaced apart from the first mirror. element, a second optical element spaced apart from the first optical element and having an internal periodic refractive index distribution, a second focusing unit spaced apart from the second optical element and including a plurality of lenses, and the second focusing unit. It may include a second image sensor provided on the rear side.

The first optical device may have an internal periodic refractive index distribution, and the thickness of the first optical device may be smaller than the thickness of the second optical device.

The first optical element may be one or a combination of two or more selected from a surface diffraction grating, a volume Bragg grating, and at least one prism.

The first mirror may be connected to a driving unit, and the first mirror may be configured to rotate by the driving unit.

Light diffracted by the second optical element may be configured to travel parallel to the rear surface of the window.

The front surface of the first mirror may be a curved surface.

It further includes a third optical element between the first mirror and the first optical element, wherein the third optical element may be a close-up lens or an achromatic lens.

It further includes a second mirror that is spaced apart from the second optical element and the second focusing unit and has a front surface inclined with respect to the rear surface of the window, and the second image sensor has a front surface parallel to the rear surface of the window. You can have it.

The second image sensor may extend in a direction perpendicular to the rear surface of the window.

It further includes a first shade adjacent to a front surface of the first mirror, and a second shade adjacent to a rear surface of the second optical element, wherein each of the first shade and the second shade diffracts in the second optical element. It may be configured to remove 0th order diffracted light components that are not present.

A hyperspectral imaging system according to an embodiment of the present invention includes a window, a first optical system configured to allow first incident light to enter, and a second optical system spaced apart from the first optical system and configured to allow second incident light to enter. It may include a sensor, an optical image stabilization (OIS) module configured to detect and control the movement of the hyperspectral sensor, and a processor connected to the first optical system and the second optical system of the hyperspectral sensor, and the OIS module. You can. At this time, the first optical system is provided on the back of the window, a first focusing part configured to focus the first incident light, and provided on the back of the first focusing part, and the light passing through the first focusing part It may include a first image sensor configured to detect first incident light, and the second optical system has a front surface inclined with respect to the rear surface of the window, and a first mirror configured to reflect the second incident light, First and second optical elements spaced apart from the first mirror and configured to diffract the second incident light, spaced apart from the second optical element and configured to focus the diffracted light diffracted by the second optical element. It may include a second focusing unit, and a second image sensor provided on the back of the second focusing unit and configured to detect the diffracted light that has passed through the second focusing unit.

The first optical element is one or a combination of two or more selected from a surface diffraction grating, a volume Bragg grating, and at least one prism, and the second optical element may be a volume Bragg grating. .

Each of the first and second optical devices is a volume Bragg grating, and the grating period of the first optical device may be different from that of the second optical device.

The OIS module includes position sensors and OIS drivers connected to the hyperspectral sensor, and an OIS control unit connected to the position sensors and the OIS drivers, respectively, wherein the OIS control unit receives location information of the position sensors. It can be configured to receive and transmit it to the OIS driving units.

A scan mirror control module connected to the first mirror of the second optical system, a focusing unit control module connected to the first focusing unit of the first optical system and the second focusing unit of the second optical system, and the processor It may further include a light source control module connected to the processor, wherein the light source control module may include an LED driver connected to the processor, and an LED light source connected to the LED driver whose spectrum is known in advance.

A hyperspectral imaging method according to an embodiment of the present invention includes setting a measurement area, photographing a reference image by the first optical system, photographing hyperspectral images by the second optical system, and the hyperspectral image. It may include synthesizing the hyperspectral images, comparing the composite image obtained by synthesizing the hyperspectral images with the reference image, and outputting the measurement results. At this time, the first optical system is provided on the back of the window, a first focusing part configured to focus the first incident light, and provided on the back of the first focusing part, and the first focusing part is provided on the back of the first focusing part, and the first focusing part is configured to focus the first incident light. 1 comprising a first image sensor configured to detect incident light, a second optical system having a front surface inclined with respect to the rear surface of the window, a first mirror configured to reflect the second incident light, the first mirror An angle converter for each wavelength that converts the second incident light to proceed at an angle that satisfies the Bragg condition for each wavelength, and an angle converter for each wavelength, which is spaced apart from the angle converter for each wavelength and diffracted light diffracted by the angle converter for each wavelength. It may include a second focusing unit configured to focus, and a second image sensor provided on the rear surface of the second focusing unit and configured to detect the diffracted light that has passed through the second focusing unit.

Photographing the hyperspectral images by the second optical system includes a plurality of photographing operations, each of which includes rotating the first mirror and photographing the hyperspectral images by the second image sensor. It may include correcting distortion of the hyperspectral image.

Capturing the hyperspectral image by the second image sensor may be performed once when the light source is turned on and once when the light source is turned off.

Before setting the measurement area, it further includes entering a measurement mode, wherein the measurement modes include a point spectrum measurement mode, a spectrum measurement mode after shape recognition, a spatial low resolution measurement mode, and a spatial high resolution measurement mode. It may be any one selected from (high resolution) measurement modes.

The angle converter for each wavelength may be any one or a combination of two or more selected from a surface diffraction grating, a volume Bragg grating, and at least one prism.

The hyperspectral sensor according to an embodiment of the present invention can filter incident light through a volume Bragg grating, so the sensor can be miniaturized, simplified, and easy to manufacture and align.

In addition, the hyperspectral imaging system according to an embodiment of the present invention and the hyperspectral imaging method using the same can minimize the scan area, solving the problem that it takes a long time to scan the entire area when the exposure time is long.

Additionally, the reliability of the hyperspectral imaging method according to an embodiment of the present invention can be improved by comparing the reference image captured by the first image sensor with the hyperspectral image captured by the second image sensor.

1 is a conceptual diagram for explaining the structure of a hyperspectral sensor according to embodiments of the present invention.
FIG. 2 is an enlarged view for explaining the first optical element and the second optical element of the hyperspectral sensor according to FIG. 1.
FIGS. 3A, 3B, 3C, and 3D are conceptual diagrams showing the path of light incident on a hyperspectral sensor according to embodiments of the present invention. FIGS. 3A, 3B, and 3C are respectively the light in the first cross section. Figure 3D shows the optical path in the second cross section.
Figures 4, 5, and 6 are conceptual diagrams for explaining the structure of a hyperspectral sensor according to other embodiments of the present invention.
FIGS. 7, 8A, 8B, 8C, and 8D are simulation results for explaining images measured by a hyperspectral sensor according to embodiments of the present invention.
Figure 9 is a conceptual diagram for explaining a hyperspectral imaging system including a hyperspectral sensor according to embodiments of the present invention.
Figure 10 is a flowchart illustrating a hyperspectral imaging method using a hyperspectral imaging system according to embodiments of the present invention.
Figure 11 is a timing diagram for explaining a hyperspectral imaging method using a hyperspectral imaging system according to embodiments of the present invention.

In order to fully understand the configuration and effects of the present invention, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

The present invention is not limited to the embodiments disclosed below, but can be implemented in various forms and various modifications and changes can be made. However, the description of this embodiment is provided to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the present invention of the scope of the invention. In the attached drawings, the components are shown enlarged in size for convenience of explanation, and the proportions of each component may be exaggerated or reduced.

The terms used in this specification are for describing embodiments and are not intended to limit the invention. Additionally, unless otherwise defined, the terms used in this specification may be interpreted as meanings commonly known to those skilled in the art.

As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, 'comprises' and/or 'comprising' refers to the presence of one or more other components, steps, operations and/or elements. or does not rule out addition.

When a layer is referred to herein as being 'on' another layer, it may be formed directly on top of the other layer, or there may be a third layer interposed between them.

Although terms such as first and second are used in this specification to describe various regions, layers, etc., these regions and layers should not be limited by these terms. These terms are merely used to distinguish one area or layer from another area or layer. Accordingly, a part referred to as a first part in one embodiment may be referred to as a second part in another embodiment. Embodiments described and illustrated herein also include complementary embodiments thereof. Parts indicated with the same reference numerals throughout the specification represent the same elements.

Hereinafter, a hyperspectral sensor according to embodiments of the present invention, a hyperspectral imaging system including the same, and a hyperspectral imaging method using the same will be described in detail with reference to the drawings.

1 is a conceptual diagram for explaining the structure of a hyperspectral sensor according to embodiments of the present invention. FIG. 2 is an enlarged view for explaining the first optical element and the second optical element of the hyperspectral sensor according to FIG. 1.

Referring to FIG. 1, the hyperspectral sensor according to the present invention may include a window (W), a first optical system (P1), and a second optical system (P2). The first optical system (P1) and the second optical system (P2) may be spaced apart from each other. The first optical system P1 may include a first focusing unit F1 and a first image sensor S1. The second optical system (P2) may include a first mirror (M1), a first optical element (OC1), a second optical element (OC2), a second focusing unit (F2), and a second image sensor (S2). . The first incident light IL1 may be incident towards the first optical system P1, and the second incident light IL2 may be incident towards the second optical system P2. The first optical system (P1) may be the same as the optical system of a general camera, and the second optical system (P2) may be a hyperspectral optics. Below, each of the configurations of the first and second optical systems P1 and P2 will be described in detail.

The window W may have a planar shape that is perpendicular to the first direction D1 and extends in the second direction D2 and the third direction D3. The first to third directions D1, D2, and D3 may be orthogonal to each other. The first and second incident lights IL1 and IL2 may be incident parallel to the first direction D1 toward the window W, but this is only an example and the present invention is not limited thereto. 2 Incident lights IL1 and IL2 may be incident from different directions intersecting the front surface Wf of the window W. In other words, the front surface (Wf) of the window (W) can view the analysis object and can receive the first and second incident lights (IL1, IL2) emitted from the analysis object. The window W may protect the first and second optical systems P1 and P2 on the rear surface Wb from external shock and/or contamination.

In this specification, the front is defined as the side that faces the light incident toward each component, and the back is defined as the side that emits the light that has passed through each component. For example, the front side may be a side facing the opposite direction of the first direction D1, and the back side may be a side facing the first direction D1.

A first focusing part F1 may be provided on the rear surface Wb of the window W. The first focusing part F1 may be provided between the window W and the first image sensor S1. The first focusing unit F1 may be configured to focus the first incident light IL1 that has passed through the window W. The first focusing unit F1 may include a plurality of lenses. The central axis of each of the plurality of lenses of the first focusing unit F1 may coincide with the central axis of the first image sensor S1. For example, the central axis of each of the plurality of lenses of the first focusing unit F1 may be parallel to the first direction D1. The plurality of lenses of the first focusing unit F1 are not limited to the illustrated shape and the illustrated side curvature and may have various shapes and curvatures.

A first image sensor (S1) may be provided on the back of the first focusing unit (F1). The first image sensor S1 may have a front surface S1f parallel to the rear Wb of the window W. The first image sensor S1 may be configured to detect the first incident light IL1 that has passed through the window W and the first focusing unit F1. The first image sensor S1 may be arranged so that its front surface S1f coincides with the focal plane of the first incident light IL1.

The first mirror M1 may be provided at a position spaced apart from the first focusing unit F1 and the first image sensor S1 in the third direction D3. The first mirror M1 may be spaced apart from the rear side Wb of the window W in the first direction D1. The first mirror M1 may have a front surface M1f that is inclined with respect to the rear surface Wb of the window W (that is, with respect to the third direction D3). The angle formed between the front surface (M1f) of the first mirror (M1) and the rear surface (Wb) of the window (W) may be greater than 0 degrees and less than 90 degrees. The central axis of the first mirror M1 may be connected to the driving unit. The first mirror M1 may be configured to rotate clockwise or counterclockwise by the driving unit. The first mirror M1 may be configured to reflect the second incident light IL2 traveling along a shifted path in the third direction D3 from the first incident light IL1.

The first and second optical elements OC1 and OC2 may be provided at positions spaced apart from the first mirror M1 in the third direction D3. The first optical device OC1 may be provided between the first mirror M1 and the second optical device OC2 in the third direction D3, and the second optical device OC2 may be provided in the third direction D3. It may be provided between the first optical element OC1 and the second focusing part F2. The first and second optical elements OC1 and OC2 may be spaced apart from the rear surface Wb of the window W in the first direction D1. The first and second optical devices OC1 and OC2 may overlap each other in the first direction D1, but this is only an example and the present invention is not limited thereto.

The first optical device OC1 may convert the second incident light IL2 to travel at an angle that satisfies the Bragg condition for each wavelength. In other words, the degree to which the first optical device OC1 changes the propagation angle of the second incident light IL2 may vary depending on the wavelength. In this specification, the first optical element OC1 may be referred to as an angle converter for each wavelength.

For example, the first optical device OC1 may be a surface diffraction grating including periodic protrusions on the surface. As another example, the first optical element OC1 may be a volume Bragg grating with an internal periodic refractive index distribution. Volume Bragg grating may be referred to as bulk Bragg grating or volume holographic grating. As another example, the first optical device OC1 may be a prism.

The first optical element OC1 may be, for example, one selected from a surface diffraction grating, a volume Bragg grating, and at least one prism, or a combination of two or more selected from a surface diffraction grating, a volume Bragg grating, and at least one prism. . When the first optical element OC1 includes a diffraction grating and at least one prism, at least one prism may be provided between the diffraction grating and the second optical element OC2, which will be described later.

For example, the second optical device OC2 may be a volume Bragg grating with an internal periodic refractive index distribution. The second optical device OC2 may be spaced apart from the first optical device OC1.

When each of the first and second optical elements OC1 and OC2 is a volume Bragg grating, the thickness of the second optical element OC2 may be different from the thickness of the first optical element OC1. For example, the thickness of the second optical device OC2 may be greater than the thickness of the first optical device OC1. Each of the first and second optical elements OC1 and OC2 may include a support portion and a grating portion (i.e., an internal portion whose refractive index changes periodically), and the first and second optical elements OC1 and OC2 Each thickness refers to the thickness of the grid portion only, excluding the thickness of the support portion. In a volume Bragg grating, as the refractive index difference (i.e., the refractive index difference between the high refractive index portion and the low refractive index portion) decreases and the thickness of the grating increases, angular selectivity may increase.

When each of the first and second optical elements OC1 and OC2 is a volume Bragg grating, the grating period (Λ ₁ ) of the first optical element (OC1) is the grating period (Λ ₂ ) of the second optical element (OC2) It may be different from For example, the lattice period Λ ₁ of the first optical device OC1 may be greater than the lattice period Λ ₂ of the second optical device OC2. The grating period (Λ ₁ ) of the first optical element (OC1) and the grating period (Λ ₂ ) of the second optical element (OC2) may satisfy the following [Equation 1], and accordingly, the The wavelength dependent aiming angle error with respect to the reference wavelength (λ) can be minimized. The aiming angle error by wavelength means the angle that deviates from the reference angle when light that has a reference wavelength and is incident at a reference angle satisfies the Bragg condition, and light that is incident at an angle that deviates from the reference angle for each wavelength satisfies the Bragg condition.

[Equation 1]

For example, in order to satisfy the above [Equation 1], the reference wavelength (λ) is about 620 nm, and the lattice period (Λ ₂ ) of the second optical element (OC2) is about 0.833 μm (i.e., the lattice density is about 0.833 μm). In the case of 1200 lines/mm), the grating period (Λ ₁ ) of the first optical element (OC1) is about 1.578 μm (that is, the grating density is about 633.8 lines/mm). When the wavelength is about 400 nm, the aiming angle error per wavelength is about 0.11 degrees, and when the wavelength is about 800 nm, the aiming angle error per wavelength is about 0.08 degrees.

For example, a first light having a wavelength of about 620 nm and incident at a reference angle and a second light having a wavelength of about 550 nm and incident at an angle deviating from the reference angle are connected to the first optical element OC1 and the second light. When diffracted by the element OC2, the first and second lights may be detected at the same location on the front surface S2f of the second image sensor S2 even though they have different wavelengths. When the second optical device OC2 has high angular selectivity, the first light satisfies the Bragg condition and is diffracted in the second optical device OC2, but the second light does not satisfy the Bragg condition and is diffracted in the second optical device OC2. It passes straight through the element OC2 without being diffracted and is not detected on the front surface S2f of the second image sensor S2. Accordingly, the second optical system P2 can operate as a hyperspectral optics.

When the first optical element OC1 is a surface diffraction grating and the second optical element OC2 is a volume Bragg grating, the second optical element OC2 may be provided in parallel with the first optical element OC1. When the first and second optical elements OC1 and OC2 are provided in parallel with each other, the lattice period Λ ₂ of the second optical element OC2 is the lattice period Λ ₁ of the first optical element OC1. It may be twice that of

Referring to FIG. 2, the first optical element OC1 may be configured to diffract the second incident light IL2 into the first diffracted light DL1, and the second optical element OC2 may be configured to diffract the second incident light IL2 into the first diffracted light DL1. It may be configured to diffract DL1) into the second diffracted light DL2. At this time, each of the first and second optical elements OC1 and OC2 may be a diffraction grating having a grating period, and in particular, the second optical element OC2 may be a volume Bragg grating.

Specifically, the first optical element OC1 may diffract the second incident light IL2 into the first diffracted light DL1 according to the grating equation expressed as [Equation 2] below.

[Equation 2]

At this time, m is the diffraction order, λ is the wavelength of the second incident light IL2, Λ ₁ is the grating period of the first optical element OC1, and α is the It is the angle formed by the perpendicular line (OC1p) of the surface of the optical element (OC1), and β is the angle formed by the first diffracted light (DL1) and the perpendicular line (OC1p) of the surface of the first optical element (OC1). am. When α and β are equal, the Bragg condition is satisfied and the diffraction efficiency of the first optical element OC1 is maximized.

In addition, the second optical element OC2 satisfies the Bragg condition and converts the first diffracted light DL1 into the second diffracted light DL2 according to the lattice equation expressed in Equation 3 below. It can be diffracted.

[Equation 3]

At this time, Λ ₂ is the grating period of the second optical element OC2, and θ is the angle formed between the first diffracted light DL1 and the perpendicular line OC2p of the surface of the second optical element OC2.

The second optical device OC2 may be configured so that the second diffracted light DL2 travels parallel to the third direction D3. More specifically, the angle formed by the perpendicular line OC2p of the surface of the second optical element OC2 with the third direction D3 is the perpendicular line OC2p between the first diffracted light DL1 and the surface of the second optical element OC2. ) may be substantially the same as the angle (θ) formed by

The second optical device OC2 may have a surface inclined with respect to the surface of the first optical device OC1. The angle ϕ at which the surface of the second optical device OC2 is inclined with respect to the surface of the first optical device OC1 is θ-β. For example, the reference wavelength (λ) is about 620 nm, the grating period (Λ ₂ ) of the second optical device (OC2) is about 0.833 μm (i.e., the grating density is about 1200 lines/mm), and the first optical device (OC2) When the lattice period (Λ ₁ ) of (OC1) is about 1.578 μm (i.e., the lattice density is about 633.8 lines/mm), θ is about 21.839 degrees, β is about 11.331 degrees, and ϕ is about 10.508 degrees.

Referring again to FIG. 1, the hyperspectral sensor according to the present invention further includes a third optical element (OC3) provided between the first mirror (M1) and the first optical element (OC1) in the third direction (D3). can do. The third optical device OC3 may be configured to collimate the second incident light IL2 reflected from the first mirror M1 and heading toward the first optical device OC1. The third optical element OC3 may be a close-up lens or an achromatic lens that reduces aberration depending on the wavelength.

When the third optical element OC3 is an achromatic lens, the third optical element OC3 may include a first achromatic lens and a second achromatic lens that are bonded to each other and have different optical characteristics and/or different structures. . Each of the first and second achromatic lenses may include glass or plastic. The N _d /V _d value (i.e., (d-line refractive index)/(Abbe number)) of the first achromatic lens may be different from the N _d /V _d value of the second achromatic lens. For example, the N _d /V _d value of the first achromatic lens is about 1.805/25.36, and the N _d /V _d value of the second achromatic lens is about 1.651/55.89. The front of the first achromatic lens, the back of the first achromatic lens (i.e., the front of the second achromatic lens), and the rear of the second achromatic lens may have different radii of curvature. As an example, the radius of curvature of the front side of the first achromatic lens is infinite (i.e., the front side of the first achromatic lens is flat), the radius of curvature of the back side of the first achromatic lens is about -285.815 mm, and the radius of curvature of the back side of the first achromatic lens is about -285.815 mm, and the radius of curvature of the back side of the first achromatic lens is about -285.815 mm, and the radius of curvature of the back side of the first achromatic lens is about -285.815 mm. The radius of curvature of the rear surface is approximately 167.715 mm. The thickness of the first achromatic lens may be substantially the same as the thickness of the second achromatic lens, but this is only an example and the present invention is not limited thereto. For example, the thickness of each of the first and second achromatic lenses may be about 0.1 mm to about 1 mm.

The second focusing part F2 may be provided at a position spaced apart from the second optical element OC2 in the third direction D3. The second focusing part F2 may be spaced apart from the rear surface Wb of the window W in the first direction D1. For example, the distance in the first direction D1 between the front surface of the second focusing part F2 and the back surface Wb of the window W is, for example, the distance between the front surface of the first focusing part F1 and the window Wb. It may be greater than the distance between the rear surfaces Wb in the first direction D1. The second focusing part F2 may be configured to focus the second diffracted light DL2 diffracted by the second optical element OC2. The second focusing unit F2 may include a plurality of lenses. The central axis of each of the plurality of lenses of the second focusing unit F2 may coincide with the central axis of the second image sensor S2. For example, the central axis of each of the plurality of lenses of the second focusing unit F2 may be parallel to the first direction D1. The plurality of lenses of the second focusing unit F2 are not limited to the illustrated shape and the illustrated side curvature and may have various shapes and curvatures.

A second image sensor S2 may be provided on the rear of the second focusing unit F2. The second image sensor S2 may have a front surface S2f parallel to the rear surface Wb of the window W. The second image sensor S2 may be configured to detect the second diffracted light DL2 that has passed through the second focusing unit F2. The second image sensor S2 may be arranged so that its front surface S2f coincides with the focal plane of the second diffracted light DL2.

The hyperspectral sensor according to the present invention includes a second mirror (M2) spaced apart from the second optical element (OC2) in the third direction (D3) and spaced apart from the second focusing unit (F2) in the first direction (D1). More may be included. The second mirror M2 may have a front surface M2f that is inclined with respect to the rear surface Wb of the window W (that is, with respect to the third direction D3). The angle formed between the front surface (M2f) of the second mirror (M2) and the rear surface (Wb) of the window (W) may be, for example, about 45 degrees, and the angle may be fixed. The second mirror M2 may be configured to reflect the second diffracted light DL2 and advance it to the second focusing unit F2.

The hyperspectral sensor according to the present invention may further include an aperture stop (AS) overlapping with one of the plurality of lenses of the second focusing unit (F2). For example, the aperture stop AS may overlap with the one closest to the second optical element OC2 among the plurality of lenses of the second focusing unit F2.

The length L ₃ of the second optical system P2 in the third direction D3 can be expressed as [Equation 4] below. The length L ₃ of the second optical system P2 in the third direction D3 may be, for example, about 10 mm to about 15 mm.

[Equation 4]

L ₃ =L _a +L _b +L _c

At this time, L _a is the distance in the third direction D3 between the center of the front surface M1f of the first mirror M1 and the center of the first optical element OC1, and L _b is the distance between the center of the front surface M1f of the first mirror M1 and the center of the first optical element OC1. ) is the distance in the third direction (D3) between the center of the second optical element (OC2) and the center of the second optical element (OC2), and L _c is the center of the second optical element (OC2) and the front surface (M2f) of the second mirror (M2) It is the distance in the third direction (D3) between the centers of .

The length L ₁ of the second optical system P2 in the first direction D1 can be expressed as [Equation 5] below. The length L ₁ of the second optical system P2 in the first direction D1 may be, for example, about 5 mm to about 15 mm.

[Equation 5]

L ₁ =L ₀ -L _g +L _m +L _f

At this time, L ₀ is the distance in the first direction (D1) between the rear (Wb) of the window (W) and the center of the front (M1f) of the first mirror (M1), and L _g is the distance between the center of the front (M1f) of the first mirror (M1) is the distance in the first direction D1 between the center of the front surface M1f of the second mirror M2 and the center of the front surface M2f of the second mirror M2, and L _m is the center of the front surface M2f of the second mirror M2. is the distance in the first direction (D1) between the front of the second focusing unit (F2), and L _f is the first direction between the front of the second focusing unit (F2) and the back of the second image sensor (S2) It is the distance to (D1).

The distance L _g in the first direction D1 between the center of the front surface M1f of the first mirror M1 and the center of the front surface M2f of the second mirror M2 is given by Equation 6 below: can be expressed. The distance L _g in the first direction D1 between the center of the front surface M1f of the first mirror M1 and the center of the front surface M2f of the second mirror M2 is, for example, about 2. mm to about 6 mm.

[Equation 6]

L _g =L _a· tan2ϕ+L _b· tan2θ

At this time, 2ϕ is the angle at which the path of the second incident light IL2 reflected from the first mirror M1 is inclined with respect to the third direction D3, and 2θ is the angle at which the path of the first diffracted light DL1 is inclined in the third direction. It is an angle inclined with respect to (D3).

Meanwhile, the length of the second optical system P2 in the second direction D2 may be equal to the length of the first mirror M1 in the second direction D2. The length of the second optical system P2 in the second direction D2 may be determined by the viewing angle in the second direction D2. The length of the second optical system P2 in the second direction D2 may be, for example, about 5 mm to about 15 mm.

FIGS. 3A, 3B, 3C, and 3D are conceptual diagrams showing the path of light incident on a hyperspectral sensor according to embodiments of the present invention. FIGS. 3A, 3B, and 3C are respectively the light in the first cross section. Figure 3D shows the optical path in the second cross section.

3A, 3B, and 3C show first incident light IL1 emitted from sources that have the same position in the first direction D1 and the second direction D2, but have different positions in the third direction D3. ) and the paths of the second incident light IL2. In the first optical system P1, first incident light IL1 incident from different directions may form images at different positions of the first image sensor S1 depending on the incident direction regardless of wavelength. On the other hand, in the second optical system P2, the second incident light IL2 incident from one direction may include first to third lights IL21, IL22, and IL23 having different wavelengths, and the first to third lights IL21, IL22, and IL23 may have different wavelengths. The third lights IL21, IL22, and IL23 may form images at different positions of the second image sensor S2 depending on the wavelength. For example, the first light IL21 may have a shorter wavelength than the second light IL22, and the second light IL22 may have a shorter wavelength than the third light IL23.

The tilt angle of the first mirror M1 in FIGS. 3A, 3B, and 3C may be different. The first mirror M1 may scan the entire area of the source extending in the third direction D3 by rotating counterclockwise (or clockwise).

3D shows first incident light IL1 and second incident light (IL1) emitted from sources that have the same positions in the first direction D1 and the third direction D3, but have different positions in the second direction D2. IL2) paths are shown. In the first optical system P1, first incident light IL1 incident from different directions may form images at different positions of the first image sensor S1 depending on the incident direction regardless of wavelength. In the second optical system P2, the second incident light IL2 incident from different directions may form images at different positions of the second image sensor S2 depending on the incident direction regardless of the wavelength.

Figures 4, 5, and 6 are conceptual diagrams for explaining the structure of a hyperspectral sensor according to other embodiments of the present invention. Hereinafter, for convenience of explanation, description of matters substantially the same as those described with reference to FIGS. 1 and 2 will be omitted and differences will be described in detail.

Referring to FIG. 4 , the third optical device OC3 described with reference to FIG. 1 may be omitted. That is, the second incident light IL2 reflected from the front surface M1f of the first mirror M1 may proceed to the first and second optical elements OC1 and OC2 without passing through a separate collimating optical system. The front surface M1f of the first mirror M1 may be a curved surface, and the rear surface of the first mirror M1 may be flat. The focal length of the first mirror M1 may be, for example, about 200 mm to 400 mm.

According to embodiments, the first mirror M1 may be a reflective grid with a periodic grid engraved on the front surface M1f. According to embodiments, the front surface M1f of the first mirror M1 may be flat, and the rear surface of the first mirror M1 may be a curved surface with a curvature. The first mirror (M1) may be configured to rotate clockwise or counterclockwise by the driving unit, and the surface on which the second incident light (IL2) is reflected is from the front (M1f) to the back (or from the back to the front (M1f)). ) may change.

Referring to FIG. 5, the hyperspectral sensor according to the present invention may include either a first shade (Sh1) or a second shade (Sh2).

The first shade Sh1 may be provided on the front surface M1f of the first mirror M1. For example, the first shade Sh1 may be configured to rotate together with the first mirror M1. The second shade (Sh2) may be provided between the second optical element (OC2) and the second mirror (M2). The second shade Sh2 may be provided on the back side of the second optical element OC2 or on the front side of the second mirror M2. Each of the first shade Sh1 and the second shade Sh2 may block a portion of the second incident light IL2 and remove the 0th order diffracted light component that is not diffracted in the second optical element OC2. For example, each of the first shade Sh1 and the second shade Sh2 may absorb or scatter a portion of the second incident light IL2. Accordingly, the image detected by the second image sensor S2 may become clearer.

Referring to FIG. 6, the second mirror M2 described with reference to FIG. 1 may be omitted. At this time, the second focusing part F2 and the second image sensor S2 may be arranged along the third direction D3. That is, the second diffracted light DL2 diffracted by the second optical element OC2 may proceed parallel to the third direction D3 and pass through the second focusing unit F2 and the second image sensor S2. ) can be reached. The front surface S2f of the second image sensor S2 may extend in a direction perpendicular to the rear surface Wb of the window W (that is, in the first direction D1 and the second direction D2).

At this time, the length L ₁ of the second optical system P2 in the first direction D1 is the first distance between the rear surface Wb of the window W and the center of the front surface M1f of the first mirror M1. It may be equal to the distance (L ₀ ) in the direction (D1). In other words, the length L ₁ of the second optical system P2 in the first direction D1 may be smaller than that described with reference to FIG. 1 .

FIGS. 7, 8A, 8B, 8C, and 8D are simulation results for explaining images measured by a hyperspectral sensor according to embodiments of the present invention. FIG. 7 is an example image taken by the first image sensor (S1, see FIG. 1), and FIGS. 8A to 8D are example images taken by the second image sensor (S2, see FIG. 1).

FIG. 7 shows an image of a plurality of elliptical shapes located on a plane parallel to the second direction D2 and the third direction D3 and perpendicular to the first direction D1. The plurality of oval shapes may be spaced apart from each other. The plurality of elliptical shapes may have different wavelength spectra.

FIGS. 8A to 8D are spectral images corresponding to the first scan area 10, the second scan area 20, the third scan area 30, and the fourth scan area 40 of FIG. 7. The first mirror M1 of FIG. 1 may be rotated to capture spectral images of each of the first to fourth scan areas 10, 20, 30, and 40. In other words, when taking spectral images of each of the first to fourth scan areas 10, 20, 30, and 40, the tilt angle of the first mirror M1 of FIG. 1 may be different.

Figure 9 is a conceptual diagram for explaining a hyperspectral imaging system including a hyperspectral sensor according to embodiments of the present invention.

Referring to FIG. 9, the hyperspectral imaging system according to the present invention includes the hyperspectral sensor 100, OIS module 200, scan mirror control module 300, focusing unit control module 400, and the hyperspectral sensor 100 described with reference to FIG. 1. It may include a processor 500 and a light source control module 600.

The OIS module 200 includes first and second position sensors (210, 220), first and second OIS actuators (optical image stabilization (OIS) actuators) (230, 240), and an OIS gyroscope. It may include an OIS gyroscope (250) and an OIS controller (OIS controller) (260). The first and second position sensors 210 and 220 may detect movement of the hyperspectral sensor 100 in the second direction (D2, see FIG. 1) and the third direction (D3, see FIG. 1). . The first and second OIS drivers 230 and 240 may suppress the movement of the hyperspectral sensor 100. The OIS control unit 260 may receive location information of the first and second position sensors 210 and 220 and transmit it to the first and second OIS drivers 230 and 240. The OIS gyroscope 250 may be connected to the OIS control unit 260. The OIS control unit 260 may receive rotation information of the OIS gyroscope 250 and transmit it to the first and second OIS drivers 230 and 240. The OIS module 200 can prevent and/or minimize movement of the hyperspectral sensor 100 due to vibration or the like.

The scan mirror control module 300 may include a scan mirror driver 310 and a scan mirror control unit 320. The scan mirror driver 310 may be connected to the first mirror (M1) of the hyperspectral sensor 100 and may rotate the first mirror (M1). The scan mirror control unit 320 may control the rotational movement of the first mirror M1 through the scan mirror driver 310.

The focusing unit control module 400 may be connected to the first and second focusing units F1 and F2 of the hyperspectral sensor 100. The focusing unit control module 400 may control the focal length, etc. of each of the first and second focusing units F1 and F2.

The processor 500 may be connected to the first and second image sensors S1 and S2 of the hyperspectral sensor 100, the OIS module 200, the scan mirror control module 300, and the focusing unit control module 400. and their overall operations can be controlled. The processor 500 may be, for example, a general-purpose processor or an application processor (AP).

The light source control module 600 may be connected to the processor 500. The light source control module 600 may include an LED driver 610 and an LED light source 620. When using an LED light source 620 whose spectrum is known in advance, the light source control module 600 can calculate the reflectance for each wavelength of the analysis target from the measured hyperspectral image. According to embodiments, if the light source control module 600 is large, it may be provided as an independent module outside the hyperspectral imaging system.

Figure 10 is a flowchart illustrating a hyperspectral imaging method using a hyperspectral imaging system according to embodiments of the present invention. Figure 11 is a timing diagram for explaining a hyperspectral imaging method using a hyperspectral imaging system according to embodiments of the present invention.

Referring to FIGS. 9, 10, and 11, the hyperspectral imaging method includes entering a measurement mode (S100), setting a measurement area (S200), taking a reference image (S300), and using a scan mirror. Rotating (S400), taking a hyperspectral image (S500), correcting the distortion of the hyperspectral image (S600), compositing the images (S700), comparing the reference image and the composite image ( S800), and outputting the measurement results (900).

Entering the measurement mode (S100) may include preparing the hyperspectral sensor 100 and determining the execution direction of software that drives the hyperspectral sensor 100 through a measurement mode desired by the user. The measurement mode may be, for example, any one selected from a point spectrum measurement mode, a spectrum measurement mode after shape recognition, a spatial low resolution measurement mode, and a spatial high resolution measurement mode.

The point spectrum measurement mode is a method of measuring only the spectrum of one point among the images of the first image sensor S1. Point spectrum measurement mode has a much faster measurement speed and smaller data amount than other modes.

The spectrum measurement mode after shape recognition is a method of analyzing the boundary of the image of the first image sensor S1 through image processing and then measuring the spectrum inside the boundary area. The spectrum measurement mode after shape recognition is useful for spectrum analysis of objects with relatively clear boundaries, such as grains and pills.

The spatial low-resolution measurement mode is a method that uses a pixel binning method to group pixels of the second image sensor S2 arranged along one direction (second direction D2 in FIG. 1). The spatial low-resolution measurement mode has a larger signal-to-noise ratio and shorter transmission time than other modes, and thus is less affected by motion blur effects and has a shorter measurement time. Meanwhile, the spatial high-resolution measurement mode uses less pixel binning than the spatial low-resolution measurement mode, and has higher spatial resolution than other measurement modes.

Entering the measurement mode (S100) may further include adjusting the focal length of the first focusing unit (F1) through the focusing unit control module 400 of the hyperspectral sensor 100 according to the distance to the analysis object. You can. If the analysis target is relatively close, the focal length of the first focusing unit (F1) may be adjusted to match the focal length of the third optical element (OC3), and if the analysis target is relatively far, the focal length of the first focusing unit (F1) may be adjusted. The distance can be adjusted to approach infinity. Additionally, entering the measurement mode (S100) may further include determining whether to place the third optical element OC3, the type of light source, and whether to use a pixel binning method. Setting the measurement area (S200) may be performed after a predetermined start signal. Setting the measurement area (S200) proceeds as follows. First, when the LED light source 620 is turned on (On), the first image sensor S1 repeatedly captures images and displays them on the display. The hyperspectral sensor 100 is moved so that the image displayed on the display is in focus (that is, the analysis target is located at the focal distance of the third optical element OC3). Afterwards, the measurement area is selected from the image shown on the display. In point spectrum measurement mode, selecting a measurement area means selecting a point on the image shown on the display. In other measurement modes, selecting a measurement area may involve determining the size and location of a rectangle containing some area of the image shown on the display.

The rotation angle interval and number of scans of the first mirror M1 can be calculated according to the set measurement area, and the calculated results can be stored in memory. For example, in the point spectrum measurement mode, the rotation angle of one selected point can be calculated and stored, and in the spectrum measurement mode after shape recognition, the rotation angle of each of the internal points of the plurality of recognized boundary areas can be calculated and stored. there is.

By setting the measurement area (S200) before taking a hyperspectral image (S500), the scan area can be minimized, and if the exposure time is long, it takes a long time to scan the entire area. It can be solved.

Taking a reference image (S300) is done once when the LED light source 620 is turned on (On) and once when the LED light source 620 is turned off (Off) by the first image sensor S1. It can be done. By subtracting the image captured with the LED light source 620 off from the image captured with the LED light source 620 turned on, the influence of ambient light can be removed.

The imaging operation (C), which includes rotating the scan mirror (S400), taking a hyperspectral image (S500), and correcting the distortion of the hyperspectral image (S600), keeps the first mirror (M1) constant. It can be repeated n times while rotating at the interval angle. The number of repetitions (n) of the photographing operation may be calculated by setting the measurement area (S200) and determined using information stored in the memory.

The photographing operation C may include a first photographing operation C1, a second photographing operation C2, a third photographing operation C3 to an n-th photographing operation Cn. Each of the first photographing operation (C1), the second photographing operation (C2), the third photographing operation (C3) to the n-th photographing operation (Cn) rotates the first mirror (M1), and then the LED light source 620 This may be performed by the second image sensor S2 once when the LED light source 620 is turned on (On) and once when the LED light source 620 is turned off (Off).

Comparing the reference image and the composite image (S800) may be to determine whether the normalized cross-correlation (NCC) value of the composite image and the reference image exceeds a predetermined threshold value. If the normalized correlation value of the composite image and the reference image does not exceed the threshold value (i.e., fails), the process may return to setting the measurement area (S200). By comparing the reference image and the synthesized image (S800), the reliability of the hyperspectral imaging method according to the present invention can be improved.

Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. You will understand that it exists. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

W: Windows
F1, F2: Focusing part
S1, S2: Image sensor
M1, M2: Mirror
OC1, OC2, OC3: Optical elements
AS: Aperture aperture
IL1, IL2: Incident light
DL1, DL2: diffracted light
100: Hyperspectral sensor
200: OIS module
300: Scan mirror control module
400: Focusing unit control module
500: Processor
600: Light source control module

Claims (20)

window;
a first focusing unit provided on the back of the window and including a plurality of lenses;
a first image sensor provided on a rear surface of the first focusing unit and having a front surface parallel to the rear surface of the window;
a first mirror spaced apart from the first focusing unit and the first image sensor and having a front surface inclined with respect to the rear surface of the window;
a first optical element spaced apart from the first mirror;
a second optical element spaced apart from the first optical element and having an internal periodic refractive index distribution;
a second focusing unit spaced apart from the second optical element and including a plurality of lenses; and
A hyperspectral sensor including a second image sensor provided on the back of the second focusing unit.
According to claim 1,
The first optical element has an internal periodic refractive index distribution,
A hyperspectral sensor in which the thickness of the first optical element is smaller than the thickness of the second optical element.
According to claim 1,
The first optical element is a hyperspectral sensor selected from a surface diffraction grating, a volume Bragg grating, and at least one prism or a combination of two or more.
According to claim 1,
The first mirror is connected to a driving unit,
The first mirror is a hyperspectral sensor configured to rotate by the driving unit.
According to claim 1,
A hyperspectral sensor configured to allow light diffracted by the second optical element to travel parallel to the rear surface of the window.
According to claim 1,
A hyperspectral sensor where the front surface of the first mirror is a curved surface with curvature.
According to claim 1,
Further comprising a third optical element between the first mirror and the first optical element,
A hyperspectral sensor wherein the third optical element is a close-up lens or an achromatic lens.
According to claim 1,
Further comprising a second mirror spaced apart from the second optical element and the second focusing unit and having a front surface inclined with respect to the rear surface of the window,
The second image sensor is a hyperspectral sensor having a front side parallel to the back side of the window.
According to claim 1,
The second image sensor is a hyperspectral sensor extending in a direction perpendicular to the rear surface of the window.
According to claim 1,
It further includes a first shade adjacent to a front surface of the first mirror, and a second shade adjacent to a rear surface of the second optical element,
Each of the first shade and the second shade is configured to remove a zero-order diffracted light component that is not diffracted in the second optical element.
A hyperspectral sensor including a window, a first optical system configured to allow first incident light to enter, and a second optical system spaced apart from the first optical system and configured to allow second incident light to enter;
an optical image stabilization (OIS) module configured to detect and control movement of the hyperspectral sensor; and
A processor connected to the first optical system and the second optical system of the hyperspectral sensor and the OIS module,
The first optical system is:
a first focusing portion provided on a rear surface of the window and configured to focus the first incident light; and
A first image sensor provided on a rear surface of the first focusing unit and configured to detect the first incident light that has passed through the first focusing unit,
The second optical system is:
a first mirror having a front surface inclined with respect to the rear surface of the window and configured to reflect the second incident light;
first and second optical elements spaced apart from the first mirror and configured to diffract the second incident light;
a second focusing unit spaced apart from the second optical element and configured to focus diffracted light diffracted by the second optical element; and
A hyperspectral imaging system comprising a second image sensor provided on a rear surface of the second focusing unit and configured to detect the diffracted light that has passed through the second focusing unit.
According to claim 11,
The first optical element is one or a combination of two or more selected from a surface diffraction grating, a volume Bragg grating, and at least one prism,
A hyperspectral imaging system wherein the second optical element is a volume Bragg grating.
According to claim 11,
Each of the first and second optical elements is a volume Bragg grating,
A hyperspectral imaging system wherein the grating period of the first optical device is different from the grating period of the second optical device.
According to claim 11,
The OIS module includes position sensors and OIS drivers connected to the hyperspectral sensor, and an OIS control unit connected to the position sensors and the OIS drivers, respectively,
The OIS control unit is configured to receive location information of the position sensors and transmit it to the OIS driving units.
According to claim 11,
a scan mirror control module connected to the first mirror of the second optical system;
a focusing unit control module connected to the first focusing unit of the first optical system and the second focusing unit of the second optical system; and
Further comprising a light source control module connected to the processor,
The light source control module is a hyperspectral imaging system including an LED driver connected to the processor, and an LED light source connected to the LED driver and having a known spectrum in advance.
A window configured to allow first and second incident light to enter, a first optical system configured to allow the first incident light passing through the window to enter, and a window spaced apart from the first optical system to allow the second incident light to enter through the window. In the hyperspectral sensor including a second optical system configured to:
Setting up the measurement area;
capturing a reference image by the first optical system;
taking hyperspectral images by the second optical system;
synthesizing the hyperspectral images;
Comparing a composite image combining the hyperspectral images with the reference image; and
Including outputting measurement results,
The first optical system is:
a first focusing portion provided on a rear surface of the window and configured to focus the first incident light; and
A first image sensor provided on a rear surface of the first focusing unit and configured to detect the first incident light that has passed through the first focusing unit,
The second optical system is:
a first mirror having a front surface inclined with respect to the rear surface of the window and configured to reflect the second incident light;
an angle conversion unit for each wavelength, which is spaced apart from the first mirror and converts the second incident light to proceed at an angle that satisfies the Bragg condition for each wavelength;
a second focusing unit spaced apart from the wavelength-specific angle converting unit and configured to focus the diffracted light diffracted by the wavelength-specific angle converting unit; and
A hyperspectral imaging method comprising a second image sensor provided on a rear surface of the second focusing unit and configured to detect the diffracted light that has passed through the second focusing unit.
According to claim 16,
Capturing the hyperspectral images by the second optical system includes a plurality of capturing operations,
Each of the above shooting operations:
rotating the first mirror;
capturing a hyperspectral image by the second image sensor; and
A hyperspectral imaging method comprising correcting distortion of the hyperspectral image.
According to claim 17,
A hyperspectral imaging method in which the capturing of the hyperspectral image by the second image sensor is performed once when the light source is turned on and once when the light source is turned off.
According to claim 16,
Before setting the measurement area, it further includes entering a measurement mode,
The measurement mode is a hyperspectral imaging method selected from a point spectrum measurement mode, a spectrum measurement mode after shape recognition, a spatial low resolution measurement mode, and a spatial high resolution measurement mode.
According to claim 16,
The wavelength-specific angle converter is a hyperspectral imaging method in which one or a combination of two or more is selected from a surface diffraction grating, a volume Bragg grating, and at least one or more prisms.

Images