KR20220131825A - Hyperspectral sensor, hyperspectral imaging system including the sensor, and hyperspectral imaging method using the system - Google Patents

Abstract

The present invention provides a miniaturized and simplified hyperspectral sensor, a hyperspectral imaging system including the same, and a hyperspectral imaging method using the same. The hyperspectral sensor includes: a window; a first focusing unit provided on the rear surface of the window and including a plurality of lenses; a first image sensor provided on the rear surface of the first focusing unit and having a front surface parallel to the rear surface of the window; a first mirror spaced apart from the first focusing unit and the first image sensor and having a front surface inclined with respect to the rear surface of the window; a first optical element spaced apart from the first mirror; a second optical element spaced apart from the first optical element and having a periodic refractive index distribution therein; a second focusing unit spaced apart from the second optical element and including a plurality of lenses; and a second image sensor provided on the rear surface of the second focusing unit.

Description

Hyperspectral sensor, hyperspectral imaging system including same, and hyperspectral imaging method using same

The present invention relates to a hyperspectral sensor, and more particularly, a hyperspectral sensor including a general camera optical system and a hyperspectral optical system, a hyperspectral imaging system including the same, and a hyperspectral imaging method for grasping a spectrum of an analysis target using the same is about

In order to detect dangerous substances, the growth status of agricultural crops, and algal blooms in lakes in a wide area, a hyperspectral sensor that can grasp the spectroscopic characteristics of the object in addition to the spatial distribution is used. Such hyperspectral sensors are being mounted on manned aircraft and small unmanned aerial vehicles. When it is mounted on a small unmanned vehicle, it has the advantage that the range of application is very wide, and research to realize this is in progress.

In particular, when the hyperspectral sensor includes many components having a focal length, the overall size of the sensor may be increased, and the structure becomes complicated and manufacturing becomes difficult to correct aberrations occurring in each of the components having a focal length. there is Accordingly, many efforts are being made to reduce the size and weight of the hyperspectral sensor.

An object of the present invention is to provide a miniaturized and simplified hyperspectral sensor, a hyperspectral imaging system including the same, and a hyperspectral imaging method using the same.

The problem to be solved by the present invention is not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

The hyperspectral sensor according to an embodiment of the present invention is provided on a window, a rear surface of the window, a first focusing part including a plurality of lenses, and provided on a rear surface of the first focusing part, the rear surface of the window and A first image sensor having a parallel front surface, the first focusing unit, and a first mirror spaced apart from the first image sensor and having a front surface inclined with respect to the rear surface of the window, and a first light spaced apart from the first mirror device, a second optical device spaced apart from the first optical device and having a periodic refractive index distribution therein, a second focusing part spaced apart from the second optical device and including a plurality of lenses, and the second focusing part It may include a second image sensor provided on the rear surface.

The first optical device may have a periodic refractive index distribution therein, and a thickness of the first optical device may be smaller than a thickness of the second optical device.

The first optical element may be any one selected from a surface diffraction grating, a volume Bragg grating, and at least one prism, or a combination of two or more.

The first mirror may be connected to a driving unit, and the first mirror may be configured to rotate by the driving unit.

The light diffracted by the second optical element may be configured to travel in parallel with the rear surface of the window.

The front surface of the first mirror may be a curved surface having a curvature.

It further includes a third optical element between the first mirror and the first optical element, wherein the third optical element may be a close-up lens or an achromatic lens.

and a second mirror spaced apart from the second optical element and the second focusing part and having a front surface inclined with respect to the rear surface of the window, wherein the second image sensor has a front surface parallel to the rear surface of the window. can have

The second image sensor may extend in a direction perpendicular to the rear surface of the window.

A first shade adjacent to the front surface of the first mirror, and a second shade adjacent to the rear surface of the second optical element, wherein each of the first shade and the second shade is diffracted in the second optical element It can be configured to remove the 0th-order diffracted light component that does not occur.

A hyperspectral imaging system according to an embodiment of the present invention includes a window, a first optical system configured to receive a first incident light, and a second optical system spaced apart from the first optical system and configured to receive a second incident light. A sensor, an optical image stabilization (OIS) module configured to detect and control the movement of the hyperspectral sensor, and the first and second optical systems of the hyperspectral sensor, and a processor connected to the OIS module. can In this case, the first optical system is provided on a rear surface of the window, a first focusing part configured to focus the first incident light, and a rear surface of the first focusing part, the first optical system passing through the first focusing part a first image sensor configured to detect a first incident light, wherein the second optical system has a front surface inclined with respect to the rear surface of the window, a first mirror configured to reflect the second incident light; first and second optical elements spaced apart from the first mirror and configured to diffract the second incident light; spaced apart from the second optical element to focus the diffracted light diffracted by the second optical element It may include a second focusing part configured to be configured, and a second image sensor provided on a rear surface of the second focusing part and configured to detect the diffracted light passing through the second focusing part.

The first optical element may be any one or a combination of two or more selected from a surface diffraction grating, a volume Bragg grating, and at least one or more prisms, and the second optical element may be a volume Bragg grating. .

Each of the first and second optical elements may be a volume Bragg grating, and a grating period of the first photo element may be different from a grating period of the second photo element.

The OIS module includes position sensors and OIS driving units connected to the hyperspectral sensor, and an OIS control unit connected to the position sensors and the OIS driving units, respectively, wherein the OIS control unit receives the position information of the position sensors It may be configured to receive and transmit to the OIS drivers.

a scan mirror control module connected to the first mirror of the second optical system, a focusing unit control module connected to the first focusing unit of the first optical system and the second focusing unit of the second optical system, and the processor; Further comprising a light source control module connected, the light source control module may include an LED driver connected to the processor, and an LED light source connected to the LED driver and knowing a spectrum in advance.

The hyperspectral imaging method according to an embodiment of the present invention includes setting a measurement area, photographing a reference image by the first optical system, photographing hyperspectral images by the second optical system, and the hyperspectral image It may include synthesizing the values, comparing the synthesized image obtained by synthesizing the hyperspectral images with the reference image, and outputting a measurement result. In this case, the first optical system is provided on the rear surface of the window, a first focusing part configured to focus the first incident light, and the first optical system provided on the rear surface of the first focusing part, and passing through the first focusing part a first image sensor configured to detect a first incident light, a second optical system having a front surface inclined with respect to the rear surface of the window, a first mirror configured to reflect the second incident light, the first mirror is spaced apart from, an angle conversion unit for each wavelength that converts the second incident light to proceed at an angle that satisfies the Bragg condition for each wavelength; a second focusing portion configured to focus the light; and a second image sensor provided on a rear surface of the second focusing portion and configured to detect the diffracted light passing through the second focusing portion.

Taking the hyperspectral images by the second optical system includes a plurality of photographing operations, and each of the photographing operations includes rotating the first mirror, photographing a hyperspectral image by the second image sensor and correcting distortion of the hyperspectral image.

Taking the hyperspectral image by the second image sensor may be performed once with the light source turned on and once with the light source turned off.

Before setting the measurement area, further comprising: inputting a measurement mode, wherein the measurement mode is a point spectrum measurement mode, a spectrum measurement mode after shape recognition, a spatial low resolution measurement mode, and a spatial high resolution (high resolution) may be any one selected from the measurement mode.

The angle converter for each wavelength may be any one selected from a surface diffraction grating, a volume Bragg grating, and at least one prism or a combination of two or more.

The hyperspectral sensor according to an embodiment of the present invention can filter incident light through a volume Bragg grating, so that the sensor can be miniaturized, simplified, and easily manufactured and aligned.

In addition, the hyperspectral imaging system and the hyperspectral imaging method using the same according to an embodiment of the present invention can minimize the scan area, so that when the exposure time is long, it takes a long time to scan the entire area.

In addition, in the hyperspectral imaging method according to an embodiment of the present invention, reliability may be improved by comparing the reference image captured by the first image sensor with the hyperspectral image captured by the second image sensor.

1 is a conceptual diagram for explaining the structure of a hyperspectral sensor according to embodiments of the present invention.
FIG. 2 is an enlarged view for explaining a first optical device and a second optical device of the hyperspectral sensor according to FIG. 1 .
3A, 3B, 3C, and 3D are conceptual views illustrating a path of light incident to a hyperspectral sensor according to embodiments of the present invention. path, and FIG. 3D shows the light path in the second section.
4, 5, and 6 are conceptual views for explaining the structure of a hyperspectral sensor according to other embodiments of the present invention.
7, 8A, 8B, 8C, and 8D are simulation results for explaining an image measured by a hyperspectral sensor according to embodiments of the present invention.
9 is a conceptual diagram illustrating a hyperspectral imaging system including a hyperspectral sensor according to embodiments of the present invention.
10 is a flowchart illustrating a hyperspectral imaging method using a hyperspectral imaging system according to embodiments of the present invention.
11 is a timing diagram for explaining a hyperspectral imaging method using a hyperspectral imaging system according to embodiments of the present invention.

In order to fully understand the configuration and effects of the present invention, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The present invention is not limited to the embodiments disclosed below, but may be implemented in various forms and various modifications and changes may be made. However, it is provided so that the disclosure of the present invention is complete through the description of the present embodiment, and to fully inform those of ordinary skill in the art to which the present invention belongs, the scope of the invention. In the accompanying drawings, the components are enlarged in size than the actual size for convenience of description, and the ratio of each component may be exaggerated or reduced.

The terminology used herein is for the purpose of describing the embodiment and is not intended to limit the present invention. Also, unless otherwise defined, terms used herein may be interpreted as meanings commonly known to those of ordinary skill in the art.

In this specification, the singular also includes the plural, unless specifically stated otherwise in the phrase. As used herein, 'comprises' and/or 'comprising' refers to the presence of one or more other components, steps, operations and/or elements mentioned. or addition is not excluded.

When a layer is referred to herein as being 'on' another layer, it may be formed directly on top of the other layer, or a third layer may be interposed therebetween.

Although terms such as first and second are used herein to describe various regions, layers, and the like, these regions and layers should not be limited by these terms. These terms are only used to distinguish one region or layer from another. Accordingly, a part referred to as the first part in one embodiment may be referred to as the second part in another embodiment. The embodiments described and illustrated herein also include complementary embodiments thereof. Parts indicated with like reference numerals throughout the specification indicate like elements.

Hereinafter, a hyperspectral sensor, a hyperspectral imaging system including the same, and a hyperspectral imaging method using the same according to embodiments of the present invention will be described in detail with reference to the drawings.

1 is a conceptual diagram for explaining the structure of a hyperspectral sensor according to embodiments of the present invention. FIG. 2 is an enlarged view for explaining a first optical device and a second optical device of the hyperspectral sensor according to FIG. 1 .

Referring to FIG. 1 , the hyperspectral sensor according to the present invention may include a window W, a first optical system P1 and a second optical system P2 . The first optical system P1 and the second optical system P2 may be spaced apart from each other. The first optical system P1 may include a first focusing part F1 and a first image sensor S1 . The second optical system P2 may include a first mirror M1 , a first optical device OC1 , a second optical device OC2 , a second focusing part F2 , and a second image sensor S2 . . The first incident light IL1 may be incident toward the first optical system P1 , and the second incident light IL2 may be incident toward the second optical system P2 . The first optical system P1 may be the same as that of a general camera, and the second optical system P2 may be hyperspectral optics. Hereinafter, each of the configurations of the first and second optical systems P1 and P2 will be described in detail.

The window W may have a planar shape that is perpendicular to the first direction D1 and extends in the second direction D2 and the third direction D3 . The first to third directions D1 , D2 , and D3 may be directions orthogonal to each other. The first and second incident lights IL1 and IL2 may be incident parallel to the first direction D1 toward the window W, but this is merely exemplary and the present invention is not limited thereto. The two incident lights IL1 and IL2 may be incident from different directions crossing the front surface Wf of the window W. In other words, the front surface Wf of the window W may face the analysis target and receive the first and second incident lights IL1 and IL2 emitted from the analysis target. The window W may protect the first and second optical systems P1 and P2 on the rear surface Wb from external impact and/or contamination.

In this specification, the front surface is defined as a surface facing the light incident toward each component, and the rear surface is defined as a surface that emits light passing through each component. For example, the front surface may be a surface facing the direction opposite to the first direction D1 , and the rear surface may be a surface facing the first direction D1 .

A first focusing portion F1 may be provided on the rear surface Wb of the window W. The first focusing portion F1 may be provided between the window W and the first image sensor S1 . The first focusing part F1 may be configured to focus the first incident light IL1 passing through the window W. The first focusing part F1 may include a plurality of lenses. A central axis of each of the plurality of lenses of the first focusing part F1 may coincide with a central axis of the first image sensor S1 . A central axis of each of the plurality of lenses of the first focusing part F1 may be parallel to the first direction D1, for example. The plurality of lenses of the first focusing portion F1 are not limited to the illustrated shape and the illustrated curvature of the side surface, but may have various shapes and curvatures.

A first image sensor S1 may be provided on the rear surface of the first focusing part F1 . The first image sensor S1 may have a front surface S1f parallel to the rear surface Wb of the window W. The first image sensor S1 may be configured to detect the first incident light IL1 passing through the window W and the first focusing portion F1 . The first image sensor S1 may be disposed such that its front surface S1f coincides with a focal plane of the first incident light IL1.

The first mirror M1 may be provided at a position spaced apart from the first focusing part F1 and the first image sensor S1 in the third direction D3 . The first mirror M1 may be spaced apart from the rear surface Wb of the window W in the first direction D1 . The first mirror M1 may have a front surface M1f inclined with respect to the rear surface Wb of the window W (ie, with respect to the third direction D3). An angle between the front surface M1f of the first mirror M1 and the rear surface Wb of the window W may be greater than 0 degrees and less than 90 degrees. A central axis of the first mirror M1 may be connected to the driving unit. The first mirror M1 may be configured to rotate clockwise or counterclockwise by the driving unit. The first mirror M1 may be configured to reflect the second incident light IL2 traveling from the first incident light IL1 to a path shifted in the third direction D3 .

The first and second optical devices OC1 and OC2 may be provided at positions spaced apart from the first mirror M1 in the third direction D3 . The first optical device OC1 may be provided between the first mirror M1 and the second optical device OC2 in the third direction D3 , and the second optical device OC2 is disposed in the third direction D3 . may be provided between the first optical device OC1 and the second focusing part F2 . The first and second optical devices OC1 and OC2 may be spaced apart from the rear surface Wb of the window W in the first direction D1 . The first and second optical devices OC1 and OC2 may overlap each other in the first direction D1 , but this is merely exemplary and the present invention is not limited thereto.

The first optical device OC1 may convert the second incident light IL2 to travel at an angle satisfying the Bragg condition for each wavelength. In other words, the degree of changing the propagation angle of the second incident light IL2 according to the wavelength of the first optical device OC1 may be different. In this specification, the first optical device OC1 may be referred to as an angle converter for each wavelength.

The first optical device OC1 may be, for example, a surface diffraction grating including periodic protrusions on the surface. As another example, the first optical device OC1 may be a volume Bragg grating having a periodic refractive index distribution therein. The volume Bragg grating may be referred to as a bulk Bragg grating or a volume holographic grating. As another example, the first optical device OC1 may be a prism.

The first optical element OC1 may be, for example, any one selected from a surface diffraction grating, a volume Bragg grating, and at least one or more prisms, or a combination of two or more selected from a surface diffraction grating, a volume Bragg grating, and at least one or more prisms. . When the first optical device OC1 includes a diffraction grating and at least one prism, the at least one prism may be provided between the diffraction grating and a second optical device OC2 to be described later.

The second optical device OC2 may be, for example, a volume Bragg grating having a periodic refractive index distribution therein. The second optical device OC2 may be spaced apart from the first optical device OC1 .

When each of the first and second optical devices OC1 and OC2 is a volume Bragg grating, the thickness of the second optical device OC2 may be different from that of the first optical device OC1 . For example, the thickness of the second optical device OC2 may be greater than the thickness of the first optical device OC1 . Each of the first and second optical elements OC1 and OC2 may include a support portion and a grating portion (ie, an inner portion whose refractive index is periodically changed), and the first and second optical elements OC1 and OC2 Each thickness means the thickness of only the grating portion excluding the thickness of the supporting portion. In the volume Bragg grating, the angular selectivity may increase as the thickness of the grating increases while the refractive index difference (ie, the refractive index difference between the high refractive index portion and the low refractive index portion) is small.

When each of the first and second optical elements OC1 and OC2 is a volume Bragg grating, the grating period Λ 1 of the first photo element OC1 is the grating period Λ ₂ of the _second photo element OC2 . may be different from For example, the lattice period Λ ₁ of the first optical device OC1 may be greater than the lattice period Λ ₂ of the second optical device OC2 . The lattice period Λ ₁ of the first optical device OC1 and the lattice period Λ ₂ of the second optical device OC2 may satisfy the following [Equation 1], and accordingly, A wavelength dependent aiming angle error for each wavelength with respect to the reference wavelength λ may be minimized. The aiming angle error for each wavelength means an angle deviating from the reference angle when light having a reference wavelength and incident from the reference angle satisfies the Bragg condition and light incident at an angle deviating from the reference angle for each wavelength satisfies the Bragg condition.

[Equation 1]

For example, in order to satisfy [Equation 1], the reference wavelength λ is about 620 nm, and the grating period Λ ₂ of the second optical device OC2 is about 0.833 μm (that is, the grating density is about 1200 lines/mm), the grating period Λ ₁ of the first optical element OC1 is about 1.578 μm (ie, the grating density is about 633.8 lines/mm). When the wavelength is about 400 nm, the aiming angle error for each wavelength is about 0.11 degrees, and when the wavelength is about 800 nm, the aiming angle error for each wavelength is about 0.08 degrees.

For example, the first light having a wavelength of about 620 nm and incident at the reference angle and the second light having a wavelength of about 550 nm and incident at an angle deviating from the reference angle are the first optical element OC1 and the second light. When diffracted by the device OC2 , the first and second lights may be detected at the same position on the front surface S2f of the second image sensor S2 even though they have different wavelengths. When the second optical device OC2 has high angular selectivity, the first light is diffracted by the second optical device OC2 by satisfying the Bragg condition, but the second light does not satisfy the Bragg condition and thus the second light It is not diffracted from the element OC2 and travels straight and is not detected on the front surface S2f of the second image sensor S2 , and accordingly, the second optical system P2 may operate as hyperspectral optics.

When the first optical device OC1 is a surface diffraction grating and the second optical device OC2 is a volume Bragg grating, the second optical device OC2 may be provided in parallel with the first optical device OC1 . When the first and second optical devices OC1 and OC2 are provided in parallel with each other, the lattice period Λ ₂ of the second optical device OC2 is the lattice period Λ ₁ of the first optical device OC1 . can be twice the

Referring to FIG. 2 , the first optical device OC1 may be configured to diffract the second incident light IL2 into the first diffracted light DL1 , and the second optical device OC2 may include the first diffracted light It may be configured to diffract the DL1 to the second diffracted light DL2 . In this case, each of the first and second optical devices OC1 and OC2 may be a diffraction grating having a grating period, and in particular, the second optical device OC2 may be a volume Bragg grating.

Specifically, the first optical device OC1 may diffract the second incident light IL2 into the first diffracted light DL1 according to a grating equation expressed by Equation 2 below.

[Equation 2]

In this case, m is the diffraction order, λ is the wavelength of the second incident light IL2, Λ ₁ is the grating period of the first optical device OC1, and α is the second incident light IL2 and the first An angle formed by a perpendicular line OC1p of the surface of the optical element OC1 , and β is an angle formed between the first diffracted light DL1 and a perpendicular OC1p of the surface of the first optical element OC1 . to be. When α and β are equal, the Bragg condition is satisfied and the diffraction efficiency of the first optical device OC1 is maximized.

In addition, the second optical device OC2 satisfies the Bragg condition, and converts the first diffracted light DL1 into the second diffracted light DL2 according to the grating equation expressed by Equation 3 below. can be diffracted.

[Equation 3]

In this case, Λ ₂ is the grating period of the second optical device OC2 , and θ is the angle formed by the first diffracted light DL1 and the perpendicular OC2p of the surface of the second optical device OC2 .

The second optical device OC2 may be configured such that the second diffracted light DL2 travels in parallel with the third direction D3 . More specifically, the angle formed by the normal OC2p of the surface of the second optical device OC2 with the third direction D3 is the first diffracted light DL1 and the normal OC2p of the surface of the second optical device OC2. ) may be substantially the same as the angle θ formed.

The second optical device OC2 may have a surface inclined with respect to the surface of the first optical device OC1 . An angle φ at which the surface of the second optical device OC2 is inclined with respect to the surface of the first optical device OC1 is θ−β. For example, the reference wavelength λ is about 620 nm, the grating period Λ ₂ of the second optical device OC2 is about 0.833 μm (ie, the grating density is about 1200 lines/mm), and the first optical device When the grating period (Λ ₁ ) of (OC1) is about 1.578 μm (i.e., the grating density is about 633.8 lines/mm), θ is about 21.839 degrees, β is about 11.331 degrees, and φ is about 10.508 degrees.

Referring back to FIG. 1 , the hyperspectral sensor according to the present invention further includes a third optical device OC3 provided between the first mirror M1 and the first optical device OC1 in a third direction D3 . can do. The third optical device OC3 may be configured to collimate the second incident light IL2 reflected from the first mirror M1 and directed toward the first optical device OC1 . The third optical element OC3 may be a close-up lens or an achromatic lens that reduces aberration according to wavelength.

When the third optical device OC3 is an achromatic lens, the third optical device OC3 may include a first achromatic lens and a second achromatic lens bonded to each other and having different optical characteristics and/or different structures. . Each of the first and second achromatic lenses may include glass or plastic. The N _d /V _d value (ie, (d-line refractive index)/(abbe number)) of the first achromatic lens may be different from the N _d /V _d value of the second achromatic lens. For example, the N _d /V _d value of the first achromatic lens is about 1.805/25.36, and the N _d /V _d value of the second achromatic lens is about 1.651/55.89. The front surface of the first achromatic lens, the rear surface of the first achromatic lens (ie, the front surface of the second achromatic lens), and the rear surface of the second achromatic lens may have different radii of curvature. For example, the radius of curvature of the front surface of the first achromatic lens is infinity (that is, the front surface of the first achromatic lens is flat), the radius of curvature of the rear surface of the first achromatic lens is about -285.815 mm, and that of the second achromatic lens The radius of curvature at the back is about 167.715 mm. The thickness of the first achromatic lens may be substantially the same as that of the second achromatic lens, but this is merely exemplary and the present invention is not limited thereto. For example, each of the first and second achromatic lenses may have a thickness of about 0.1 mm to about 1 mm.

The second focusing portion F2 may be provided at a position spaced apart from the second optical device OC2 in the third direction D3 . The second focusing portion F2 may be spaced apart from the rear surface Wb of the window W in the first direction D1 . The distance in the first direction D1 between the front surface of the second focusing portion F2 and the rear surface Wb of the window W is, for example, the distance between the front surface of the first focusing portion F1 and the window W It may be greater than the distance in the first direction D1 between the rear surfaces Wb. The second focusing part F2 may be configured to focus the second diffracted light DL2 diffracted by the second optical element OC2 . The second focusing part F2 may include a plurality of lenses. A central axis of each of the plurality of lenses of the second focusing part F2 may coincide with a central axis of the second image sensor S2 . A central axis of each of the plurality of lenses of the second focusing part F2 may be parallel to the first direction D1, for example. The plurality of lenses of the second focusing portion F2 may have various shapes and curvatures without being limited to the illustrated shape and curvature of the illustrated side surface.

A second image sensor S2 may be provided on the rear surface of the second focusing part F2 . The second image sensor S2 may have a front surface S2f parallel to the rear surface Wb of the window W. The second image sensor S2 may be configured to detect the second diffracted light DL2 passing through the second focusing part F2 . The second image sensor S2 may be disposed such that its front surface S2f coincides with a focal plane of the second diffracted light DL2 .

The hyperspectral sensor according to the present invention includes a second mirror M2 spaced apart from the second optical element OC2 in a third direction D3 and spaced apart from the second focusing part F2 in a first direction D1. may include more. The second mirror M2 may have a front surface M2f inclined with respect to the rear surface Wb of the window W (ie, with respect to the third direction D3). An angle between the front surface M2f of the second mirror M2 and the rear surface Wb of the window W may be, for example, about 45 degrees, and the angle may be fixed. The second mirror M2 may be configured to reflect the second diffracted light DL2 to travel to the second focusing portion F2 .

The hyperspectral sensor according to the present invention may further include an aperture stop AS overlapping any one of the plurality of lenses of the second focusing unit F2. For example, the aperture stop AS may overlap the one closest to the second optical element OC2 among the plurality of lenses of the second focusing part F2 .

The length L ₃ of the second optical system P2 in the third direction D3 may be expressed by the following [Equation 4]. The length L ₃ of the second optical system P2 in the third direction D3 may be, for example, about 10 mm to about 15 mm.

[Equation 4]

L ₃ =L _a +L _b +L _c

In this case, L _a is the distance in the third direction D3 between the center of the front surface M1f of the first mirror M1 and the center of the first optical device OC1 , and L _b is the first optical device OC1 . ) is the distance in the third direction D3 between the center of the second optical device OC2 and the center of the second optical device OC2, and L _c is the center of the second optical device OC2 and the front surface M2f of the second mirror M2. is the distance in the third direction D3 between the centers of .

The length L ₁ of the second optical system P2 in the first direction D1 may be expressed by the following [Equation 5]. The length L ₁ of the second optical system P2 in the first direction D1 may be, for example, about 5 mm to about 15 mm.

[Equation 5]

L ₁ =L ₀ -L _g +L _m +L _f

In this case, L ₀ is a distance in the first direction D1 between the center of the rear surface Wb of the window W and the front surface M1f of the first mirror M1, and L _g is the first mirror M1. is the distance in the first direction D1 between the center of the front surface M1f of the second mirror M2 and the center of the front surface M2f of the second mirror M2, and L _m is the center of the front surface M2f of the second mirror M2 is a distance in the first direction D1 between the front surface of the second focusing part F2 and L _f is the first direction between the front surface of the second focusing part F2 and the rear surface of the second image sensor S2 It is the distance to (D1).

The distance L _g in the first direction D1 between the center of the front surface M1f of the first mirror M1 and the center of the front surface M2f of the second mirror M2 is obtained by the following [Equation 6] can be expressed The distance L _g in the first direction D1 between the center of the front surface M1f of the first mirror M1 and the center of the front surface M2f of the second mirror M2 is, for example, about 2 mm to about 6 mm.

[Equation 6]

L _g =L _{a ·} tan2φ+L _{b ·} tan2θ

In this case, 2φ is the angle at which the path of the second incident light IL2 reflected from the first mirror M1 is inclined with respect to the third direction D3, and 2θ is the path of the first diffracted light DL1 in the third direction. It is the inclination angle with respect to (D3).

Meanwhile, the length of the second optical system P2 in the second direction D2 may be the same as the length of the first mirror M1 in the second direction D2 . A length of the second optical system P2 in the second direction D2 may be determined by a viewing angle in the second direction D2 . A length of the second optical system P2 in the second direction D2 may be, for example, about 5 mm to about 15 mm.

3A, 3B, 3C, and 3D are conceptual views illustrating a path of light incident to a hyperspectral sensor according to embodiments of the present invention. path, and FIG. 3D shows the light path in the second section.

3A, 3B and 3C show the first incident light IL1 emitted from sources having the same position in the first direction D1 and the second direction D2 but different positions in the third direction D3. ) and paths of the second incident light IL2 are shown. In the first optical system P1 , the first incident light IL1 incident from different directions may form an image at different positions of the first image sensor S1 according to the incident direction regardless of wavelength. On the other hand, in the second optical system P2 , the second incident light IL2 incident from any one direction may include first to third lights IL21 , IL22 , and IL23 having different wavelengths, and The third lights IL21 , IL22 , and IL23 may form images at different positions of the second image sensor S2 according to wavelengths. For example, the first light IL21 may have a shorter wavelength than the second light IL22 , and the second light IL22 may have a shorter wavelength than the third light IL23 .

The angle at which the first mirror M1 is inclined in FIGS. 3A, 3B and 3C may be different from each other. The first mirror M1 may scan the entire area of the source extending in the third direction D3 by rotating it counterclockwise (or clockwise).

3D shows the first incident light IL1 and the second incident light emitted from sources having the same position in the first direction D1 and the third direction D3 but having different positions in the second direction D2. IL2) pathways. In the first optical system P1 , the first incident light IL1 incident from different directions may form an image at different positions of the first image sensor S1 according to the incident direction regardless of wavelength. In the second optical system P2 , the second incident light IL2 incident from different directions may form an image at different positions of the second image sensor S2 according to the incident direction regardless of wavelength.

4, 5, and 6 are conceptual views for explaining the structure of a hyperspectral sensor according to other embodiments of the present invention. Hereinafter, for convenience of description, descriptions of the items substantially the same as those described with reference to FIGS. 1 and 2 will be omitted and differences will be described in detail.

Referring to FIG. 4 , the third optical device OC3 described with reference to FIG. 1 may be omitted. That is, the second incident light IL2 reflected from the front surface M1f of the first mirror M1 may travel to the first and second optical elements OC1 and OC2 without passing through a separate collimating optical system. A front surface M1f of the first mirror M1 may be a curved surface having a curvature, and a rear surface of the first mirror M1 may be a flat surface. The focal length of the first mirror M1 may be, for example, about 200 mm to 400 mm.

According to embodiments, the first mirror M1 may be a reflective grating in which a grating having a period is engraved on the front surface M1f. According to embodiments, the front surface M1f of the first mirror M1 may be a flat surface, and the rear surface of the first mirror M1 may be a curved surface having a curvature. The first mirror M1 may be configured to rotate clockwise or counterclockwise by the driving unit, and the surface on which the second incident light IL2 is reflected is from the front surface M1f to the rear surface (or from the rear surface to the front surface M1f). ) may be changed.

Referring to FIG. 5 , the hyperspectral sensor according to the present invention may include any one of a first shade Sh1 and a second shade Sh2.

The first shade Sh1 may be provided on the front surface M1f of the first mirror M1 . The first shade Sh1 may be configured to rotate together with the first mirror M1, for example. The second shade Sh2 may be provided between the second optical element OC2 and the second mirror M2 . The second shade Sh2 may be provided on the rear surface of the second optical device OC2 or on the front surface of the second mirror M2 . Each of the first and second shades Sh1 and Sh2 may block a portion of the second incident light IL2 to remove a 0th-order diffracted light component that is not diffracted from the second optical device OC2 . For example, each of the first shade Sh1 and the second shade Sh2 may absorb or scatter a portion of the second incident light IL2 . Accordingly, the image detected by the second image sensor S2 may become clearer.

Referring to FIG. 6 , the second mirror M2 described with reference to FIG. 1 may be omitted. In this case, the second focusing part F2 and the second image sensor S2 may be arranged along the third direction D3 . That is, the second diffracted light DL2 diffracted by the second optical element OC2 may travel in parallel with the third direction D3 and pass through the second focusing part F2 and the second image sensor S2 ) can be reached. The front surface S2f of the second image sensor S2 may extend in a direction perpendicular to the rear surface Wb of the window W (ie, the first direction D1 and the second direction D2).

At this time, the length L ₁ of the second optical system P2 in the first direction D1 is the first between the center of the rear surface Wb of the window W and the front surface M1f of the first mirror M1. It may be equal to the distance L ₀ in the direction D1. In other words, the length L ₁ of the second optical system P2 in the first direction D1 may be smaller than that described with reference to FIG. 1 .

7, 8A, 8B, 8C, and 8D are simulation results for explaining an image measured by a hyperspectral sensor according to embodiments of the present invention. 7 is an exemplary image captured by the first image sensor S1 (see FIG. 1 ), and FIGS. 8A to 8D are exemplary images captured by the second image sensor S2 (see FIG. 1 ).

7 illustrates images of a plurality of elliptical shapes positioned on a plane parallel to the second direction D2 and the third direction D3 and perpendicular to the first direction D1. The plurality of elliptical shapes may be spaced apart from each other. The plurality of elliptical shapes may have different wavelength spectra.

8A to 8D are spectral images corresponding to the first scan area 10 , the second scan area 20 , the third scan area 30 , and the fourth scan area 40 of FIG. 7 . The first mirror M1 of FIG. 1 may be rotated to capture a spectrum image of each of the first to fourth scan areas 10 , 20 , 30 , and 40 . In other words, when a spectrum image of each of the first to fourth scan regions 10 , 20 , 30 , and 40 is taken, the angle at which the first mirror M1 of FIG. 1 is inclined may be different from each other.

9 is a conceptual diagram illustrating a hyperspectral imaging system including a hyperspectral sensor according to embodiments of the present invention.

Referring to FIG. 9 , the hyperspectral imaging system according to the present invention includes the hyperspectral sensor 100 , the OIS module 200 , the scan mirror control module 300 , the focusing unit control module 400 described with reference to FIG. 1 , It may include a processor 500 and a light source control module 600 .

The OIS module 200 includes first and second position sensors 210 and 220 , first and second OIS actuators 230 and 240 , and an OIS gyroscope. It may include an (OIS gyroscope) 250 and an OIS controller (OIS controller) 260 . The first and second position sensors 210 and 220 may detect movement of the hyperspectral sensor 100 in the second direction (D2, see FIG. 1) and the third direction (D3, see FIG. 1). . The first and second OIS drivers 230 and 240 may suppress the movement of the hyperspectral sensor 100 . The OIS controller 260 may receive the location information of the first and second location sensors 210 and 220 and transmit it to the first and second OIS drivers 230 and 240 . The OIS gyroscope 250 may be connected to the OIS control unit 260 . The OIS controller 260 may receive rotation information of the OIS gyroscope 250 and transmit it to the first and second OIS drivers 230 and 240 . The OIS module 200 may prevent and/or minimize the movement of the hyperspectral sensor 100 due to vibration or the like.

The scan mirror control module 300 may include a scan mirror driver 310 and a scan mirror controller 320 . The scan mirror driver 310 may be connected to the first mirror M1 of the hyperspectral sensor 100 , and may rotate the first mirror M1 . The scan mirror controller 320 may control the rotational movement of the first mirror M1 through the scan mirror driver 310 .

The focusing unit control module 400 may be connected to the first and second focusing units F1 and F2 of the hyperspectral sensor 100 . The focusing part control module 400 may control a focal length of each of the first and second focusing parts F1 and F2 .

The processor 500 may be connected to the first and second image sensors S1 and S2 of the hyperspectral sensor 100 , the OIS module 200 , the scan mirror control module 300 , and the focusing unit control module 400 . and can control their overall operation. The processor 500 may be, for example, a general-purpose processor or an application processor (AP).

The light source control module 600 may be connected to the processor 500 . The light source control module 600 may include an LED driver 610 and an LED light source 620 . When using the LED light source 620 whose spectrum is known in advance, the light source control module 600 may calculate reflectance for each wavelength of the analysis target from the measured hyperspectral image. According to embodiments, the light source control module 600 may be provided as an independent module outside the hyperspectral imaging system when the volume is large.

10 is a flowchart illustrating a hyperspectral imaging method using a hyperspectral imaging system according to embodiments of the present invention. 11 is a timing diagram for explaining a hyperspectral imaging method using a hyperspectral imaging system according to embodiments of the present invention.

9, 10 and 11 , the hyperspectral imaging method includes entering a measurement mode (S100), setting a measurement area (S200), photographing a reference image (S300), and using a scan mirror Rotating (S400), taking a hyperspectral image (S500), correcting the distortion of the hyperspectral image (S600), synthesizing the image (S700), comparing the reference image and the composite image ( S800), and outputting the measurement result (900).

Entering the measurement mode ( S100 ) may include preparing the hyperspectral sensor 100 and determining the execution direction of software for driving the hyperspectral sensor 100 through the measurement mode desired by the user. The measurement mode may be, for example, any one selected from a point spectrum measurement mode, a spectrum measurement mode after shape recognition, a spatial low resolution measurement mode, and a spatial high resolution measurement mode.

The point spectrum measurement mode is a method of measuring only the spectrum of any one point in the image of the first image sensor S1. The point spectrum measurement mode has a much faster measurement speed and less data than other modes.

The spectrum measurement mode after shape recognition is a method of measuring the spectrum inside the boundary region after analyzing the boundary of the image of the first image sensor S1 by image processing. After shape recognition, the spectrum measurement mode is useful for spectral analysis of objects with relatively clear boundaries, such as grains and pills.

The spatial low-resolution measurement mode is a method using a pixel binning method of bundling pixels of the second image sensor S2 arranged along one direction (the second direction D2 in FIG. 1 ). The spatial low-resolution measurement mode has a larger signal-to-noise ratio and shorter transmission time than other modes, and accordingly, the effect of motion blur effect is small and the measurement time is short. On the other hand, the spatial high-resolution measurement mode uses less pixel binning than the spatial low-resolution measurement mode, and has higher spatial resolution than other measurement modes.

Entering the measurement mode (S100) may further include adjusting the focal length of the first focusing unit F1 through the focusing unit control module 400 of the hyperspectral sensor 100 according to the distance from the analysis target. can When the analysis target is relatively close, the focal length of the first focusing part F1 may be adjusted to match the focal length of the third optical device OC3 , and when the analysis target is relatively far away, the focal length of the first focusing part F1 . The distance can be adjusted to approach infinity. In addition, inputting the measurement mode ( S100 ) may further include determining whether to dispose the third optical device OC3 , a type of light source, whether to use a pixel binning method, and the like. Setting the measurement area ( S200 ) may be performed after a predetermined start signal. Setting the measurement area (S200) proceeds as follows. First, in a state in which the LED light source 620 is turned on (On), the first image sensor S1 repeatedly captures an image and displays it on the display. The hyperspectral sensor 100 is moved so as to be in focus based on the image displayed on the display (ie, the analysis target is located at the focal length of the third optical element OC3). Then, a measurement area is selected from the image shown on the display. In point spectrum measurement mode, selecting a measurement area is selecting a point in the image shown on the display. In other measurement modes, selecting the measurement area may be determining the size and position of a rectangle including a partial area of the image shown on the display.

The rotation angle interval of the first mirror M1, the number of scans, etc. may be calculated according to the set measurement area, and the calculated result may be stored in a memory. For example, in the point spectrum measurement mode, the rotation angle of a selected point may be calculated and stored, and in the spectrum measurement mode after shape recognition, the rotation angle of each of the internal points of a plurality of recognized boundary regions may be calculated and stored. have.

According to setting the measurement area (S200) before taking the hyperspectral image (S500), the scan area can be minimized, and when the exposure time is long, the problem that it takes a long time to scan the entire area can be solved

Taking the reference image (S300) is performed once in the state in which the LED light source 620 is turned on (On), and once in the state in which the LED light source 620 is turned off (Off), by the first image sensor (S1). can be performed. By subtracting the image photographed in the state that the LED light source 620 is turned off (Off) from the image photographed in the state in which the LED light source 620 is turned on (On), the influence of ambient light may be removed.

The photographing operation (C) including rotating the scan mirror (S400), photographing the hyperspectral image (S500), and correcting the distortion of the hyperspectral image (S600) is performed by setting the first mirror M1 to a constant It can be repeated n times while rotating by the angle of the interval. The number of repetitions (n) of the photographing operation may be determined using information stored in the memory calculated by setting the measurement area ( S200 ).

The photographing operation C may include a first photographing operation C1 , a second photographing operation C2 , and a third photographing operation C3 to an n-th photographing operation Cn. Each of the first photographing operation C1 , the second photographing operation C2 , and the third photographing operation C3 to the nth photographing operation Cn rotates the first mirror M1 and then the LED light source 620 . This may be performed by the second image sensor S2 once in the on state (On) and once in the off state (Off) of the LED light source 620 .

Comparing the reference image and the synthetic image ( S800 ) may be determining whether a normalized cross-correlation (NCC) value of the synthetic image and the reference image exceeds a predetermined threshold value. If the normalized correlation value of the composite image and the reference image does not exceed the threshold value (ie, fail), the process may return to setting the measurement area ( S200 ). By comparing the reference image and the composite image ( S800 ), the reliability of the hyperspectral imaging method according to the present invention may be improved.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains can practice the present invention in other specific forms without changing the technical spirit or essential features. You will understand that there is Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

W: Windows
F1, F2: focusing part
S1, S2: image sensor
M1, M2: Mirror
OC1, OC2, OC3: Optical device
AS: Aperture Aperture
IL1, IL2: incident light
DL1, DL2: diffracted light
100: hyperspectral sensor
200: OIS module
300: scan mirror control module
400: focusing unit control module
500: processor
600: light source control module

Claims (20)

window;
a first focusing part provided on the rear surface of the window and including a plurality of lenses;
a first image sensor provided on a rear surface of the first focusing part and having a front surface parallel to the rear surface of the window;
a first mirror spaced apart from the first focusing part and the first image sensor and having a front surface inclined with respect to the rear surface of the window;
a first optical element spaced apart from the first mirror;
a second optical device spaced apart from the first optical device and having a periodic refractive index distribution therein;
a second focusing unit spaced apart from the second optical element and including a plurality of lenses; and
Hyperspectral sensor including a second image sensor provided on the rear surface of the second focusing part.
The method of claim 1,
The first optical element has a periodic refractive index distribution therein,
A thickness of the first optical device is smaller than a thickness of the second optical device.
The method of claim 1,
The first optical element is any one or a combination of two or more selected from a surface diffraction grating, a volume Bragg grating, and at least one or more prisms.
The method of claim 1,
The first mirror is connected to the driving unit,
The first mirror is a hyperspectral sensor configured to be rotated by the driving unit.
The method of claim 1,
The light diffracted by the second optical element is configured to travel in parallel with the rear surface of the window.
The method of claim 1,
The front surface of the first mirror is a hyperspectral sensor having a curvature.
The method of claim 1,
a third optical device between the first mirror and the first optical device;
and the third optical element is a close-up lens or an achromatic lens.
The method of claim 1,
and a second mirror spaced apart from the second optical element and the second focusing part and having a front surface inclined with respect to the rear surface of the window;
The second image sensor is a hyperspectral sensor having a front surface parallel to the rear surface of the window.
The method of claim 1,
The second image sensor is a hyperspectral sensor extending in a direction perpendicular to the rear surface of the window.
The method of claim 1,
a first shade adjacent to the front surface of the first mirror, and a second shade adjacent to the rear surface of the second optical element;
and each of the first shade and the second shade is configured to remove a 0th-order diffracted light component that is not diffracted in the second optical element.
a hyperspectral sensor including a window, a first optical system configured to receive a first incident light, and a second optical system spaced apart from the first optical system and configured to receive a second incident light;
an optical image stabilization (OIS) module configured to detect and control the movement of the hyperspectral sensor; and
Including a processor connected to the first optical system and the second optical system of the hyperspectral sensor, and the OIS module,
The first optical system includes:
a first focusing part provided on a rear surface of the window and configured to focus the first incident light; and
a first image sensor provided on a rear surface of the first focusing part and configured to detect the first incident light passing through the first focusing part;
The second optical system includes:
a first mirror having a front surface inclined with respect to the rear surface of the window and configured to reflect the second incident light;
first and second optical elements spaced apart from the first mirror and configured to diffract the second incident light;
a second focusing unit spaced apart from the second optical device and configured to focus the diffracted light diffracted by the second optical device; and
and a second image sensor provided on a rear surface of the second focusing part and configured to detect the diffracted light passing through the second focusing part.
12. The method of claim 11,
The first optical element is any one or a combination of two or more selected from a surface diffraction grating, a volume Bragg grating, and at least one or more prisms,
wherein the second optical element is a volume Bragg grating.
12. The method of claim 11,
each of the first and second optical elements is a volume Bragg grating;
The grating period of the first optical element is different from the grating period of the second optical element.
12. The method of claim 11,
The OIS module includes position sensors and OIS driving units connected to the hyperspectral sensor, and an OIS control unit connected to the position sensors and the OIS driving units, respectively,
The OIS control unit is a hyperspectral imaging system configured to receive the position information of the position sensors and transmit it to the OIS drivers.
12. The method of claim 11,
a scan mirror control module connected to the first mirror of the second optical system;
a focusing part control module connected to the first focusing part of the first optical system and the second focusing part of the second optical system; and
Further comprising a light source control module connected to the processor,
The light source control module is a hyperspectral imaging system including an LED driver connected to the processor, and an LED light source connected to the LED driver and knowing a spectrum in advance.
A window configured to receive the first and second incident lights, a first optical system configured to receive the first incident light passing through the window, and a spaced apart from the first optical system to receive the second incident light passing through the window In the hyperspectral sensor comprising a second optical system configured to be,
setting the measurement area;
photographing a reference image by the first optical system;
taking hyperspectral images by the second optical system;
compositing the hyperspectral images;
comparing the reference image with a composite image obtained by synthesizing the hyperspectral images; and
outputting a measurement result,
The first optical system includes:
a first focusing part provided on a rear surface of the window and configured to focus the first incident light; and
a first image sensor provided on a rear surface of the first focusing part and configured to detect the first incident light passing through the first focusing part;
The second optical system includes:
a first mirror having a front surface inclined with respect to the rear surface of the window and configured to reflect the second incident light;
an angle converter for each wavelength that is spaced apart from the first mirror and converts the second incident light to travel at an angle satisfying the Bragg condition for each wavelength;
a second focusing unit spaced apart from the angle converting unit for each wavelength and configured to focus the diffracted light by the angle converting unit for each wavelength; and
Hyperspectral imaging method including a second image sensor provided on a rear surface of the second focusing part and configured to detect the diffracted light passing through the second focusing part
17. The method of claim 16,
Taking the hyperspectral images by the second optical system includes a plurality of shooting operations,
Each of the photographing operations is:
rotating the first mirror;
taking a hyperspectral image by the second image sensor; and
Hyperspectral imaging method comprising correcting distortion of the hyperspectral image.
18. The method of claim 17,
Taking the hyperspectral image by the second image sensor is a hyperspectral imaging method that is performed once with a light source on and once with a light source off.
17. The method of claim 16,
Prior to setting the measurement area, further comprising inputting a measurement mode,
The measurement mode is one selected from a point spectrum measurement mode, a spectrum measurement mode after shape recognition, a spatial low resolution measurement mode, and a spatial high resolution measurement mode.
17. The method of claim 16,
The angle conversion unit for each wavelength is any one or a combination of two or more selected from a surface diffraction grating, a volume Bragg grating, and at least one prism.

Images