KR20180055244A - Mobile spectral imaging system and method - Google Patents

Abstract

According to an embodiment of the present invention, a mobile spectral imaging system comprises: a user terminal including a light source for irradiating an imaging target with light, and an imaging unit for imaging the imaging target; and a spectral imaging device mounted on the user terminal to obtain a spectral image on the imaging target by the light emitted from the light source, and including a probe being in contact with the imaging target. Since a reference target is mounted on the probe, the spectral image on the imaging target and a spectral image on a reference target can be simultaneously obtained.

Description

[0001] MOBILE SPECTRAL IMAGING SYSTEM AND METHOD [0002]

The present invention relates to a mobile spectrographic imaging system and method, and more particularly to a mobile spectral imaging system and method in which a reference target is mounted on a probe so that a spectroscopic image of the object to be imaged and the reference target can be acquired simultaneously.

Spectroscopic imaging and analytical techniques are widely used as a quantitative optical imaging tool in a variety of biomedical fields. These techniques make it possible to obtain a spectral signature at each pixel in the spectral image. As a result, an image cube containing spectral information as well as spatial information can be obtained.

In particular, the spectroscopic information can be applied to quantitatively discriminate the biological samples of interest from backgrounds that can not be distinguished by conventional monochrome and RGB color imaging methods, since different biological components generally reflect discriminable spectroscopic indicators Because.

In spectral analysis, various methods based on distance measurement can be applied to quantitatively and robustly partition the region of interest in the image cube. In addition, spectral classifications such as linear de-combination or principal component analysis have been shown to be able to separate very close spectral indicators.

Spectroscopic imaging and analytical techniques are thus used to noninvasively diagnose various diseases such as gastric cancer, breast cancer, and skin diseases. In particular, it is useful in distinguishing spatially and spectrally heterogeneous skin lesions among the unidentifiable features by utilizing predefined spectral reflectance characteristics during the diagnosis of various malignant and non-malignant skin diseases. For example, spectroscopic imaging and analysis techniques have been found to distinguish malignant melanoma, one of the malignant skin diseases, from dysplastic nevus and pigment cell nevus at an early stage. This method can also be applied to assess the severity of acne lesions by applying a combination of linear discriminant functions. Using spectral imaging and linear discriminant function classifiers, various acne lesions can be successfully extracted and classified.

In addition, spectral imaging and analysis methods in the visible and near infrared wavelength ranges are being used as noninvasive tools useful in edema pathology.

Spectroscopic imaging and analytical techniques are also used for the quantitative diagnosis of skin autoimmune diseases and psoriasis.

In recent years, various user terminals based on smart phones have gained attention in that they can acquire personal healthcare information. As a result, smartphone-based devices have been developed, and in particular, several portable skin diagnostic systems have been developed for the early detection of skin lesions.

However, most devices are based on RGB color imaging as well as conventional image processing techniques. Conventional monochromatic or RGB color imaging is associated with early detection and inspection of skin lesions, Is low. To overcome these limitations, there is a need to develop a more improved mobile skin diagnostic system with high quantitative characteristics.

For example, KR 10-2016-7004141, filed on July 22, 2014, discloses " optical detection of skin diseases ".

An object according to an embodiment is a mobile spectral imaging system and method capable of acquiring calibration information from a spectral image for a reference target, the spectral image for the object being captured and the reference target being simultaneously obtainable by mounting a reference target on the probe, .

An object of the present invention is to provide a method and apparatus capable of generating a reliable spectral indicator irrespective of a preprocessing process of an RGB camera mounted on a smartphone such as an autoblance or an autocontrast, A mobile spectral imaging system and method capable of generating a reliable spectral indicator irrespective of the type of a user terminal and improving interphone repeatability between heterogeneous user terminals.

The object of the embodiment is to improve portability and accuracy and to be useful for skin diagnosis and quantitative management and to provide a quantitative / continuous monitoring of skin diseases at low cost in the home And to provide a mobile spectral imaging system and method that can perform early detection of skin diseases.

An object according to an embodiment is to provide a mobile spectral imaging system and method that can be applied to the detection of other affected areas such as cervical tumors and malignant melanoma in a ubiquitous environment.

An object according to one embodiment is to perform spectral imaging and analysis on normal skin and point regions and to quantitatively diagnose the severity of an acne area by applying a ratio metric spectral analysis technique, The present invention provides a mobile spectral imaging system and method capable of effectively monitoring the therapeutic effect of a drug applied to a patient.

According to an aspect of the present invention, there is provided a mobile spectroscopic imaging system including a user terminal including a light source for irradiating light to an object to be imaged, and a photographing unit for photographing an image of the object to be imaged; And a spectroscopic imaging device, mounted on the user terminal, for acquiring a spectroscopic image of the object to be imaged by the light emitted from the light source, wherein the spectroscopic imaging device includes a probe in contact with the object to be imaged, The probe may be equipped with a reference target, and a spectroscopic image of the object to be imaged and a spectroscopic image of the reference target may be simultaneously obtained.

According to one aspect of the present invention, the probe includes a ring-shaped fixing member, and the fixing member may be formed with a groove for mounting the reference target.

According to one aspect, the reference target may include a plurality of reference compartments, and the plurality of reference compartments may be radially spaced along the inner circumferential surface of the holding member.

According to one aspect, the reference target is provided in a ring shape, and the reference target can be mounted so as to protrude inward from the inner circumferential surface of the fixing member.

According to one aspect of the present invention, there is provided a spectroscopic image processing server for processing a spectroscopic image obtained in the spectroscopic imaging device, wherein the spectroscopic image processing server transmits the spectroscopic image from the user terminal, A color conversion unit for converting the color signal into a color signal; A shading unit for shading the spectral image converted into an achromatic color in the color conversion unit; And a spectroscope correcting unit for correcting the spectroscopic image shaded in the shader.

According to one aspect, in the color conversion unit, the spectral image is converted into an achromatic color by the following equation,

Where R is the intensity of the red pixels in the spectral image,

G is the intensity of the green pixels in the spectral image,

B is the intensity of the blue pixels in the spectral image,

I is the intensity of the spectral image converted to achromatic color.

According to one aspect of the present invention, the spectroscopic image in the spectroscopic correction section is corrected by the following equation,

Where x and y are the x-axis and y-axis coordinates of the region of interest in the spectral image,

lambda is the wavelength of the light measured in the spectral image,

는 x, y 및 λ에서 분광 이미지의 보정된 강도이고,

Is the corrected intensity of the spectral image at x, y and lambda,

는 x, y 및 λ에서 분광 이미지의 원래 강도이고,

Is the original intensity of the spectral image at x, y and lambda,

는 λ에서 분광 이미지의 보정 계수고,

Is the correction coefficient of the spectral image at?

는 λ에서 측정된 분광 이미지 내 참조 영역의 최대 강도이고,

Is the maximum intensity of the reference area in the spectroscopic image measured at lambda,

는 x, y 및 λ에서 측정된 분광 이미지 내 참조 영역의 강도이다.

Is the intensity of the reference area in the spectral image measured at x, y, and lambda.

는 상기 복수 개의 분광 이미지에 각각 포함된 참조 영역의 평균 강도 중 최대값을 특정 파장에서 측정된 분광 이미지에 포함된 참조 영역의 평균 강도로 나눔으로써 산출될 수 있다.According to one aspect, the spectroscopic imaging device obtains a plurality of spectroscopic images for different wavelengths,

Can be calculated by dividing the maximum value of the average intensities of the reference areas included in the plurality of spectral images by the average intensities of the reference areas included in the spectral images measured at a specific wavelength.

According to another aspect of the present invention, there is provided a spectroscopic imaging method including a user terminal, a spectroscopic imaging device mounted on the user terminal to acquire a spectroscopic image of an object to be imaged, and a spectroscopic image processing server The method comprising: providing a spectroscopic imaging system comprising: Obtaining a spectroscopic image of an object to be imaged by the spectroscopic imaging device; Transmitting the obtained spectroscopic image to the spectroscopic image server; Converting the transmitted spectral image into an achromatic color; Wherein the achromatic-converted spectral image is shaded; The shaded spectral image is corrected; And a step in which the corrected spectroscopic image is spectroscopically classified. In the step of obtaining a spectroscopic image of an object to be imaged by the spectroscopic imaging device, at least one region of interest is disposed at the center of the spectroscopic image, A reference region may be disposed outside the region of interest.

According to one aspect, the step of correcting the shaded spectral image includes extracting a spectral index of the reference region and the spectral index of the region of interest, respectively; Calculating a correction coefficient from a spectral index of the extracted reference area; And multiplying a spectral index of the extracted region of interest by the correction coefficient.

According to the mobile spectrographic imaging system and method according to one embodiment, a reference target is mounted on the probe, so that a spectroscopic image for the object to be imaged and the reference target can be acquired at the same time, and correction information can be obtained from the spectroscopic image for the reference target have.

According to the mobile spectral imaging system and method according to one embodiment, it is possible to generate a reliable spectral indicator irrespective of the RGB camera preprocessing process installed in a smartphone such as Autoblance, autocontrast, etc., , It is possible to generate a reliable spectral index irrespective of the type of the user terminal, and it is possible to improve the repeatability (Interphone repeatability) between the heterogeneous user terminals.

According to the mobile spectral imaging system and method according to one embodiment, portability and accuracy are improved, and it is very useful for skin diagnosis and quantitative management. It is also useful for a patient showing symptoms of a disease, Quantitative / continuous monitoring can be performed and early detection of skin diseases can be performed.

The mobile spectral imaging system and method according to one embodiment can be applied to the detection of other lesion areas such as cervical tumors and malignant melanoma in a ubiquitous environment.

According to the mobile spectral imaging system and method according to one embodiment, spectral imaging and analysis can be performed on normal skin and point regions, and quantitative analysis of the severity of the acne area can be performed by applying the ratio quantitative spectral analysis technique And can effectively monitor the therapeutic effects of the drugs applied for acne treatment.

1A is a schematic diagram of a mobile spectroscopic imaging system in accordance with one embodiment, and FIG. 1B is a photographic image of a mobile spectroscopic imaging system in accordance with one embodiment.
Figure 2a is a schematic view of a spectroscopic imaging device, and Figure 2b is a photographic image of a spectroscopic imaging device.
Figure 3A shows the transmission of light to a plurality of optical filters.
Figure 3b shows the intrinsic spectrum of the light source.
4A and 4B show a case where the reference target is provided with a plurality of reference sections.
5A and 5B show a case where the reference target is provided in a ring shape.
6 shows the configuration of the interface circuit.
7 shows a configuration of a spectral image processing server.
8 schematically shows a process in which a spectral image is processed in a spectral image processing server.
Figures 9A and 9B schematically illustrate the process by which a spectral image is corrected.
10 is a flow chart illustrating a spectroscopic imaging method according to one embodiment.
11A to 11C show experimental results on the resolving power of the spectral image.
Figures 12a and 12b show experimental results for reproducibility or repeatability.
Figures 13a and 13b show a comparison between the results obtained by multispectral imaging and analysis.
14A to 14C show spectral classification of a point region.
Figures 15A-15C illustrate ratio quantitative spectral imaging and analysis for acne areas.

Hereinafter, embodiments will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are shown in different drawings. In the following description of the embodiments, detailed description of known functions and configurations incorporated herein will be omitted when it may make the best of an understanding clear.

In describing the components of the embodiment, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected or connected to the other component, Quot; may be "connected," "coupled," or "connected. &Quot;

The components included in any one embodiment and the components including common functions will be described using the same names in other embodiments. Unless otherwise stated, the description of any one embodiment may be applied to other embodiments, and a detailed description thereof will be omitted in the overlapping scope.

FIG. 1A is a schematic view of a mobile spectrographic imaging system according to one embodiment, FIG. 1B is a photographic image of a mobile spectral imaging system according to an embodiment, FIG. 2A is a schematic view of a spectral imaging device, FIG. 3A shows the transmission of light for a plurality of optical filters, FIG. 3B shows the intrinsic spectrum of the light source, FIGS. 4A and 4B show a case where the reference target is provided with a plurality of reference segments, Fig. 5 shows the configuration of the interface circuit, Fig. 7 shows the configuration of the spectral image processing server, Fig. 8 shows the configuration of the spectral image processing server, Figures 9A and 9B schematically illustrate the process by which a spectral image is corrected.

1A and 1B, a mobile spectroscopy imaging system 10 according to one embodiment may include a user terminal 100, a spectroscopic imaging device 200, and a spectroscopic image processing server 300. [

The user terminal 100 may be, for example, a smart phone, or a mobile device that is convenient for a user to carry.

Referring to FIG. 2A, the user terminal 100 may include a light source 102 and a photographing unit 104.

The light source 102 is provided as an LED light source, and can irradiate light to the object to be photographed. The photographing unit 104 is provided, for example, with a CMOS camera, and can photograph an image of an object to be photographed. As described above, the light source 102 and the photographing unit 104 provided in the user terminal 100 can be used for illumination and image acquisition of a region of interest on the skin, respectively.

In the user terminal 100, an application A or an app app can be operated.

The application A controls the operation of the spectroscopic imaging device 200 to perform spectroscopic imaging of skin lesions and to transmit spectroscopic images to the spectroscopic image processing server 300 from the user terminal 100 via either WiFi or LTE communication. And display final images or spectroscopic indicators on the screen of the user terminal 100. [0033] FIG.

The user terminal 100 described above may be equipped with a spectroscopic imaging device 200.

The spectroscopic imaging device 200 may acquire a spectroscopic image of an object to be imaged by the light emitted from the light source 102 of the user terminal 100.

At this time, the spectral image may be provided as a plurality of spectral images obtained by light of different wavelengths.

2A, a spectroscopic imaging device 200 includes a filter wheel 210, a vertical polarimeter 220, a plano concave lens 230, a mirror 240, an ND filter 250, a horizontal polarimeter 260, A magnifying glass 270 may be included.

The filter wheel 210 may include a plurality of optical filters 212.

The plurality of optical filters 212 may be generated by dicing a linear variable filter.

At this time, the filter wheel 210 can be rotated by the servo motor 113 (see Fig. 6) provided in the interface circuit 110 so that one of the plurality of optical filters 212 is aligned with the light source 102 have.

3A, the plurality of optical filters 212 includes a first optical filter 212a, a second optical filter 212b, a third optical filter 212c, a fourth optical filter 212d, a fifth optical filter 212b, An optical filter 212e, a sixth optical filter 212f, a seventh optical filter 212g, an eighth optical filter 212h and a ninth optical filter 212i, It is a matter of course.

At this time, the first optical filter 212a, the second optical filter 212b, the third optical filter 212c, the fourth optical filter 212d, the fifth optical filter 212e, the sixth optical filter 212f, The seventh optical filter 212g, the eighth optical filter 212h and the ninth optical filter 212i may all be provided with the same size, for example, 3.5 mm (width) x 4 mm (height).

The center wavelength of the first optical filter 212a is 440 (FWHM: 9 nm), the center wavelength of the second optical filter 212b is 460 (FWHM: 9 nm), and the center wavelength of the third optical filter 212c (FWHM: 10 nm), the center wavelength of the fourth optical filter 212d is? 515 (FWHM: 12 nm), and the central wavelength of the fifth optical filter 212e is? 550 nm (FWHM: 14 nm), the center wavelength of the sixth optical filter 212f is-585 nm (FWHM: 15 nm), the center wavelength of the seventh optical filter 212g is-620 nm (FWHM: , The center wavelength of the eighth optical filter 212h is -655 nm (FWHM: 18 nm), and the center wavelength of the ninth optical filter 212i is-690 nm (FWHM: 19 nm). However, it is to be understood that the center wavelength of the optical filter 212 is not limited to this, and can be freely selected depending on the dicing region or shape of the linear filter.

The plurality of optical filters 212 can transmit light of different wavelengths within a wavelength range of 440 nm to 690 nm and can use a single light source 102 to measure a wavelength range of 440 nm to 690 nm It is possible to obtain nine spectral images continuously. More than nine spectral images can be obtained with additional optical filters.

Referring to FIG. 3B, the intrinsic spectrum of the light source 102 provided in the user terminal 100 is as follows.

It can be seen that the variation of the normalized intensity along the wavelength is continuously generated, for example, when the wavelengths are 440 nm, 460 nm, and 554 nm, the inflection point for the normalized intensity variation is formed.

Light transmitted through one of the plurality of optical filters 212 described above may be transmitted to the vertical polarimeter 220.

The vertical polarimeter 220 can polarize light in a vertical direction.

Further, a plane concave lens 230 may be optically coupled to the vertical polarimeter 220.

The plane concave lens 230 can diverge light emitted from the light source 102, in particular, light vertically polarized in the vertical polarimeter 220, with respect to the object to be imaged.

At this time, the subject to be photographed may be the skin S. However, the subject to be photographed is not limited to this, and it can be anywhere in a body part where a lesion diagnosis is required.

Further, a mirror 240 may be further disposed between the plane concave lens 230 and the object to be photographed so that the plane concave lens 230 can diverge to the object to be imaged. The mirror 240 may cause the light transmitted to the flat concave lens 230 to be reflected to the object to be imaged.

The flat concave lens 230 and the mirror 240 can provide wide illumination on the object to be imaged, particularly the skin S. [ At this time, the light transmitted on the skin S may interact with the skin to be reflected or scattered.

The ND filter 250 is spaced apart from the object to be photographed, and can receive light reflected from the object.

A horizontal polarimeter 260 may be optically coupled to the ND filter 250.

The horizontal polarimeter 260 can polarize the light in the horizontal direction and reduce the spectral reflection light from the object to be picked, thereby selectively collecting the light scattered from the object.

The magnifying glass 270 may be optically coupled to the horizontal polarimeter 260.

The magnifying glass 270 may be, for example, 10x.

The light collected in the magnifying glass 270 may be recorded in the photographing unit 104 of the user terminal 100.

At this time, the optical paths in the spectroscopic imaging device 200 are formed parallel to each other, so that the spectroscopic imaging device 200 can be downsized, and the distance between the photographing part 104 of the user terminal 100 and the object to be photographed is, And the focal length may vary depending on the type of the magnifying glass 270. [

In particular, with reference to Figure 2B, the spectroscopic imaging device 200 may comprise a probe P in contact with an object to be imaged.

The probe P may be mounted with a reference target WT.

It is a matter of course that the reference target WT may be provided with, for example, a white target, and targets of different colors may also be used as a reference target. The spectral index or intensity of the spectral image for the reference target WT can be utilized to determine the correction factor upon correction of the spectral image.

At this time, since the reference target WT is mounted on the probe P, since the reference target WT is always positioned within the field of view of the spectroscopic imaging device 200, the spectroscopic image for the reference target Can be obtained simultaneously. In other words, the spectroscopic image for the object to be photographed and the spectroscopic image for the reference target can be simultaneously obtained through one shot.

Also, the spectroscopic imaging device 200 may be surrounded by a light-tight enclosure. Whereby light from the surrounding environment can be blocked when the probe P contacts the skin to obtain a spectroscopic image for the skin.

4A and 4B, the probe P may include a fixing member 202 provided in a ring shape. At this time, a groove (not shown) may be formed in the fixing member 202 to mount the reference target WT.

Specifically, the reference target WT may include a plurality of reference segments WT1, WT2, WT3, WT4. The plurality of reference compartments WT1, WT2, WT3 and WT4 may be mounted in grooves (not shown) formed in the inner circumferential surface of the fixing member 202 and spaced apart in the radial direction along the inner circumferential surface of the fixing member 202 .

A portion of the light from the light source 102 of the user terminal 100 is illuminated toward the skin and the light from the light source 102 of the user terminal 100 And the remaining part can be illuminated toward the reference target WT.

At the center of the spectroscopic image obtained by the spectroscopic imaging device 200, a spectroscopic image of the skin appears, and at the outside of the spectroscopic image of the skin a spectroscopic image of the reference target WT may appear. At this time, at least one region of interest is placed in the spectroscopic image for the skin, and the spectroscopic image for the reference target (WT) can be arranged in the reference region.

In particular, since the reference target WT comprises a plurality of reference compartments WT1, WT2, WT3, WT4, the spectral image for the reference target WT in the spectral image acquired by the spectral imaging device 200 intermittently Can be displayed.

5A and 5B, the reference target WT may be provided in a ring shape of various shapes. At this time, a groove 2022 may be formed in the fixing member 202 to mount the reference target WT.

Concretely, the reference target WT can be projected inward from the inner circumferential surface of the fixing member 202 by engaging the ring-shaped fixing member 202 and the ring-shaped reference target WT.

A portion of the light from the light source 102 of the user terminal 100 is illuminated toward the skin and the light from the light source 102 of the user terminal 100 And the remaining part can be illuminated toward the reference target WT.

At the center of the spectroscopic image obtained by the spectroscopic imaging device 200, a spectroscopic image of the skin appears, and at the outside of the spectroscopic image of the skin a spectroscopic image of the reference target WT may appear. At this time, at least one region of interest is placed in the spectroscopic image for the skin, and the spectroscopic image for the reference target (WT) can be arranged in the reference region.

In particular, since the reference target WT is provided in various ring shapes, the spectral image for the reference target WT can be variously displayed according to the shape of the ring, such as a circle, a square, etc., outside the spectral image for the skin.

1A and 6, the user terminal 100 and the spectroscopic imaging device 200 may be connected by an interface circuit 110. [

The interface circuit 110 synchronizes the user terminal 100 and the spectroscopic imaging device 200 and controls the drive of the filter wheel 210 (see Figure 2a).

Specifically, the interface circuit 110 includes a microcontroller unit 111, a Bluetooth low-energy module 112, a servo motor 113, a battery 114, a step-up regulator 115, and a low- Drop-out adjuster 116. The drop-

The MCU 111 receives a command signal from the user terminal 100 through the BLE communication by the Bluetooth low energy module 112 and transmits the command signal through the ART communication for rotation of the filter wheel 210 driven by the motor. And transmits a command signal to the motor 113.

Also, the battery 114 may be provided with a 3.7 V lithium-polymer battery and is used to supply power to the interface circuit 110.

The step-up regulator 115 and the low-dropout regulator 116 are for adjusting the voltage. The step-up regulator 115 is a filter wheel (not shown) mounted on the servomotor 113 And the low-dropout regulator 116 provides a positive voltage of 7.4 V to the servomotor 113 to control the input voltage To 3.3V.

Referring again to FIG. 1, the user terminal 100 may be connected to the spectral image processing server 300 by wired / wireless communication.

The spectral image processing server 300 is constructed for spectral classification and data storage of spectral images obtained at the spectral imaging device 200.

At this time, the spectroscopic images obtained in the spectroscopic imaging device 200 can be processed in advance to effectively classify the spectroscopic images obtained in the spectroscopic imaging device 200. [

7, the spectral image processing server 300 may include a color conversion unit 310, a shading unit 320, a spectral correction unit 330, and a spectral classification unit 340.

The color conversion unit 310 may receive the spectroscopic image transmitted from the user terminal 100 and may convert the spectroscopic image into an achromatic color. The shading unit 320 may perform a shading process on the spectral image converted into the achromatic color by the color conversion unit 310. The spectroscope correcting unit 330 may correct the spectroscopic image shaded in the shader 320.

Specifically, with reference to Figures 1 and 8, an application (A) has been developed for analysis of skin lesions and spectral imaging.

The application A is driven in the user terminal 100 and controls one white-light image for the reference target and the other white-light image for the 440 nm to 690 nm range by controlling the filter wheel through the interface circuit 110 under the Bluetooth connection. It is possible to obtain nine spectral images (or image cubes) at successive wavelengths.

Is selected from the thus obtained spectroscopic images, and is transmitted to the spectral image processing server 300 through WiFi or LTE communication. After receiving the selected spectral image, the spectral image processing server 300 sequentially performs classification processes using achromatic transformation, shading and correction, normalization, and SAM (spectral angle measure). That is, a spectral image can be processed through an achromatic transformation, shading, correction, and normalization processes before a classification process using a spectral angle measure (SAM).

For example, the spectral image may be provided as an RGB color image, and achromatic conversion may be performed in the color conversion unit 310 by the following equation.

Where R is the intensity of the red pixels in the spectral image,

G is the intensity of the green pixels in the spectral image,

B is the intensity of the blue pixels in the spectral image,

I is the intensity of the spectral image converted to achromatic color.

Shading and correction are intended to reduce artifacts caused by nonuniform illumination on the skin. In the shading unit 320 and spectral correction unit 330, shading and correction can be performed by the following equation have.

Where x and y are the x-axis and y-axis coordinates of the region of interest in the spectral image,

lambda is the wavelength of the light measured in the spectral image,

는 x, y 및 λ에서 분광 이미지의 보정된 강도이고,

Is the corrected intensity of the spectral image at x, y and lambda,

는 x, y 및 λ에서 분광 이미지의 원래 강도이고,

Is the original intensity of the spectral image at x, y and lambda,

는 λ에서 분광 이미지의 보정 계수고,

Is the correction coefficient of the spectral image at?

는 λ에서 측정된 분광 이미지 내 참조 영역의 최대 강도이고,

Is the maximum intensity of the reference area in the spectroscopic image measured at lambda,

는 x, y 및 λ에서 측정된 분광 이미지 내 참조 영역의 강도이다.

Is the intensity of the reference area in the spectral image measured at x, y, and lambda.

및

는 프로브에 장착된 참조 타겟에 의해 촬영된 참조 영역과 관련된 것으로서, 프로브에 참조 타겟이 장착됨으로써 한 번의 촬영을 통해 분광 이미지의 강도 및 참조 영역의 강도를 용이하게 측정할 수 있다.Especially,

And

Is associated with a reference area imaged by a reference target mounted on the probe. By mounting the reference target on the probe, the intensity of the spectral image and the intensity of the reference area can be easily measured through one shot.

는 복수 개의 분광 이미지에 각각 포함된 참조 영역의 평균 강도 중 최대값을 특정 파장에서 측정된 분광 이미지에 포함된 참조 영역의 평균 강도로 나눔으로써 산출될 수 있다.Also,

Can be calculated by dividing the maximum value of the average intensities of the reference regions included in the plurality of spectral images by the average intensity of the reference region included in the spectral image measured at a specific wavelength.

9A and 9B, one reference region R1 and a plurality of regions of interest I1 and I2 may appear in a plurality of spectral images IC. At this time, one reference area R1 is an area photographed by the reference target WT, and a plurality of areas of interest I1 and I2 are areas photographed by normal skin.

First, in order to correct the spectral image, the spectral index of the reference target is extracted.

At this time, the spectral index of the reference target can be represented by intensity, and the intensity of the reference target according to each wavelength can be extracted.

Then, correction coefficients for each wavelength are extracted.

에 대응되는 것으로서, 복수 개의 분광 이미지에 각각 포함된 참조 영역의 평균 강도 중 최대값을 특정 파장에서 측정된 분광 이미지에 포함된 참조 영역의 평균 강도로 나눔으로써 산출될 수 있다.At this time, the correction coefficients for the respective wavelengths

, And can be calculated by dividing the maximum value among the average intensities of the reference regions included in the plurality of spectral images by the average intensity of the reference region included in the spectral image measured at a specific wavelength.

For example, if the average intensity of the reference region included in the first spectral image is 100, the average intensity of the reference region included in the second spectral image is 150, and the average intensity of the reference region included in the third spectral image is 50 , The maximum value is 150.

Here, the correction coefficient for the first wavelength is 1.5, which is obtained by dividing 150 by the average intensity of the reference region included in the first spectral image, 100, the correction coefficient for the second wavelength is 1, and the correction coefficient for the third wavelength is 150 divided by the average intensity of the reference region included in the third spectral image,

Then, when the average intensity of the reference region included in the spectral image measured at a specific wavelength is multiplied by a correction coefficient, the intensity of the spectral image at the entire wavelength can be made equal to 150. [

Thus, the correction coefficient is extracted from the spectral index for the reference target WT, and the spectral image for the measurement target, in particular, the spectral index for the measurement target can be corrected using the extracted correction coefficient.

For example, when a spectral index for normal skin and point is selected, a corrected spectral index can be obtained by multiplying each spectral index by a correction coefficient.

Specifically, since the correction coefficient for the first wavelength is 1.5, the spectral indexes for the normal skin and point measured at the first wavelength are multiplied by 1.5, and the correction coefficient for the third wavelength is 3, The spectral indices for the normal skin and point measured can be multiplied by 3, respectively.

Corrected spectral indices of skin regions can thus be obtained in all pixels included in the spectral image. And these calibration procedures may be performed each time spectral imaging is performed to obtain a corrected spectral index at every pixel included in the spectral image.

After this preprocessing is performed, spectroscopic classification of the spectroscopic image can be performed in the spectroscopic classification unit 340 by the predefined reference spectroscopic indicators.

Spectroscopic angular measurement (SAM) can be utilized for spectroscopic classification.

In the SAM, the spectroscopic indices measured at all pixels and the spectroscopic angles between the reference spectroscopic indices can be calculated by the following equation.

Where s _i is the spectral index of the i-th pixel, μ _j is the j-th reference spectral index, and L is the number of spectral images.

After the measurement of the spectroscopic angles, a predefined color is assigned to the corresponding pixels for the reference indicator having the measured spectroscopic index and the minimum angle.

Finally, the spectrally classified image is delivered to the application A of the user terminal 100, and the final classified image and the corresponding spectral indicators used for this classification are displayed in the application A.

A mobile spectroscopic imaging system according to one embodiment has been described, and a spectroscopic imaging method according to one embodiment will be described below.

10 is a flow chart illustrating a spectroscopic imaging method according to one embodiment.

Referring to Fig. 10, spectroscopic imaging can be performed as follows.

First, a spectroscopic imaging system is provided that includes a user terminal, a spectroscopic imaging device mounted on the user terminal to acquire a spectroscopic image of an object to be imaged, and a spectroscopic image processing server for processing the obtained spectroscopic image.

At this time, the spectroscopic imaging system is the same as the mobile spectroscopic imaging system according to one embodiment.

Subsequently, a spectroscopic image for the object to be imaged is obtained by the spectroscopic imaging device (S20).

At this time, the spectral image is a plurality of spectral images obtained by light of different wavelengths, at least one region of interest is arranged in the center of the spectral image obtained in the spectral imaging device, and a reference region is arranged outside the region of interest .

This is because the reference target is mounted on the probe of the spectroscopic imaging device as described above, and the captured image becomes the reference region, and the image of the object to be photographed, for example, the skin, .

Subsequently, the obtained spectroscopic image is transmitted to the spectroscopic image server (S30).

This is for processing or classifying the spectral image obtained in the spectral imaging device.

In particular, the spectral image can be processed or preprocessed as follows.

The transmitted spectroscopic image is converted into an achromatic color (S40), the spectroscopic image converted into achromatic color is shaded (S50), and the shaded spectroscopic image is corrected (S60).

In particular, correction of the spectral image can be performed as follows.

(S61), a correction coefficient is calculated from a spectral index of the extracted reference area (S62), and the correction coefficient is added to the spectral index of the extracted ROI (S63).

At this time, a plurality of spectral images can be separately processed.

Then, the corrected spectral image is spectrally classified (S70).

The spectral classification may be made by the SAM method as described above and the spectrally classified image or spectral classification result may be delivered to application A and stored and displayed in application A.

Thus, a mobile spectroscopic imaging system and method according to one embodiment can acquire a spectroscopic image for an object to be imaged and a reference target simultaneously, acquire correction information from a spectroscopic image for a reference target, and can acquire correction information such as Autoblance, autocontrast, It is possible to generate a reliable spectral index irrespective of the preprocessing process of the RGB camera mounted on the smartphone and generate a constant spectral index irrespective of the intensity of light from the light source (for example, LED intensity) And it is possible to improve the repeatability (Interphone repeatability) between heterogeneous user terminals.

In addition, it is possible to carry out quantitative / continuous monitoring of skin diseases at home at a low cost without going to a hospital, which is useful for diagnosis and quantitative management of skin, Early detection can be performed.

Hereinafter, the results of experiments conducted by applying the mobile spectral imaging system and method according to one embodiment will be described.

Figs. 11A to 11C show experimental results on the resolving power of a spectral image, Figs. 12A and 12B show experimental results on reproducibility or repeatability, Figs. 13A and 13B show results of spectral imaging and analysis, Figs. 14A to 14C show spectral classification of a point region, and Figs. 15A to 15C show ratio quantitative spectral imaging and analysis for acne areas.

11A to 11C, the performance (resolution) evaluation results of the mobile spectral imaging system according to one embodiment are as follows.

To test the optical resolution of a mobile spectroscopic imaging system according to one embodiment, the highest set of frequency lines that can be decomposed is measured.

In particular, the profiles of the intensities along the vertical and horizontal solid lines are shown in Figure 11A, with the image of the resolution target being shown in Figures 11b and 11c, with a frequency line set of 51 indicating 51 cycles / mm.

In the intensity profiles, the lines in the vertical and horizontal directions are clearly distinguished. A total of five lines can be distinguished in the vertical and horizontal directions in a set of 51 frequencies. Thus, these results indicate that the system can dissolve 51 cycles per millimeter.

Referring to Figures 12A and 12B, experimental results for reproducibility or repeatability of a mobile spectral imaging system according to one embodiment are as follows.

To this end, a total of 15 spectral indicators for the reference target are repeatedly obtained with fixed and automatic white balance settings, and then the mean spectral indices and their standard deviations are compared with the resultant values obtained using the spectroscope.

In particular, Figures 12a and 12b illustrate average spectral indices when obtained with a spectroscope and with fixed and automatic white balance settings in a mobile spectral imaging system according to one embodiment.

It can be seen that the average spectral indices for the reference target obtained using the mobile spectroscopic imaging system according to one embodiment exhibit similar intensities at designated wavelengths. In addition, the profiles of the average spectroscopic indices obtained using the mobile spectroscopic imaging system according to one embodiment are very similar to those of the average spectroscopic indices obtained using the spectroscope. Which indicates that it exhibits excellent reproducibility or repeatability in the construction of spectroscopic indicators by using a mobile spectroscopic imaging system according to one embodiment.

Referring to Figures 13a and 13b, the results obtained using a mobile spectral imaging system and a liquid-crystal tunable filter (LCTF) -based spectral imaging system according to one embodiment are compared so that the performance of a mobile spectral imaging system according to one embodiment Can be evaluated.

An LCTF-based multispectral imaging system includes an objective lens, a liquid crystal tuning filter, twelve high intensity white light emitting diodes, and a monochrome CMOS camera.

At this time, the light emitted from the LEDs is illuminated to the object to be imaged after passing through the polarizer. The light reflected from the object to be photographed is collected by the objective lens and passes through the liquid crystal adjustment filter. Finally, it is finally recorded by the CMOS camera.

For performance comparisons, the three letters 'M', 'S', and 'I' are printed on white paper in blue, green, and green, respectively. Spectral imaging of characters is performed to obtain character images at designated wavelengths, and spectral classification of spectral images is performed using the developed program. At the designated wavelengths, 'M' and 'S' of the character images showed the highest intensities at 440 nm and 515 nm, while 'I' showed higher intensities above 620 nm and were not distinguished from white paper.

Thus, the spectrally classified images obtained using the mobile spectrographic imaging system according to one embodiment were similar to the spectrally classified images obtained using the liquid-crystal tunable filter (LCTF) based spectral imaging system.

However, many non-classified pixels have been found in spectrally classified images obtained using a liquid-crystal tunable filter (LCTF) -based spectral imaging system. In the 'M' classification, the spectrally divided images obtained using a liquid-crystal tunable filter (LCTF) based spectroscopic imaging system exhibited undifferentiated rates within 11.5%, while using a mobile spectroscopic imaging system according to one embodiment The obtained spectroscopic images showed only 0.07% undifferentiated ratio.

Also, in the 'S' classification, the spectrally divided images obtained using a liquid-crystal tunable filter (LCTF) based spectral imaging system exhibit undifferentiated ratios within 0.8%, whereas the mobile spectral imaging system according to one embodiment The spectral classification images obtained using this method showed 0% undifferentiated ratio.

Therefore, these results indicate that a spectral imaging system according to one embodiment during spectral imaging and analysis provides much better or similar performance than a liquid-crystal tunable filter (LCTF) based spectral imaging system.

However, liquid-crystal tunable filter (LCTF) -based spectroscopic imaging systems have the disadvantage that they require relatively bulky, expensive, high-intensity illumination.

14A to 14C, a mobile spectroscopic imaging system according to an embodiment can perform spectral imaging and analysis of a point region.

Previously, the point regions could be almost distinguished from the cases of malignant melanoma at the early stage. For spectral imaging and analysis of the point region, nine point images were acquired at different wavelengths (440, 460, 480, 515, 550, 585, 620, 655, and 690 nm) using the SpectroVision system. Images are transmitted to a spectral image processing server, and spectral classification is performed by reference spectral indicators for point and normal skin.

In particular, FIG. 14A is the original RGB point image, FIG. 14B is the shaded image, and FIG. 14C shows the spectrally classified image and the normalized reference spectral indicators for the point and normal skin.

At this time, the spectral index for normal skin has a higher intensity in the range of 460 to 585 nm compared to the spectral index for point.

Also, in the spectrally classified image, green represents normal skin, while red represents point region, and point region is clearly separated from normal regions.

As such, the mobile spectroscopic imaging system according to one embodiment can acquire spectrally classified images even for the point region.

15A to 15C, the experimental results for the ratio quantitative spectral imaging and analysis on acne areas are as follows.

At this time, the mobile spectroscopic imaging system according to one embodiment can be applied to the ratio metrology spectroscopy technique and can be quantitatively monitored for the acne area.

In particular, Figures 15A and 15B are conventional spectral-classified images, and Figure 15C shows a ratio metric spectral-classified image and corresponding spectral indicators.

In the existing spectroscopically categorized image, red refers to acne areas, and green refers to normal skin areas.

On the other hand, in the ratio quantitative spectroscopic image, red and green respectively show severe acne and normal skin, respectively. And blue, yellow, and orange represent severity levels of acne areas for normal areas.

These results indicate that quantitative spectral imaging and analysis can be used to quantitatively and precisely diagnose areas of acne and effectively monitor the therapeutic effects of drugs applied to areas of acne.

Although the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, And various modifications and changes may be made thereto without departing from the scope of the present invention. Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

10: Mobile spectral imaging system
100: User terminal
102: Light source
104:
200: spectroscopic imaging device
210: Filter wheel
220: vertical polarimeter
230: flat concave lens
240: mirror
250: ND filter
260: Horizontal polarimeter
270: Magnifying glass
P: Probe
WT: reference target
300: Spectral image processing server
310: Color conversion unit
320: Shader
330: spectroscopic correction unit
340:

Claims (10)

A user terminal comprising: a light source for irradiating light to a subject to be photographed; and a photographing unit for photographing an image of the subject to be photographed; And
A spectroscopic imaging device mounted on the user terminal for acquiring a spectroscopic image of the object to be imaged by the light emitted from the light source;
Lt; / RTI >
Wherein the spectroscopic imaging device includes a probe in contact with the object to be imaged and the probe is equipped with a reference target to obtain a spectroscopic image for the object to be imaged and a spectroscopic image for the reference target simultaneously.
The method according to claim 1,
Wherein the probe comprises:
A fixing member provided in a ring shape;
Lt; / RTI >
Wherein the fixing member has a groove formed therein for mounting the reference target.
3. The method of claim 2,
Wherein the reference target includes a plurality of reference segments,
Wherein the plurality of reference segments are spaced apart in a radial direction along an inner circumferential surface of the securing member.
3. The method of claim 2,
Wherein the reference target is provided in a ring shape,
Wherein the reference target is mounted so as to protrude inward from an inner circumferential surface of the fixing member.
The method according to claim 1,
A spectral image processing server for processing the spectral image obtained in the spectral imaging device;
Further comprising:
Wherein the spectral image processing server comprises:
A color conversion unit for transmitting the spectroscopic image from the user terminal and converting the spectroscopic image into an achromatic color;
A shading unit for shading the spectral image converted into an achromatic color in the color conversion unit; And
A spectral correcting unit for correcting a spectral image shaded by the shading unit;
/ RTI >
6. The method of claim 5,
Wherein the spectral image in the color conversion unit is converted into an achromatic color by the following equation,

Where R is the intensity of the red pixels in the spectral image,
G is the intensity of the green pixels in the spectral image,
B is the intensity of the blue pixels in the spectral image,
I is the intensity of the spectral image converted to an achromatic color.
6. The method of claim 5,
The spectroscopic image in the spectroscopic correcting unit is corrected by the following equation,

Where x and y are the x-axis and y-axis coordinates of the region of interest in the spectral image,
lambda is the wavelength of the light measured in the spectral image,

Is the corrected intensity of the spectral image at x, y and lambda,

Is the original intensity of the spectral image at x, y and lambda,

Is the correction coefficient of the spectral image at?

Is the maximum intensity of the reference area in the spectroscopic image measured at lambda,

Is the intensity of the reference area in the spectral image measured at x, y and lambda.
8. The method of claim 7,
The spectroscopic imaging device obtains a plurality of spectroscopic images for different wavelengths,
remind

Is calculated by dividing the maximum value of the average intensities of the reference regions included in the plurality of spectral images by the average intensity of the reference region included in the spectral image measured at a specific wavelength.
A spectroscopic imaging system comprising a user terminal, a spectroscopic imaging device mounted on the user terminal for acquiring a spectroscopic image of an object to be imaged, and a spectroscopic image processing server for processing the obtained spectroscopic image;
Obtaining a spectroscopic image of an object to be imaged by the spectroscopic imaging device;
Transmitting the obtained spectroscopic image to the spectroscopic image server;
Converting the transmitted spectral image into an achromatic color;
Wherein the achromatic-converted spectral image is shaded;
The shaded spectral image is corrected; And
Wherein the corrected spectral image is spectrally classified;
Lt; / RTI >
Wherein at least one region of interest is located in the center of the spectral image and a reference region is located outside of the region of interest, wherein the spectral image is acquired by the spectral imaging device.
10. The method of claim 9,
Wherein the shaded spectral image is corrected comprises:
Extracting the spectral indexes of the reference region and the ROI;
Calculating a correction coefficient from a spectral index of the extracted reference area; And
Multiplying a spectral index of the extracted region of interest by the correction coefficient;
/ RTI >

Images