KR20240054725A - Complex type skin analysis device - Google Patents

Abstract

The present invention relates to a skin analysis device that can easily perform skin analysis using multispectral images while increasing the accuracy of analysis by using both visible light and near-infrared rays. Multiple images taken by irradiating light with different wavelengths are An analysis device for analyzing skin using a visible light source that irradiates visible light of different wavelengths in the visible light range; A visible light camera that captures visible light reflected from a visible light source onto the skin; A near-infrared light source that irradiates near-infrared rays of one or more wavelengths in the near-infrared region; A near-infrared camera that photographs the reflection of near-infrared rays emitted from a near-infrared light source onto the skin; and a control unit that controls the device and processes images.
The present invention has the effect of improving the accuracy of skin analysis by making it possible to easily obtain images for skin analysis through an easy-to-use structure and at the same time using both visible light and near-infrared rays to increase analysis efficiency for images in low wavelength ranges.

Description

Complex skin analysis device {COMPLEX TYPE SKIN ANALYSIS DEVICE}

The present invention relates to a device for performing various skin analyzes using digital images, and more specifically, to a device for analyzing skin using digital images taken using visible light and near-infrared rays.

Currently, the incidence of various skin diseases is steadily increasing worldwide due to various factors such as changes in diet, environmental pollution, and excessive exposure to ultraviolet rays. Because various skin diseases can have a significant impact on a patient's life, both mentally and physically, even if the symptoms are mild, interest is growing from the perspective of treatment that goes beyond the scope of simple cosmetics. In order to improve the effectiveness of treatment and prevent serious damage caused by skin diseases, it is very important to diagnose skin diseases accurately and preemptively at an early stage, and various diagnostic methods and devices are being researched.

A multi-spectral image refers to an image of multiple spectral bands with different wavelength ranges, and necessary data can be extracted using images taken in various wavelength bands. Since it is possible to obtain information about targets such as tissues non-invasively by analyzing these multispectral images, it is useful for diagnosing and analyzing various characteristics of the target, and can also be used for diagnosing skin diseases. .

Multispectral imaging for skin diagnosis must be performed while light with a specific wavelength is irradiated to the skin. In this process, multiple optical filters are applied to one light source to irradiate light with different wavelengths whenever necessary. A lighting device for taking multispectral images has also been developed.

However, there is a problem in that information such as wrinkles or depth of the skin, thickness of the lesion area, or blood flow is not reflected in the analysis due to the nature of the light in the wavelength range used in existing multispectral images not to penetrate the depth of the skin. This may be due to the visible light camera used to acquire the image as well as the light source.

Additionally, when using a plurality of filters for one light source, there is a problem of not being able to radiate light of the correct wavelength due to the limitations of the filters. In addition, since it takes time to replace the optical filter, there is a disadvantage that the time for taking multispectral images is too long. To solve this problem, if the number of images in the multispectral image, that is, the number of wavelength bands, is reduced, the analysis effect is lowered.

Therefore, there is a growing demand for devices that can easily perform analysis using multispectral images and increase analysis efficiency at the same time.

Republic of Korea registered patent 10-1922137

The present invention is intended to solve the problems of the prior art described above, and its purpose is to provide a device that can easily perform skin analysis using multispectral images while improving the accuracy of analysis by using both visible light and near-infrared rays.

The complex skin analysis device according to the present invention to achieve the above object is an analysis device that analyzes the skin using a plurality of images taken by irradiating light with different wavelengths, and is an analysis device with different wavelengths in the visible light range. A visible light source that irradiates visible light; A visible light camera that captures visible light reflected from a visible light source onto the skin; A near-infrared light source that irradiates near-infrared rays of one or more wavelengths in the near-infrared region; A near-infrared camera that photographs the reflection of near-infrared rays emitted from a near-infrared light source onto the skin; and a control unit that controls the device and processes images.

The visible light source may be composed of light sources capable of irradiating visible light having three or more wavelengths in the range of 365 to 700 nm.

The near-infrared light source may be configured as a light source capable of irradiating near-infrared rays having one or more wavelengths in the range of 600 to 1400 nm.

The visible light source consists of light sources capable of irradiating visible light having three or more wavelengths in the range of 365 to 700 nm, and the near-infrared light source is capable of irradiating near infrared rays having two or more wavelengths in the range of 600 to 1400 nm. It is composed of light sources, and the visible light source and the near-infrared light source may include one or more light sources that irradiate light in the range of 600 to 700 nm.

At this time, the visible light spectral image obtained from the visible light camera and the NIR image obtained from the near-infrared camera are standardized and used. Specifically, the NIR image taken in the 600~700nm range is corrected to match the visible light spectral image taken in the 600~700nm range. , Visible light spectral images and NIR images can be standardized by correcting NIR images taken in different ranges to those in which NIR images taken in the 600-700 nm range have been corrected.

In addition, it is desirable to standardize the visible light spectral image obtained from a visible light camera and the NIR image obtained from a near-infrared camera, and to perform skin analysis using the standardized visible light spectral image and NIR image together.

It is preferable that the control unit matches the image area of the visible light spectral image captured by the visible light camera with the image area of the NIR image captured by the near-infrared camera.

Because it is equipped with two types of cameras, there are differences in the areas captured by each camera, but they can be used together for skin analysis by matching the image areas in the control unit.

The near-infrared light source and the near-infrared camera may be located inside the device or may be installed on a near-infrared attachment configured to be externally attachable and detachable.

By placing the near-infrared light source and near-infrared camera on an attachment that can be attached and detached from the main body rather than installing it inside the main body of the device, the existing skin analysis device containing only a visible light source and a visible light camera is used as is, and the near-infrared area is strengthened. A type skin analysis device can be configured.

Images captured by a near-infrared camera reflect information such as skin depth direction and wrinkles, and in addition, information on blood flow volume or blood flow speed can be additionally obtained and used for analysis.

Using near-infrared rays, it is possible to obtain information about the amount or speed of blood flow inside the skin, and by applying this to skin analysis, the accuracy of the analysis can be improved.

It further includes a grip part that the user can hold and use, two control buttons installed on both ends of the grip part, and a sensor to detect the posture of the device, so that only the control button located relatively above the two control buttons operates. It can be controlled.

Two operation buttons are installed on both ends of the grip section, allowing users to select and use one of the two operation buttons located on both ends of the grip section depending on the shooting direction. To prevent malfunction, only one operation button is used depending on the posture of the device. It can be activated, and the sensor that detects the posture of the device can be a G-sensor or a horizontal sensor.

The present invention, constructed as described above, can easily obtain images for skin analysis through an easy-to-use structure and at the same time uses visible light and near-infrared rays together, so that the information about the skin surface obtained by visible light can be combined with the skin depth obtained by near-infrared rays. By adding information, the efficiency of image analysis is increased, which has the effect of improving the accuracy of skin analysis.

Accordingly, there is an effect of increasing the accuracy of skin analysis and providing skin analysis results that are differentiated from existing ones.

Figure 1 is an exploded perspective view showing the structure of a complex skin analysis device according to the first embodiment of the present invention.
Figure 2 is an exploded perspective view showing the structure of a complex skin analysis device according to a second embodiment of the present invention.
Figure 3 is a diagram for explaining the near-infrared attachment applied to the complex skin analysis device according to the second embodiment of the present invention.
Figure 4 is a flowchart for explaining the operation process of the complex skin analysis device according to an embodiment of the present invention.
Figure 5 is a diagram illustrating the process of standardizing visible light spectral images and NIR images in the complex skin analysis device according to an embodiment of the present invention.

Embodiments according to the present invention will be described in detail with reference to the attached drawings.

However, the embodiments of the present invention may be modified into various other forms, and the scope of the present invention is not limited to the embodiments described below. The shapes and sizes of elements in the drawings may be exaggerated for clearer explanation, and elements indicated by the same symbol in the drawings are the same element.

And throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected," but also the case where it is "electrically connected" with another element in between. In addition, when a part is said to "include" or "have" a certain component, this does not mean excluding other components, unless specifically stated to the contrary, but rather means that it can further include or be provided with other components. .

Additionally, terms such as “first” and “second” are used to distinguish one component from another component, and the scope of rights should not be limited by these terms. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

Figure 1 is an exploded perspective view showing the structure of a complex skin analysis device according to the first embodiment of the present invention.

The complex skin analysis device of this embodiment includes a main body 100, a circuit board 200, an upper cover 300, a light source unit 400, a lower cover 500, and a cradle 700.

The main body 100 is a component for installing components such as the circuit board 200 inside, and in this embodiment, it is composed of an empty case divided into two.

The complex skin analysis device of this embodiment includes a grip portion with a cylindrical central portion so that the user can hold it in the hand and use it, and has two operation buttons (110, 120) at different heights at both ends of the grip portion. It is installed so that it is exposed to the outside. As will be explained in detail later, the complex skin analysis device of this embodiment is used in such a way that the user holds the main body 100 in his hand and performs imaging by placing one end in close contact with the skin. At this time, the device is The direction of arrangement may be in the form shown depending on the shooting direction, or in the case of shooting in an upward direction, it may be arranged in the opposite direction from what is shown. At this time, if the operation buttons 110 and 120 for operating the device are installed in only one place, it may be difficult to operate the button depending on the shooting direction. Therefore, in this embodiment, the operation of the button is performed using the finger regardless of the direction in which the device is held. Two operation buttons (110, 120) were installed at different heights so that the operation buttons (110, 120) could be operated. In this specification, the direction is described based on the case of shooting downward according to the direction shown in the drawing, but the shooting direction is not limited to downward. In addition, the two operation buttons (110, 120) arranged at different heights may be operated at all times, but in order to prevent malfunction, only one of the two operation buttons (110, 120) may be controlled to operate. For example, regardless of the shooting direction, it can be controlled so that only the operation buttons located relatively above operate and the operation buttons located relatively below do not operate, and the control of these operation buttons can be controlled by the gravity sensor installed in the device. A horizontal sensor can detect the device's posture and perform this automatically. As specifically shown, when the shooting direction is arranged to face downward, only the operation button 110 located relatively above is activated and the operation button 120 located below is deactivated to prevent malfunction. When used in the opposite direction so that the shooting direction is upward, only the operation button 120 located relatively above is activated and the operation button 110 located below is deactivated to prevent malfunction.

The circuit board 200 is a component that operates the complex skin analysis device of this embodiment and is composed of a PCB, includes circuits that can control various components included in the complex skin analysis device, and is used to drive them. A program can be installed to configure a control unit that controls the device. Furthermore, it may include a memory in which programs can be installed and data can be stored, and may include a communication unit that can exchange data and signals with an external device by wire or wirelessly.

The upper cover 300 is installed above the main body 100, and a display for operating the device and checking analysis results is installed. A separate button may be installed to operate the device, or it may be configured to operate without buttons by applying a touch-type display.

The light source unit 400 is installed at the inner lower part of the main body 100, and a light source and a camera for taking multi-spectral images are installed.

In general, a multispectral image has between 3 and 15 spectral bands in the visible light region with a wavelength range of 365 to 700 nm, and the visible light source of this embodiment can be adjusted variously within this range, but the full width at half maximum (full width at half maximum) of each light source is It is desirable to select the FWHM) wavelength to be 30 nm or less so that they do not overlap. In addition, the arrangement of the light source can be selected in various ways, but it is desirable to arrange the visible light source in a circle around the visible light camera and arrange the light source of the same wavelength symmetrically to secure the amount of light and prevent shadows due to light bias. do. The light source may include a white LED, which is necessary in the image processing process. This light source can be controlled so that a plurality of light sources for each wavelength are sequentially irradiated for multi-spectral image shooting.

The camera can use a visible light camera, which is generally used to capture multi-spectral images, and may include various image sensors that can acquire digital spectral images by capturing the reflected light emitted from visible light sources and reflecting off the image area. You can. When a plurality of visible light sources for each wavelength are controlled to be sequentially irradiated, the visible light camera may be controlled to acquire images for each wavelength band at each irradiation point.

On the other hand, the complex skin analysis device of the present invention is characterized in that it further includes a near-infrared light source and a near-infrared camera that irradiates wavelengths in the near-infrared region, compared to the conventional image of only a plurality of wavelengths in the visible light region. there is.

The arrangement form of the near-infrared light source is not particularly limited, and may be placed together with the visible light sources or may be placed in a separate location separate from the visible light sources. Additionally, the installation location of the near-infrared camera is not particularly limited, and may be placed next to the visible light camera or may be placed in another location separate from the visible light camera.

The near-infrared light source can be selected and used in the 600~1400nm wavelength range. A near-infrared light source of a single wavelength can be used, but near-infrared light sources of various wavelengths can also be used together. A near-infrared camera uses a camera that includes an image sensor that can acquire a digital image by photographing reflected light from near-infrared light irradiated from a near-infrared light source in the above-mentioned wavelength range and reflected off the image area.

In the general multispectral image acquisition process, a light source that includes the near-infrared range of 600 to 700 nm is often used, but when using a visible light camera with the characteristics of a visible light source and characteristics suitable for it, images taken in the near-infrared range are used. It has the disadvantage of relatively low analysis efficiency. Accordingly, the present invention can increase analysis efficiency by additionally providing a separate near-infrared light source that irradiates light in the near-infrared region and a near-infrared camera specialized for imaging in the near-infrared region.

In addition, near-infrared rays are not simply reflected from the surface of the skin, but also include reflected light reflected from within a certain depth of the skin, so it is different from skin analysis through multispectral imaging that conventionally used only visible light reflected from the surface of the skin. Differentiated analysis can be performed.

Furthermore, by using near-infrared rays reflected from within a certain depth of the skin, additional information such as blood flow can be obtained, and by reflecting this in skin analysis, it is possible to obtain better skin analysis results.

The specific process of performing skin analysis using both visible light and near-infrared light will be described in more detail later.

The lower cover 500 is installed below the main body 100, and visible light and near-infrared rays emitted from the light source unit 400 installed inside are irradiated to the skin through the through hole, and visible light and near-infrared rays reflected from the skin are transmitted through the through hole. It is incident through and imaging is performed by a visible light camera and a near-infrared camera installed in the light source unit 400. In this embodiment, a sensor 510 is installed to measure skin conditions such as temperature and moisture by contacting the skin. In particular, the sensor 510 is installed on an inclined surface forming a predetermined angle with the surface where the through hole for imaging is formed. It was configured so that the measurement process and imaging process through (510) can be performed separately. Furthermore, a blackout cap 520 was installed around the through hole to minimize the influence of external light, and the blackout cap 520 was made of a flexible material to effectively block external light even in curved parts of the body. . Since this blackout cap 520 is a part that comes into contact with the skin, it is designed to be attachable and detachable from the lower cover 500, making it easy to replace or clean.

The cradle 700 is a component that mounts the complex skin analysis device of this embodiment so that it can be stored upright. The cradle 700 protects the sensor 510 installed at the bottom and the internal light source unit 400 through the through hole, and performs charging of the battery 600 installed inside the main body 100 through the charging terminal. It may be possible.

Figure 2 is an exploded perspective view showing the structure of a complex skin analysis device according to a second embodiment of the present invention.

The complex skin analysis device of this embodiment includes a main body 100, a circuit board 200, an upper cover 300, a light source unit 400, a lower cover 500, a cradle 700, and a near-infrared attachment 800. Includes.

The second embodiment is characterized in that the near-infrared light source and the near-infrared camera are included in the detachable near-infrared attachment 800, and other than that, the configuration of the first embodiment can be applied. Therefore, the following will be described with reference to the first embodiment. It mainly explains the differences.

In the second embodiment, since the near-infrared light source and the near-infrared camera are included in a separate near-infrared attachment 800, the light source unit 400 may include only the visible light source and the visible light camera used to acquire general multispectral images. Through this configuration, the complex skin analysis device of the present invention can be manufactured by adding a near-infrared attachment 800 to a multispectral imaging device manufactured in a conventional manner.

The near-infrared attachment 800 is connected to the device main body by wire or wirelessly, so that not only can it be controlled together with the visible light source and visible light camera of the light source unit 400, but also can transmit captured images to the device main body.

Figure 3 is a diagram for explaining the near-infrared attachment applied to the complex skin analysis device according to the second embodiment of the present invention.

The near-infrared attachment 800 of this embodiment is configured such that a near-infrared camera 810 and a near-infrared light source 820 are disposed around the through hole formed in the lower cover 500. In this structure, the near-infrared attachment 800 can capture images in the near-infrared region while taking multi-spectral images in the visible light region through the visible light source and visible light camera of the light source unit 400. As described above, the near-infrared light source 820 may emit near-infrared rays of the same wavelength or may be composed of a plurality of light sources that irradiate near-infrared rays of different wavelength bands. In addition, the near-infrared light source 820 and the near-infrared camera 810 can be used as they are in the first embodiment, so detailed descriptions are omitted.

Figure 4 is a flowchart for explaining the operation process of the complex skin analysis device according to an embodiment of the present invention.

Using a visible light source and a visible light camera, spectroscopic images for multiple wavelengths of visible light are acquired, and NIR images for the near infrared (NIR) range are acquired using a near infrared light source and a near infrared camera. At this time, the process of acquiring a multi-spectral image for visible light can be applied to any conventional process of acquiring a multi-spectral image. And the NIR image may be an image of a single wavelength of near-infrared rays, or it may be a plurality of images of near-infrared rays of multiple wavelength bands.

In addition, not only NIR images but also NIR analysis information such as blood flow can be obtained through a near-infrared camera.

Image correction based on correction targets is performed on spectral images of visible light and NIR images. In order to standardize the intensity of each wavelength of images taken using light sources of a plurality of different wavelengths, image correction is performed using a correction target (white target). Since this image correction can be applied without limitation by using the same or modified techniques used in the analysis process using conventional multispectral images, a detailed description is omitted.

Then, the areas of the image acquired from the visible light camera and the image acquired from the near-infrared camera are matched. Since the complex skin analysis device of the present invention is separately equipped with a visible light camera suitable for the visible light region and a near-infrared camera suitable for the near-infrared region, differences may occur in the image area even when the same photographed region is photographed. Therefore, a matching process is needed to match the areas of the visible light spectral image acquired from the visible light camera and the NIR image acquired from the near infrared camera.

In the illustrated embodiment, the image area matching process is performed in the main body of the analysis device, and subsequent steps are performed in an external device connected to the analysis device by wire or wirelessly. To achieve this, a process is performed to transmit and confirm image data with matched image areas wired or wirelessly. The external device may be a smart device with an application for image processing and analysis installed, or a PC or server with a program for image processing and analysis installed.

When a visible light spectral image and a NIR image with matching image areas are transmitted, the image is corrected and standardized so that the overlapping wavelength band in the visible light spectral image and the NIR image can be used as a single analysis data.

Figure 5 is a diagram illustrating the process of standardizing visible light spectral images and NIR images in the complex skin analysis device according to an embodiment of the present invention.

Because the visible light spectral image acquired from a visible light camera and the NIR image acquired from a near-infrared camera contain information in different wavelength bands, in order to use them together, the data of these images must be corrected and standardized. At this time, since the data in the visible light spectral image has been corrected using a correction target, it can be standardized by matching the data in the NIR image to the visible light spectral image based on the data in the overlapping wavelength range in the visible light spectral image and the NIR image. You can. In this way, by standardizing the visible light spectral image and the NIR image, skin analysis can be performed using the visible light spectral image and the NIR image together.

Existing image sensors used in the process of acquiring visible light spectral images, such as general CMOS sensors, have a problem of not obtaining accurate information due to low reception sensitivity in the near-infrared region. The present invention improves the accuracy of analysis because accurate image data can be obtained in the near-infrared region by using a near-infrared camera using an image sensor that can obtain accurate images in the near-infrared wavelength range.

In addition, because near-infrared rays in a specific wavelength range are reflected at a certain depth of the skin, NIR images taken in this wavelength range reflect the depth of the lesion or the state of wrinkles in the skin. Therefore, not only does the accuracy of analysis improve because the information available in skin analysis increases, but also analysis results including results that could not be obtained in analysis using existing visible light spectral images can be obtained.

Furthermore, it is possible to obtain information such as blood flow volume and blood flow speed using near-infrared rays reflected at a certain depth of the skin, and by additionally reflecting this information, analysis that is differentiated from analysis using existing visible light spectroscopic images can be performed. there is.

Then, spectral indices are derived for each pixel from the standardized spectral images and NIR images. These spectral indicators are for skin analysis, and since the visible light spectral image corrected using a correction target and the NIR image are standardized accordingly, indicators for image analysis can be obtained for each pixel of the two images, and each The method of obtaining indicators for analysis of pixels is not particularly limited.

AI (artificial intelligence) analysis is performed using spectral index data obtained for each pixel on images obtained at various wavelengths in the visible and near-infrared regions. AI analysis is performed through an artificial intelligence algorithm, and the deep learning-based artificial intelligence algorithm is pre-trained using training data. The artificial intelligence algorithm is not particularly limited, but CNN models, which are excellent in processing image data, can be used.

The method of displaying AI analysis results is not particularly limited and various forms are possible. For example, as a method of displaying the analysis results, a disease severity image that indicates the severity of the disease in the lesion area using colors, etc. can be displayed. At this time, since the complex skin analysis device of this embodiment uses NIR images containing information such as the thickness of the lesion area and skin wrinkles for analysis, it is more accurate than existing disease severity images or is displayed in existing disease severity images. It can provide results that include information that was otherwise missing. Furthermore, analysis results reflecting NIR information such as blood flow volume or blood flow speed can be provided.

The present invention has been described above through preferred embodiments, but the above-described embodiments are merely illustrative examples of the technical idea of the present invention, and various changes are possible without departing from the technical idea of the present invention. Anyone with ordinary knowledge will be able to understand. Therefore, the scope of protection of the present invention should be interpreted based on the matters stated in the patent claims, not the specific embodiments, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of the present invention.

100: main body
110, 120: Control button
200: circuit board
300: Top cover
400: Light source unit
500: Lower cover
510: sensor
520: Blackout cap
600: Battery
700: Cradle
800: Near infrared attachment
810: Near-infrared camera
820: Near-infrared light source

Claims (10)

It is an analysis device that analyzes the skin using multiple images taken by irradiating light with different wavelengths,
A visible light source that irradiates visible light of different wavelengths in the visible light range;
A visible light camera that captures visible light reflected from a visible light source onto the skin;
A near-infrared light source that irradiates near-infrared rays of one or more wavelengths in the near-infrared region;
A near-infrared camera that photographs the reflection of near-infrared rays emitted from a near-infrared light source onto the skin; and
A complex skin analysis device comprising a control unit that controls the device and processes images.
In claim 1,
The visible light source is a complex skin analysis device characterized in that it consists of light sources capable of irradiating visible light having three or more wavelengths in the range of 365 to 700 nm.
In claim 1,
The near-infrared light source is a complex skin analysis device, characterized in that it consists of a light source capable of irradiating near-infrared rays having one or more wavelengths in the range of 600 to 1400 nm.
In claim 1,
The visible light source consists of light sources capable of irradiating visible light with three or more wavelengths in the range of 365 to 700 nm,
The near-infrared light source consists of light sources capable of irradiating near-infrared rays with two or more wavelengths in the range of 600 to 1400 nm,
A complex skin analysis device, wherein the visible light source and the near-infrared light source include one or more light sources that irradiate light in the range of 600 to 700 nm.
In claim 4,
Standardizes visible light spectroscopic images obtained from visible light cameras and NIR images obtained from near infrared cameras.
NIR images taken in the 600~700nm range are corrected to match the visible light spectral images taken in the 600~700nm range, and NIR images taken in other ranges are corrected to match the corrected NIR images taken in the 600~700nm range. A complex skin analysis device characterized by standardizing visible light spectral images and NIR images.
In claim 4,
Standardize the visible light spectral image obtained from the visible light camera and the NIR image obtained from the near infrared camera,
A complex skin analysis device that performs skin analysis using standardized visible light spectral images and NIR images together.
In claim 1,
A complex skin analysis device, characterized in that the control unit matches the image area of the visible light spectral image taken by the visible light camera with the image area of the NIR image taken by the near-infrared camera.
In claim 1,
A complex skin analysis device, characterized in that the near-infrared light source and the near-infrared camera are installed on a near-infrared attachment configured to be detachable from the device.
In claim 1,
A complex skin analysis device characterized in that information on blood flow volume or blood flow speed is obtained from a near-infrared camera and used for analysis.
In claim 1,
It further includes a grip part that the user can hold and use, two operation buttons installed on both ends of the grip part, and a sensor that detects the posture of the device,
A complex skin analysis device, characterized in that among the two operation buttons, only the operation button located relatively above is controlled to operate.

Images