KR102052263B1 - Standard image data filming-obtaining apparatus for determining inflammatory skin diseases - Google Patents

Abstract

The present invention relates to a photographing apparatus for acquiring standard image data for diagnosing inflammatory skin disease severity. More particularly, the present invention can efficiently perform imaging for diagnosing inflammatory skin diseases such as atopic skin disease. The present invention relates to a photographing apparatus for acquiring standard image data for diagnosing severity of an inflammatory skin disease that can be processed into data and provided.
According to the present invention, the skin disease diagnosis as an imaging device, the base plate; A main frame installed to stand on an upper surface of the base plate; A photographing unit support arm installed to be perpendicular to the main frame in a lateral direction; Support arm driving means configured to drive the photographing unit support arm in a liftable manner on the main frame; A plurality of photographing units installed on the photographing unit support arms; A display unit provided at one side of the main frame to display a predetermined screen under control of a controller; And a control unit provided in the main frame to control operations of the support arm driving unit, the photographing unit, and the display unit, and to process the image into standard image data for diagnosing a lesion based on the photographed image from the photographing unit. Characterized in that it comprises a.

Description

IMAGE STANDARD IMAGE DATA FILMING-OBTAINING APPARATUS FOR DETERMINING INFLAMMATORY SKIN DISEASES}

The present invention relates to a photographing apparatus for acquiring standard image data for diagnosing inflammatory skin disease severity. More particularly, the present invention can efficiently perform imaging for diagnosing inflammatory skin diseases such as atopic skin disease. The present invention relates to a photographing apparatus for acquiring standard image data for diagnosing severity of an inflammatory skin disease that can be processed into data and provided.

Inflammatory skin skin problem is an inflammatory reaction such as itching, swelling, erythema and peeling due to various irritating factors that cause a series of inflammatory reactions in the epithelium of the skin. It refers to the accompanying skin disease.

Such inflammatory skin diseases include allergic dermatitis caused by various causes, bites caused by animals or insects, skin fungus, acne and the like. Among them, allergic dermatitis collectively refers to dermatitis caused by various allergens, including atopic dermatitis, contact dermatitis, urticaria, food allergy, and drug allergy.

Among allergic dermatitis, atopic dermatitis is a serious problem due to the recent increase in the westernization of the diet, housing environment, increased stress, and contaminants due to industrial development.

This atopic dermatitis is a highly itchy eczema that occurs in people with atopic dermatitis. Atopic dermatitis, a type of allergic dermatitis, corresponds to 80-90% of pediatric eczema. It also happens a lot. Inflammatory dermatitis, such as atopic dermatitis, is a chronic disease of more than 1 trillion won per year. Atopic dermatitis was observed in 13.5 children per 100 children and 5 adults per 100 years. The average number of patients treated for atopic dermatitis in the past 5 years is 1.44 million. Direct / hepatic beauty per person with atopic dermatitis totals 4,992,000 won, resulting in an annual prevalence rate of 10% and social expenses of 5.8 trillion (Ann Dermtol 2015).

The severity of atopic dermatitis is determined by the clinician's examination, and the follow-up is based on the severity readings and the photographic data of the patient's lesion.

However, in determining the severity of the atopic dermatological disease, the results of the examination differ from clinician, and even if the same clinician performs the examination, the results may vary depending on the external environment such as physical condition.

In other words, atopic dermatitis is currently diagnosed entirely with gross findings, and subjectivity by the clinician is involved, which leads to lack of objectivity. In particular, atopic dermatitis is used to determine the severity of pidgin that determines treatment. It is a difficult task.

The fundamental reason for this problem is that there is no stage in dermatology, imaging and analysis through radiology, which is a general medical practice. Therefore, in order to systematically measure the severity of inflammatory dermatitis in dermatology, there is an urgent need for a device capable of providing standardized imaging, which is being established in radiology.

(Document 0001) Republic of Korea Patent Publication No. 10-1623431 (Registration Date: 2016. 05. 17.) (Document 0002) Republic of Korea Patent Publication No. 10-1015369 (Registration Date: February 10, 2011) (Patent 0003) Republic of Korea Patent Publication No. 10-2015-0059689 (published date: 2015. 06. 02.)

An object of the present invention is to solve the above-mentioned problems, a standard image for the severity of inflammatory skin disease to improve the objective research and diagnostic environment for skin diseases by introducing a new image diagnosis area in the dermatology field A photographing apparatus for acquiring data is provided.

Another object of the present invention is to establish an efficient imaging condition for the diagnosis of inflammatory skin diseases such as atopic skin disease, and to obtain standard image data for diagnosing the severity of inflammatory skin disease to be processed and provided with standardized image data. The present invention provides a photographing apparatus.

In order to achieve the above object, the present invention provides a diagnostic apparatus for diagnosing skin diseases, comprising: a base plate; A main frame installed to stand on an upper surface of the base plate; A photographing unit support arm installed to be perpendicular to the main frame in a lateral direction; Support arm driving means configured to drive the photographing unit support arm in a liftable manner on the main frame; A plurality of photographing units installed on the photographing unit support arms; A display unit provided at one side of the main frame to display a predetermined screen under control of a controller; And a control unit provided in the main frame to control operations of the support arm driving unit, the photographing unit, and the display unit, and to process the image into standard image data for diagnosing a lesion based on the photographed image from the photographing unit. Characterized in that it comprises a.

The photographing unit support arm may include a moving frame interlocked with the support arm driving means and extended to both sides with respect to the main frame, and a photographing unit mounting frame coupled to both ends of the moving frame.

In addition, the support arm driving means includes a coupling block provided in the moving frame, and a linear motor provided in the main frame and coupled to the coupling block to drive the moving frame to be linearly movable. The arm further includes folding means configured to be foldably coupled to the photographing unit mounting frame with respect to the moving frame, wherein the folding means is axially rotatably coupled to an end of the moving frame and the photographing unit mounting frame. The shaft is configured to be connected to a drive motor capable of rotating forward and backward and receiving the control signal of the controller to rotate the drive.

In addition, the movement guide slot is formed in the longitudinal direction of the shooting unit installation frame, the movement block is moved along the movement guide slot and the shooting unit is installed, the main frame or the shooting unit support arm is provided in the A drive motor for receiving the control signal of the control unit for rotational driving and a drive force transmission member for converting the rotational driving force of the drive motor to a linear movement driving force to transmit to the moving block.

In addition, the control unit may include: a support arm driving control unit for controlling the support arm driving unit to raise and lower the photographing unit support arm; a photographing control unit for controlling photographing of the photographing unit; and control logic of preset setting conditions. A display controller for displaying and guiding a photographing environment to be photographed according to the present invention, a processing unit for processing standard image data for diagnosing severity of skin disease based on the image photographed by the photographing unit, and a standard processed by the processing unit. It includes a storage unit for storing the image data.

The processing unit may receive the captured image of the diagnosis target lesion by the photographing unit, calculate feature information for determining a region of interest in the photographed image, and calculate the feature information in the photographed image using the calculated feature information. From the input image using a first neural network that detects a region of interest, takes the detected region of interest as an input, and outputs at least one lesion value as an indicator for diagnosing the type or severity of the lesion. Standard image data including information for diagnosing a target lesion is generated.

In addition, the ROI includes at least a part of an image corresponding to the lesion in the photographed image of the lesion, and the feature information includes a center coordinate of the ROI in the image, and the feature information. The calculation may include resizing the photographed image to a predefined size, dividing the resized image to generate a grid cell having a predetermined size, and having a center inside the generated grid cell and a target lesion of the captured image. Generating a plurality of boundary cells indicating the probability of the lesion including at least a part thereof, and calculating the feature information using the generated grid cells and boundary cells, wherein the captured image is input, and the boundary The center coordinate of the cell or the lesion in the boundary cell Calculates a probability of the existence of the center coordinates of each of the generated boundary cells and the lesion in the boundary cells using a second neural network outputting a probability, and the probability and the boundary coordinates of which the center coordinates are calculated. The feature information may be calculated using a cell, and the detection of the ROI may include determining whether a probability that the lesion is present in the boundary cell is greater than or equal to a preset threshold among boundary cells including the image of the lesion. Considering whether the lesion is removed by removing a part of boundary cells including overlapped images of the photographed image, the ROI is detected by using the remaining boundary cells, and the generation of the standard image data includes: The detected plurality of shot images are stored in one or more pieces according to preset criteria. Classify the lesion by the lesions, label the severity of the lesions included in the classified photographed images, set some of the plurality of photographed images labeled with the severity as training data, and set the rest as verification data; And generating standard image data including information for diagnosing the lesion in the photographed image by using the first neural network trained by learning the first neural network based on verification data.

The first neural network may include at least one convolutional layer for extracting a convolutional feature through a convolution operation, and a pulley connected layer connected to one end of the convolutional layer to classify the type or extent of the lesion. It is learned in advance by determining the weight of the connection strength between the layer and the neurons inside the pulley connected layer, and the arrangement structure and the number of placement of the convolutional layer and the pulley connected layer are mutually dependent on the type of the lesion to be diagnosed. The second neural network is configured to be provided differently, and the second neural network is connected to at least one convolutional layer extracting a convolutional feature through a convolutional operation and one end of the convolutional layer, in which a center coordinate of the boundary cell and the lesion in the boundary cell exist. Calculate probability Includes at least one pulley connected layer, and pre-learns by determining weights of connection strengths between the convolutional layer and neurons inside the pulley connected layer based on the training data and the correct answer data about the predetermined boundary cell. And a residual block including a path connecting adjacent convolution layers and a residual path crossing at least one of the adjacent convolution layers to be connected to another convolution layer, wherein the residual block receives an input of the residual block. Reference is made to the input of another adjacent residual block.

As apparent from the above description, the imaging device for obtaining standard image data for severity diagnosis of inflammatory skin disease of the present invention can provide standardized image data for diagnosis of inflammatory skin disease such as atopic skin disease. It is effective in supporting the medical examination of patients and in ensuring that patients are accompanied by accurate and reliable treatment.

At the same time, it is possible to preoccupy the market of atopic dermatitis diagnostic devices, in which more than 1 million patients spend more than 1 trillion won per year. Furthermore, it can contribute to the national competitiveness in the medical field by securing technological advantage in the global market. It is a very useful invention.

1 is a perspective view showing a photographing apparatus for obtaining standard image data for diagnosing severity of inflammatory skin disease according to the present invention.
Figure 2 is a front view showing a photographing apparatus for obtaining standard image data for diagnosing the severity of inflammatory skin disease according to the present invention.
Figure 3 is a side view showing a photographing device for obtaining standard image data for diagnosing the severity of inflammatory skin disease according to the present invention.
Figure 4 is a side view showing the folded state of the recording unit support arm constituting the imaging device for obtaining standard image data for severity diagnosis of inflammatory skin disease according to the present invention.
5 is a block diagram showing the configuration of a control unit constituting a photographing apparatus for obtaining standard image data for diagnosing severity of inflammatory skin disease according to the present invention.
6 is a diagram illustrating a data processing process in a processing unit of a control unit constituting a photographing apparatus for obtaining standard image data for diagnosing severity of inflammatory skin disease according to the present invention.
7 is a view showing the structure of a convolutional neural network used by the feature information calculation unit of the control unit constituting the photographing apparatus for acquiring standard image data for severity diagnosis of inflammatory skin disease according to the present invention.
8 is a view showing the lattice cell and the boundary cell generated by the processing unit of the control unit constituting the photographing apparatus for obtaining standard image data for diagnosing inflammatory skin disease severity according to the present invention.
9 is an exemplary view of the image taken by the lesion, severity of atopic dermatitis.
Figure 10 is an illustration of the images taken by the lesion, severity of atopic dermatitis.
11 is a view showing the structure of a convolutional neural network used by the diagnostic unit of the control unit constituting the photographing apparatus for obtaining standard image data for severity diagnosis of inflammatory skin disease according to the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention.

First of all, in adding reference numerals to the components of the drawings, it should be noted that the same reference numerals have the same reference numerals as much as possible even if displayed on different drawings. In addition, in describing the present invention, when it is determined that the detailed description of the related well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

1 is a perspective view of a photographing apparatus for obtaining standard image data for diagnosing inflammatory skin disease severity according to the present invention, and FIG. 2 is a photographing apparatus for obtaining standard image data for diagnosing severity of inflammatory skin disease according to the present invention. 3 is a side view illustrating a photographing apparatus for obtaining standard image data for diagnosing inflammatory skin disease severity according to the present invention, and FIG. 4 is for obtaining standard image data for diagnosing inflammatory skin disease severity according to the present invention. A side view showing a folded state of the photographing unit support arm constituting the photographing apparatus.

As shown in Figures 1 to 4, the imaging device for obtaining standard image data for diagnosing the severity of inflammatory skin disease according to the present invention, the base plate 100 of a predetermined size; A main frame 200 installed to stand on an upper surface of the base plate 100; A photographing unit support arm 300 installed at right angles to the main frame 200 in a lateral direction; Support arm driving means for operating the photographing unit support arm 300 to be movable up and down in the main frame 200; A plurality of photographing units 400 installed on the photographing unit support arms 300; A display unit 500 provided on one side of the front surface of the main frame 100; And provided in the main frame 100 to control operations of the support arm driving means, the photographing unit 400, and the display unit 500, and standard image data for diagnosing a lesion based on a photographed image from the photographing unit. And a control unit 600 (see FIG. 5) for processing into a furnace.

The base plate 100 is not particularly limited as long as it is a shape or a shape capable of supporting and fixing the main frame 200.

Preferably, the lower surface of the base plate 100 may be configured with a rolling wheel to secure the mobility of the image capturing apparatus for obtaining the standard image data, when moving the image capturing apparatus for obtaining the standard image data for fixing at the installation position It may further include a fixing member which can be fixed by releasing or withdrawing the folded state at the installation position in the folded state or the retracted state.

Subsequently, the main frame 200 may be installed in the base plate 100 in a predetermined length, and may be configured as a housing-type frame in which other components are accommodated or slidably coupled.

Next, the photographing unit support arm 300 is coupled to the support arm driving means to be described later and the moving frame 310 is provided to extend to both sides with respect to the main frame 200, and both ends of the moving frame 310 The photographing unit installation frame 320 is coupled to each other, and the photographing unit installation frame 320 includes a folding means configured to be foldably coupled to the moving frame (310).

The folding means is not particularly limited as long as the photographing unit mounting frame 320 is configured to be folded and unfolded by manual operation with respect to the moving frame 310 or to be automatically folded and unfolded by driving of a motor or the like.

For example, the folding means is rotatably coupled to the recording unit mounting frame 320 to the end of the moving frame 310, the axis is forward and reverse rotation and transmits the control signal of the control unit 600 It may be configured to be connected to a drive motor that receives rotational drive. Accordingly, the controller 600 may be automatically operated based on a manager command input through a control panel constituting the controller 600 or a predetermined control logic (for example, at the start and end of photographing).

In addition, the installation frame 320 of the photographing unit support arm 300 is further provided with a recording unit moving means for moving the photographing unit 400 along its longitudinal direction.

Specifically, the photographing unit moving means is a movement guide slot 321 formed in the longitudinal direction of the installation frame 320, the movement block is moved along the movement guide slot 331 and the shooting unit is installed, and the movement And mounting frame driving means for driving the block.

The installation frame driving means is provided in the main frame 200 or the photographing unit support arm 300, the drive motor for driving the rotation by receiving the control signal of the control unit 600, and linearly rotates the driving force of the drive motor And a driving force transmitting member for converting the moving driving force to the moving block. The driving force transmitting member may be adopted as a rack gear driving method or a pulley interlock driving method.

The control of the driving motor may be automatically performed based on a manager command or a predetermined control logic input through a control panel constituting the controller 600.

Of course, it is possible to employ a configuration that can omit the installation frame driving means and manually move the moving block and then fix it, but it is preferably configured to be automatically moved for accurate and reliable shooting in the left and right photographing unit. Do.

Next, the support arm driving means for operating the recording unit support arm 300 to move up and down in the main frame 200, the coupling block is configured in the moving frame 310, and the main frame 200 And a linear motor coupled to the coupling block and configured to linearly move the moving frame. Here, since the configuration of the linear motor can adopt a known one, a detailed description thereof will be omitted for clarity and simplification of the present invention.

In addition, the plurality of photographing units 400 installed in the photographing unit support arm 300 may employ a camera capable of macro photographing.

The photographing unit 400 may be configured to automatically operate based on a manager (operator) command or a predetermined control logic input through a control panel constituting the controller 600.

Subsequently, the display unit 500 may guide the patient to a posture to be taken for shooting and a shooting situation such as a shooting situation, for example, by a manager's voice during shooting, but the shape of the human body in the display unit 500. Through what position to take through, the recording unit 400 can be made to provide information about what is currently shooting at what position.

Next, the controller 600 will be described in detail with reference to FIGS. 5 to 7. 5 is a block diagram showing the configuration of a control unit constituting a photographing apparatus for acquiring standard image data for diagnosing inflammatory skin disease severity according to the present invention, and FIG. 6 is a standard image for diagnosing inflammatory skin disease severity according to the present invention. FIG. 7 is a diagram illustrating a data processing process in a processing unit of a control unit constituting an image capturing apparatus, and FIG. 7 shows feature information of a control unit constituting a photographing apparatus for obtaining standard image data for diagnosing severity of inflammatory skin disease according to the present invention. It is a figure which shows the structure of the convolutional neural network which additionally uses.

Prior to the description of the control unit, the configuration of the control unit may be implemented by executing a computer program by the computing device, and may be configured integrally or separately from the main frame 200.

The control unit 600 controls the support arm driving means to support the driving unit support arm 300 to move up and down the support arm driving control unit 610, and the shooting control unit for controlling the shooting of the recording unit 400 ( 620, a display controller 630 for displaying and guiding a photographing environment (a posture and a part to be photographed, etc.) photographed according to a control logic of a preset setting condition, based on an image photographed by the photographing unit 400. And a data processing unit 640 for processing standard image data for diagnosing severity of skin disease, and a storage unit 650 for storing standard image data processed by the data processing unit 640.

The support arm driving control unit 610 and the photographing control unit 620 are controlled by a manager's control in a spaced apart manager control panel, and the display unit control unit 630 displays a manager's selection operation as a display environment according to a preset program. Is made.

The data processing unit 640 largely includes a specific information calculating unit 641, an ROI detection unit 642, and a diagnosis unit 643.

The feature information calculator 641 includes a preprocessor and a calculator.

The specific information calculating unit 641 receives the captured image of the diagnosis target lesion by the photographing unit 400 and calculates feature information for determining a region of interest in the image.

The feature information calculator 641 is information for automatically extracting a region of interest including a lesion mainly from an image of a diagnosis target lesion, and calculates specific information including a center coordinate of the region of interest on the image. Is done.

In detail, the feature information calculator 641 may use an image recognition algorithm based on a neural network to calculate feature information for setting a region of interest in a captured image of a target lesion. For example, the image recognition algorithm used by the feature information calculator 641 may be implemented in a neural network to which a YOLO (You Only Look Once) structure is applied. Unlike the neural network to which the YOLO (You Only Look Once) neural network is applied to the conventional Fast R-CNN structure, the YOLO neural network has the advantage of simultaneously classifying the boundary cell position and the boundary cell class at the final output of the neural network.

For example, the second neural network used by the feature information calculating unit 641 includes a neural network to which a YOLO structure is applied, and in addition, a gradient block including a residual block provided with a residual connection. By solving the problem, it is possible to achieve improved image recognition performance over an algorithm using a neural network to which the conventional YOLO structure is applied.

In addition, the feature information calculator 641 has a center inside a grid cell, which will be described later, and includes the plurality of boundary cells indicating the probability of the lesion including at least a part of the captured image of the target lesion. Feature information can be calculated.

The preprocessor of the feature information calculator 641 resizes the image to a predetermined size, divides the resized image, generates a grid cell having a predetermined size, and preprocesses the photographed image.

In detail, the preprocessor may resize the size of the photographed image of the lesion to the size necessary for the second neural network, in order to calculate feature information for extracting the ROI including the lesion. In addition, the preprocessor may generate a predetermined grid cell by dividing the resized image, and may generate a total of 49 grid cells, such as 7 * 7.

Next, the calculation unit of the feature information calculator 641 receives a photographed image, and generates a second neural network outputting a center coordinate of the boundary cell or a probability that the lesion is present in the boundary cell. Generate the boundary cells dependent on the grid cells, and calculate the center coordinates of each of the generated boundary cells and the probability that the lesion exists in the boundary cells.

Specifically, the second neural network used by the calculation unit is connected to at least one convolutional layer 643a for extracting a convolutional feature through a convolutional operation, and one end of the convolutional layer and the center coordinates of the boundary cell and the boundary within the boundary cell. It may include at least one pulley connected layer 643b that calculates the probability that the lesion is present.

In addition, the second neural network used by the calculator may further include a pooling layer 643c that is alternately disposed with the convolutional layer 643a in addition to the convolutional layer 643a and the pulley connected layer 643b.

The second neural network used by the calculation unit is a composite product neural network to which the YOLO structure is applied, and includes a residual block provided with a residual connection to at least a part of the convolutional layer 643a included in the neural network, thereby achieving improved image recognition performance. . That is, the second neural network used by the calculator may be implemented using an Atopic Region Detection (ARD) algorithm in which a residual connection is applied in addition to the YOLO algorithm.

At least a portion of the convolutional layer 643a of the second neural network is a path connecting the adjacent convolutional layer 643a and a residual path connected to another convolutional layer 643a by skipping at least one adjacent convolutional layer 643a. It includes a residual block is provided, the input of the residual block may be referred to as the input of another adjacent residual block.

In addition, the second neural network used by the calculator may be pre-learned by determining weights of connection strengths between the convolutional layer and the neurons inside the pulley connected layer based on the training data and the correct answer data about the predetermined boundary cell. have.

When the feature information calculating unit 641 of the data processing unit 640 constituting the control unit 600 calculates the feature information using the second neural network will be described. FIG. 8 illustrates lattice cells and boundary cells generated by a processing unit of a control unit constituting a photographing apparatus for obtaining standard image data for diagnosing severity of inflammatory skin disease according to the present invention.

The feature information calculator 641 receives an image of a diagnosis target lesion, and the preprocessor 642 resizes the input image and generates a grid cell at a predetermined interval.

The calculation unit of the feature information calculator 641 has a center coordinate inside the grid cell using a second neural network, and generates any number of boundary cells S1, S2, S3, S4, and S5 that are dependent on the grid cell. At the same time, the class of the image included in the boundary cell can be classified.

The boundary cells S1, S2, S3, S4, and S5 generated by the calculation unit using the second neural network may be specified by the relative coordinates of the boundary cells in the center coordinates of the boundary cells or the entire resized image. In addition, boundary cells S1, S2, S3, S4, and S5 generated by the calculation unit using the second neural network, respectively, have information on the probability that the lesion image exists inside the boundary cell, or when the lesion image exists. It can have probability information about what kind it is.

For example, the preprocessing unit of the feature information calculating unit 641 generates a grid cell of width * length 7 * 7 from the resized image, and boundary cells S1, S2, S3, S4, If the number of S5) is 2, a total of boundary cells generated by the calculation unit may be generated in total of 7 * 7 * 2.

Next, the ROI detector 642 of the data processor 640 detects the ROI in the image by using the calculated feature information. The ROI includes at least a part of an image corresponding to the lesion among the photographed images of the lesion, and boundary cells S1, S2, S3, S4, and the like generated by the calculator of the feature information detector 641. At least one of S5) may be provided.

For example, the ROI detection unit 642 may include boundary cells S1, S2, S3, S4, and S5 having a different probability that a lesion exists in the boundary cell generated by the calculation unit of the feature information calculator 641. The boundary cells S1 and S2 having the highest probability of having a lesion therein may be detected as the ROI. For this purpose, among the boundary cells S1 and S2 in which any one lesion existing in the photographed image overlaps the image, the lesion is considered in consideration of whether or not the probability that the lesion exists in the boundary cell is greater than or equal to a preset threshold. A part of the boundary cell S2 including the captured image may be removed.

For example, the ROI detection unit 642 may determine that the probability that the lesion is present in the boundary cell is not greater than or equal to a preset threshold value among boundary cells S2 and S3 including the lesion image at the lower right side shown in FIG. 8. It can remove (here, remove boundary cell S3). Accordingly, the ROI detection unit 642 may detect one boundary cell S1 including the lesion image shown in the lower right in FIG. 8 as the ROI.

In addition, the ROI detection unit 642 uses only one boundary cell by using a Nom-maximal Suppression Non-Maximum Suppression (NMS) algorithm in order to remove a boundary cell having a probability that a lesion is present in the cell below the set threshold. I can leave it. That is, when the boundary cells in which the lesions overlap on the captured image exist among the plurality of boundary cells generated by the calculation unit, the ROI detection unit 642 uses the NMS algorithm to select an arbitrary lesion image on the image where the lesion was captured. Only one boundary cell can be left per. The ROI detector 642 may detect one boundary cell remaining per lesion image as the ROI.

Next, the diagnosis unit 643 of the data processing unit 640 receives the detected ROI, and outputs at least one lesion value as an indicator for diagnosing the type or severity of the lesion. (Neural Network) is used to diagnose the target lesion from the input image. 9 is an illustration of the image taken by the lesion, the severity of atopic dermatitis, Figure 10 is an illustration of the image taken by the lesion, severity of atopic dermatitis, Figure 11 is for the diagnosis of inflammatory skin disease severity according to the present invention It is a figure which shows the structure of the convolutional neural network which the diagnostic part of the control part which comprises the imaging device for acquiring standard image data uses.

Lesions associated with atopic dermatitis diseases include Erythema, Papulation, Excoriation and Lichenification.

In the present invention, the diagnosis unit 643 of the data processing unit 640 automatically detects a region of interest mainly containing the lesion in the captured image including the lesion of atopic dermatitis, and types of lesions in the detected region of interest and the lesion. The severity of each type can be diagnosed objectively, and the severity of each lesion can be classified according to the purpose and criteria of diagnosis.

The diagnosis unit 643 of the data processor 640 may receive the detected ROI, and may diagnose the type and severity of the lesion included in the ROI input using the pre-learned first neural network.

The first neural network and the second neural network may be provided as neural networks to which an image recognition algorithm based on a synthetic multiplicative neural network is applied.

Although the diagnosis unit 643 of the data processing unit 640 may generate a diagnostic model using the first neural network trained by dividing the lesions, the diagnostic model may be generated by using the trained first neural network without dividing each lesion. The lesions can be generated and generated using the generated diagnostic model. When the diagnostic unit 643 of the data processing unit 640 generates a diagnostic model using the first neural network trained by the lesions, the at least one generated by using the first neural network trained by the lesions. By combining the diagnostic models of the new combination diagnostic model can be generated.

In addition, the diagnosis unit 643 of the data processing unit 640 classifies the plurality of captured images from which the ROI is detected by one or more lesions according to preset criteria, and severity of the lesions included in the classified captured images. Label the.

For example, the diagnosis unit 643 may generate a training data for learning the first neural network and verification data for verifying the trained first neural network. Detecting the region of interest included mainly, classifying the plurality of photographed images in which the region of interest is detected into three types of lesions, and the severity of the lesions in the photographed images in which the plurality of regions of interest classified by the type of lesions are detected. Are labeled with no lesion '0', severity '1', severity '2', and severity '3', and at least some of the labeled photographed images are set as learning data and the remaining parts as correct data, and the learning data and correct answer. The first neural network can be learned based on the usage data.

In the present invention, the diagnostic unit 643 of the data processing unit 640 may use 70% of the labeled photographed image as training data and 30% as verification data. For example, in the present invention, the diagnostic unit 643 uses 1400 learning data for learning erythema and scraping, and uses 308 learning data for lichenification. . In addition, the neural network can be trained in advance by utilizing Tensorflow, which can easily create an artificial neural network model in various hardware environments for smooth lesion-specific model learning.

Subsequently, the diagnosis unit 643 may determine the strength of the connection between the neuron inside the convolutional layer and the pulley connected layer based on the training data set as at least a part of the captured image and the verification data set as the other part. The first neural network is trained by determining weights associated with the first neural network.

In the present invention, the first neural network includes at least one convolutional layer for extracting a convolutional feature through a convolution operation, and a pulley connected layer connected to one end of the convolutional layer to classify the type or extent of the lesion, wherein the convolutional layer And the arrangement structure and the number of arrangements of the pulley connected layers may be provided differently according to the type of the lesion to be diagnosed.

For example, the 'Gradient Descent' method may be used to adjust the weight of the connection strength between the convolutional layer in the neural network and the neuron in the fully connected layer.

[Equation 1]

는 가중치,

는 가중치를 입력으로 하는 손실 함수(Loss Function),

는 손실 함수의 기울기(gradient),

는 러닝 레이트(learning rate)를 의미한다 상기 수학식 1은 학습부(340)가 경사 하강법(gradient descent)을 이용하여 손실 함수를 최소화하는 가중치를 결정하는 과정을 나타낸다. 예를 들어, 진단부(643)는 현재 가중치에서 손실 함수의 기울기의 음수에 비례하는 만큼 이동하는 것을 반복하여 손실 함수 를 최소화 하는 가중치를 결정할 수 있다.here,

Is the weight,

Is a loss function with weights as input,

Is the gradient of the loss function,

Equation 1 represents a process in which the learner 340 determines a weight for minimizing a loss function by using a gradient descent. For example, the diagnosis unit 643 may determine a weight that minimizes the loss function by repeating the current weighting in proportion to the negative number of the slope of the loss function.

In addition, the diagnosis unit 643 receives a weight and minimizes a loss function that defines a difference between an output of the first neural network according to the weight and an actual output of the first neural network derived based on the verification data. The first neural network may be trained in consideration of the running rate and the number of learning repetitions. The running rate of the present invention may be experimentally determined to minimize the loss function, and may have a value within the range of 001 to 0001. The function may have another function output according to the number of learning repetitions in addition to the running rate, and the learner 340 may determine the weight in consideration of the number of learning repetitions in order to minimize the loss function. That is, the diagnosis unit 643 may learn the first neural network in consideration of the running rate and the number of learning repetitions to minimize the loss function.

Subsequently, the diagnosis unit 643 may learn a neural network based on the training data and the verification data provided for each lesion, and generate a diagnostic model for each lesion. For example, the erythema diagnosis model may be trained to determine only the severity of the erythema based on the training data and diagnostic data related to the erythema, and only the severity of the thymus is determined based on the training data and the diagnostic data related to lichenification. You can train the diagnostic model. In the present invention, the diagnosis unit 643 may not only learn a diagnosis model for diagnosing the severity of each lesion, but also learn a diagnosis model for diagnosing both the type and the severity of the lesion.

In addition, the diagnosis unit 643 combines the types of lesions to be diagnosed according to the types of the lesions to be diagnosed. For example, a combination diagnosis model may be generated by combining a diagnosis model according to the first neural network learned for each lesion, and the lesion may be diagnosed using the combined diagnosis model.

The data processing unit 640 for configuring the control unit 600 and processing the standard image data for diagnosing skin disease severity based on the captured image is as follows.

The data processor 640 resizes the input photographed image to a predetermined size, and divides the resized image to generate a grid cell having a predetermined size.

The data processing unit 640 receives a captured image as an input, and is dependent on the grid cell using a second neural network outputting a center coordinate of a boundary cell or a probability that the lesion exists in the boundary cell. Boundary cells are generated, and the center coordinates of each of the generated boundary cells and the probability of the presence of lesions in the boundary cells are calculated.

Subsequently, the data processing unit 640 detects an ROI that is a region mainly containing the lesion by using the calculated characteristic information, uses the detected ROI as an input, and an indicator for diagnosing the type or severity of the lesion. The target lesion is diagnosed from the input image by using a first neural network that outputs at least one lesion value. Here, the data processing unit 640 outputs a lesion value including the type of the lesion and the severity of the corresponding lesion type by using the neural network to generate standard image data for diagnosis of the lesion along with the input image, and generates the generated standard image. The data is stored in the storage 650.

In summary, the imaging device for acquiring standard image data for diagnosing inflammatory skin disease severity according to the present invention can provide standardized image data for diagnosing inflammatory skin diseases such as atopic skin disease. It also benefits patients with the need for accurate and prompt treatment.

In addition, it is possible to preoccupy the market of atopic dermatitis diagnostic devices, in which more than 1 million patients spend more than 1 trillion won annually. It brings the benefits of improvement.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains various modifications, changes, and substitutions without departing from the essential characteristics of the present invention. will be. Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical spirit of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by the embodiments and the accompanying drawings. . The scope of protection of the present invention should be interpreted by the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: base plate, 200: main frame
300: shooting unit support arm, 310: moving frame
320: shooting unit installation frame, 321: moving guide slot
400: shooting unit, 500: display unit
600: control unit, 610: support arm drive control unit
620: shooting control unit, 630: display control unit
640: data processing unit, 650: storage unit

Claims (8)

Skin disease diagnosis as an imaging device,
Base plate;
A main frame installed to stand on an upper surface of the base plate;
A photographing unit support arm installed to be perpendicular to the main frame in a lateral direction;
Support arm driving means configured to drive the photographing unit support arm in a liftable manner on the main frame;
A plurality of photographing units installed on the photographing unit support arms;
A display unit provided at one side of the main frame to display a predetermined screen under control of a controller; And,
A control unit provided in the main frame to control operations of the support arm driving unit, the photographing unit, and the display unit, and to process the image into the standard image data for diagnosing the lesion based on the photographed image from the photographing unit; But
The control unit includes: a support arm driving control unit for controlling the support arm driving means to drive the photographing unit support arm up and down;
A photographing controller for controlling photographing of the photographing unit, a display controller for displaying and guiding a photographing environment to be photographed according to a control logic of a preset setting condition;
A processing unit for processing the standard image data for diagnosing severity of skin disease based on the image photographed by the photographing unit;
A storage unit for storing standard image data processed by the processing unit;
The processing unit may receive a captured image of a diagnosis target lesion by the photographing unit, calculate feature information for determining a region of interest in the photographed image, and use the calculated feature information to determine the region of interest in the photographed image. A target lesion from the input image by using a first neural network that detects the target region as an input and outputs at least one lesion value as an index for diagnosing the type or severity of the lesion. Contains diagnostic information,
The region of interest includes at least part of an image of a portion corresponding to the lesion among the photographed images of the lesion,
The feature information may include a center coordinate of the ROI in the image,
The feature information may be calculated by resizing the photographed image to a predetermined size, dividing the resized image to generate a grid cell having a predetermined size, and having a center inside the generated grid cell and photographing a target lesion. And generating a plurality of boundary cells representing the probability of the lesion including at least a portion of the extracted image to calculate the feature information using the generated grid cells and boundary cells, wherein the photographed image is input. The center coordinates of the boundary cells or the center coordinates of each of the generated boundary cells and the lesions in the boundary cells using a second neural network outputting a probability of the presence of the lesions in the boundary cells. Compute the probability of existence, the probability and the center coordinates are calculated Using a cell boundary is made to calculate the characteristic information,
The detection of the ROI may be performed by overlapping the photographed image of the lesion in consideration of whether a probability that the lesion is present in the boundary cell among the boundary cells including the image of the lesion is overlapped. Removing the part of the boundary cell including the detected boundary cell, and detecting the region of interest using the remaining boundary cell,
The generation of the standard image data may include classifying the plurality of photographed images from which the ROI is detected by one or more lesions according to preset criteria, labeling the severity of the lesions included in the classified photographed images, and the severity. Some of the plurality of labeled images are set as training data and others as verification data, and the first neural network is trained based on the training data and the verification data to learn the photographed image. The apparatus for acquiring standard image data for diagnosing severity of inflammatory skin disease, characterized in that the standard image data including information for diagnosing the lesion is generated.
The method of claim 1,
The photographing unit support arm may include: a moving frame interlocked with the support arm driving means and extended to both sides with respect to the main frame;
And a recording unit installation frame coupled to both ends of the moving frame, respectively. The photographing apparatus for acquiring standard image data for diagnosing severity of inflammatory skin disease.
The method of claim 2,
The support arm driving means includes a coupling block provided in the moving frame, and a linear motor provided in the main frame and coupled to the coupling block to linearly move the moving frame.
The photographing unit support arm further includes folding means configured to foldably couple the photographing unit installation frame to the moving frame,
The folding means, the end of the moving frame and the photographing unit installation frame is rotatably coupled to the shaft, the shaft is configured to be connected to a drive motor that is rotated forward and backward and rotated by receiving the control signal of the control unit Imaging device for obtaining standard image data for diagnosing severity of inflammatory skin disease.
The method according to claim 2 or 3,
A movement guide slot formed in the longitudinal direction of the photographing unit installation frame;
A moving block moved along the moving guide slot and installed with the shooting unit;
A driving motor provided in the main frame or the photographing unit support arm and rotating to receive the control signal of the controller;
And a driving force transmission member for converting the rotational driving force of the driving motor into a linear movement driving force and transferring the rotational driving force to the moving block.
delete delete delete The method of claim 1,
The first neural network includes at least one convolutional layer for extracting a convolutional feature through a convolution operation, and a pulley connected layer connected to one end of the convolutional layer to classify the type or extent of the lesion, wherein the convolutional layer and The weight of the connection strength between neurons in the pulley connected layer is determined in advance, and the arrangement structure and the number of arrangements of the convolutional layer and the pulley connected layer are different from each other according to the type of the lesion to be diagnosed. If possible,
The second neural network is connected to at least one convolutional layer for extracting a convolutional feature through a convolutional operation and at least one of the convolutional layers to calculate a center coordinate of the boundary cell and a probability that the lesion is present in the boundary cell. It includes a pulley connected layer of, and pre-learning by determining the weight of the connection strength between the convolutional layer and the neurons inside the pulley connected layer based on the training data and the correct answer data about the predetermined boundary cell, And a residual block provided with a path connecting the convolutional layer and a residual path crossing at least one of the adjacent convolutional layers and connected to another convolutional layer, wherein the residual block is adjacent to an input of the residual block. Standard image data obtaining device for recording for inflammatory skin disease characterized by severe diagnosis yirueojim to see the type of residual blocks.

Images