KR20190118368A - Standard image information service method and service system for determining inflammatory skin diseases - Google Patents

Abstract

The present invention relates to a method and system for providing a standard image information service for diagnosing severity of inflammatory skin diseases and, more specifically, to a method and system for providing a standard image information service for diagnosing severity of inflammatory skin diseases, which support an ocular inspection of a clinician by providing a service of standardized image information for diagnosis of inflammatory skin diseases such as atopic skin diseases, and perform a rapid and reliable treatment on a patient. According to the present invention, the method for providing a standard image information service for diagnosing severity of inflammatory skin diseases comprises: a photographed image acquiring step of acquiring a photographed image by photographing a lesion affected part of a patient; a processing step of receiving the acquired photographed image and processing the same into standardized information for diagnosing a lesion; and an information transmission step of transmitting the processed standardized information to each registered hospital. In addition, the system for providing a standard image information service for diagnosing severity of inflammatory skin diseases comprises: a lesion photographing device unit for acquiring a photographed image by photographing a lesion affected part of a patient; an image processing unit for receiving the photographed image acquired in the lesion photographing device unit and processing the same into standardized information for diagnosing a lesion; a storage server unit for storing the standardized information processed in the processing unit and transmitting the same to a demand source server unit; and a demand source server unit registered to be connected with the storage server unit and provided in each hospital, which is a source of demand, to receive the standardized information from the storage server unit.

Description

Standard image information service method and service system for severity of inflammatory skin disease {STANDARD IMAGE INFORMATION SERVICE METHOD AND SERVICE SYSTEM FOR DETERMINING INFLAMMATORY SKIN DISEASES}

The present invention relates to a standard image information service method and service system for diagnosing severity of inflammatory skin disease, and more particularly, to standardized image information for diagnosis of inflammatory skin disease such as atopic skin disease. The present invention relates to a standard imaging information service method and service system for diagnosing the severity of inflammatory skin disease, which supports and enables patients to carry out prompt and reliable treatment.

Inflammatory skin skin problem is an inflammatory reaction such as itching, swelling, erythema, or peeling due to various stimulating factors that cause a series of inflammatory reactions in the epithelium of the skin. It refers to the accompanying skin disease.

Such inflammatory skin diseases include allergic dermatitis caused by various causes, bites caused by animals or insects, skin fungus, acne and the like. Among them, allergic dermatitis collectively refers to dermatitis caused by various allergens, including atopic dermatitis, contact dermatitis, urticaria, food allergy, and drug allergy.

Among allergic dermatitis, atopic dermatitis is a serious problem due to the recent increase in the westernization of the diet, housing environment, increased stress, and contaminants due to industrial development.

This atopic dermatitis is an itchy eczema caused by people with atopic dermatitis, and atopic dermatitis, a type of allergic dermatitis, corresponds to 80-90% of pediatric eczema. It also happens a lot. Inflammatory dermatitis, such as atopic dermatitis, is a chronic disease of more than 1 trillion won per year. Atopic dermatitis was observed in 13.5 children per 100 children and 5 adults per 100 years. The average number of patients treated for atopic dermatitis in the past 5 years is 1.44 million. Direct / hepatic beauty per person with atopic dermatitis totals 4,992,000 won, resulting in an annual prevalence rate of 10% and social expenses of 5.8 trillion (Ann Dermtol 2015).

The severity of atopic dermatitis is determined by the clinician's examination, and the follow-up is based on the severity readings and the photographic data of the patient's lesion.

However, in determining the severity of the atopic dermatological disease, the results of the examination differ from clinician, and even if the same clinician performs the examination, the results may vary depending on the external environment such as physical condition.

In other words, atopic dermatitis is currently diagnosed entirely with gross findings, and subjectivity by the clinician is involved, which leads to lack of objectivity. In particular, atopic dermatitis is used to determine the severity of pidgin that determines treatment. It is a difficult task.

The fundamental reason for this problem is that there is no stage in dermatology, imaging and analysis through radiology, which is a general medical practice. Therefore, in order to systematically measure the severity of inflammatory dermatitis in dermatology, there is an urgent need for a device capable of providing standardized imaging, which is being established in radiology.

(Document 0001) Republic of Korea Patent Publication No. 10-1623431 (Registration Date: 2016. 05. 17.) (Document 0002) Republic of Korea Patent Publication No. 10-1015369 (Registration Date: February 10, 2011) (Patent 0003) Republic of Korea Patent Publication No. 10-2015-0059689 (published date: 2015. 06. 02.)

An object of the present invention is to solve the above-mentioned problems, a standard image for the severity of inflammatory skin disease to improve the objective research and diagnostic environment for skin diseases by introducing a new image diagnosis area in the dermatology field An information service method and service system are provided.

Another object of the present invention is to establish an efficient imaging condition for the diagnosis of inflammatory skin diseases, such as atopic skin disease, standard image information service for diagnosing the severity of inflammatory skin disease to be processed and provided as standardized image data The present invention provides a method and a service system.

In order to achieve the above object, the present invention includes a photographed image acquisition step of acquiring a photographed image by photographing the lesion lesion of the patient; A processing step for processing the received captured image into standardized information for diagnosing a lesion; And an information transmitting step of transmitting the processed standardized information to each registered hospital.

The photographing image acquiring step may include a base plate, a main frame erected on an upper surface of the base plate, a photographing unit support arm installed at right angles to the main frame, and a photographing on the main frame. A support arm driving means configured to drive the unit support arm in a liftable manner; a plurality of photographing units provided on the photographing unit support arm; and provided on one side of the main frame to display a predetermined screen under the control of the following controller. And a photographing image obtained by the lesion photographing apparatus including a display unit and a control unit provided in the main frame to control the operation of the support arm driving unit, the photographing unit, and the display unit.

In addition, the support arm driving means includes a coupling block provided in the moving frame, and a linear motor provided in the main frame and coupled to the coupling block to linearly move the moving frame, the photographing unit support arm A moving frame interlocked with the support arm driving means and extending to both sides with respect to the main frame, a shooting unit mounting frame coupled to both ends of the moving frame, and the shooting unit mounting frame with respect to the moving frame. A folding means configured to be foldably coupled, a moving guide slot formed in the longitudinal direction of the photographing unit installation frame, a moving block moved along the moving guide slot and installed with the shooting unit, the main frame or the The photographing unit is provided on the support arm. And a driving motor for receiving a negative control signal and rotating the drive motor, and a driving force transmitting member for converting the rotational driving force of the driving motor into a linear moving driving force and transmitting the rotational driving force to the moving block. The photographing unit installation frame is rotatably coupled to the shaft, and the shaft is configured to be connected to a drive motor that rotates forward and backward and receives the control signal of the controller.

The processing step may include: a feature information calculating step of receiving a captured image of a diagnosis target lesion in the photographing step and calculating feature information for determining a region of interest in the captured image; A region of interest detection step of detecting the region of interest in the photographed image by using the calculated feature information; Information of diagnosing a target lesion from the photographed image using a first neural network using the detected ROI as an input and outputting at least one lesion value as an indicator for diagnosing the type or severity of the lesion. Generating diagnostic information; And a storing step of storing the information generated in the diagnosis information generating step in a server.

In addition, the feature information calculating step, resizing the image to a predetermined size, dividing the resized image to generate a grid cell of a predetermined size, having a center inside the generated grid cell, Generating a plurality of boundary cells representing the probability of the lesion being included, including at least a portion of an image of the target lesion photographed, and receiving the captured image as an input, the coordinates of the center of the boundary cell or the boundary cells in the boundary cell; Calculating a center coordinate of each of the generated boundary cells and a probability of the lesion present in the boundary cell by using a second neural network outputting a probability of existence of the lesion. Grid cells and boundary cells are used, or the probability and the center coordinates are calculated The feature information is calculated using a boundary cell, and the ROI detecting step includes: whether or not the probability that the lesion is present in the boundary cell is greater than or equal to a predetermined threshold value among boundary cells including the captured image of the lesion. In consideration of this, a portion of a boundary cell including the overlapped image of the lesion is removed, and the region of interest is detected by using the removed boundary cell. The plurality of detected photographed images are classified by one or more lesions according to preset criteria, the severity of the lesions included in the classified photographed images is labeled, and some of the photographed images labeled with the severity are included in the training data, Set the rest of the data as verification data, The first neural network is trained on the basis of data and verification data, and the information for diagnosing the lesion in the photographed image is generated using the trained first neural network.

The first neural network may include at least one convolutional layer for extracting a convolutional feature through a convolution operation and a pulley connected layer connected to one end of the convolutional layer to classify the type or extent of the lesion, wherein the convolutional layer And pre-learned by determining weights of connection strengths between neurons within the pulley connected layer, and the arrangement structure and the number of placement of the convolutional layer and the pulley connected layer are different depending on the type of the lesion to be diagnosed. The second neural network is connected to at least one convolutional layer for extracting a convolutional feature through a convolutional operation and one end of the convolutional layer, and has a probability that there is a center coordinate of the boundary cell and the lesion in the boundary cell. Calculate Includes at least one pulley connected layer, and pre-learns by determining weights of connection strengths between the convolutional layer and neurons inside the pulley connected layer based on the training data and the correct answer data about the predetermined boundary cell. And a residual block including a path connecting adjacent convolution layers and a residual path crossing at least one of the adjacent convolution layers to be connected to another convolution layer, wherein the residual block receives an input of the residual block. Reference is made to the input of another adjacent residual block.

In addition, a standard image information service system for diagnosing severity of inflammatory skin disease, comprising: a lesion photographing apparatus unit for capturing a lesion lesion of a patient and acquiring a captured image; An image processing processor configured to receive the captured image acquired by the lesion imaging apparatus and process the processed image into standardized information for diagnosing the lesion; A storage server unit for storing standardized information processed by the processing unit and transmitting the processed standardized information to a demand destination server unit; And a demand destination server unit registered to be associated with the storage server unit and provided to each hospital as a demand destination to receive standardized information from the storage server unit.

In addition, the lesion imaging device unit, the base plate; A main frame installed to stand on an upper surface of the base plate; A photographing unit support arm installed to be perpendicular to the main frame in a lateral direction; Support arm driving means configured to drive the photographing unit support arm in a liftable manner on the main frame; A plurality of photographing units installed on the photographing unit support arms; A display unit provided at one side of the main frame to display a predetermined screen under control of a controller; And a control unit provided in the main frame to control operations of the support arm driving unit, the photographing unit, and the display unit.

In addition, the support arm driving means includes a coupling block provided in the moving frame, and a linear motor provided in the main frame and coupled to the coupling block to linearly move the moving frame, the photographing unit support arm A moving frame interlocked with the support arm driving means and extending to both sides with respect to the main frame, a shooting unit mounting frame coupled to both ends of the moving frame, and the shooting unit mounting frame with respect to the moving frame. A folding means configured to be foldably coupled, a moving guide slot formed in the longitudinal direction of the photographing unit installation frame, a moving block moved along the moving guide slot and installed with the shooting unit, the main frame or the The photographing unit is provided on the support arm. And a driving motor for receiving a negative control signal and rotating the drive motor, and a driving force transmitting member for converting the rotational driving force of the driving motor into a linear moving driving force and transmitting the rotational driving force to the moving block. The photographing unit installation frame is rotatably coupled to the shaft, and the shaft is configured to be connected to a drive motor that is rotated forward and backward and rotated by receiving a control signal of the controller, and the controller controls the support arm driving means to photograph. And a support arm driving control unit for raising and lowering a unit support arm, a shooting control unit for controlling the shooting of the shooting unit, and a display control unit for displaying and guiding a shooting environment for shooting according to control logic of a preset setting condition. .

The image processing unit may include a feature information calculation unit configured to receive a captured image of a lesion to be diagnosed by the lesion imaging unit and calculate feature information for determining a region of interest in the image; A region of interest detection unit for detecting the region of interest in the image using information, and the detected region of interest as an input, and outputting at least one lesion value as an indicator for diagnosing the type or severity of the lesion; And a diagnostic information generator for generating information for diagnosing the target lesion from the input image using a neural network, wherein the feature information calculator resizes the image to a predefined size, and resizes the resized image. A preprocessing unit for generating a grid cell having a predetermined size by dividing a by The boundary cell dependent on the grid cell is generated by using a second neural network that takes a photographed image as an input and outputs a center coordinate of the boundary cell or a probability that the lesion exists in the boundary cell. And a calculation unit configured to calculate a center coordinate of each generated boundary cell and a probability that the lesion exists in the boundary cell, wherein the feature information calculator includes a preprocessed image and a boundary cell from which the probability and the center coordinate are calculated. To calculate the feature information.

In addition, the ROI includes at least a part of an image corresponding to the lesion in the photographed image of the lesion, and the feature information includes a center coordinate of the ROI in the image, and the feature information calculator The feature information is calculated using a plurality of boundary cells having a center inside the generated grid cell and including at least a portion of an image in which the target lesion is photographed and indicating a probability of the lesion being present.

The first neural network may include at least one convolutional layer for extracting a convolutional feature through a convolution operation and a pulley connected layer connected to one end of the convolutional layer to classify the type or extent of the lesion, wherein the convolutional layer And pre-learned by determining weights of connection strengths between neurons within the pulley connected layer, and the arrangement structure and the number of placement of the convolutional layer and the pulley connected layer are different depending on the type of the lesion to be diagnosed. The second neural network is connected to at least one convolutional layer for extracting a convolutional feature through a convolutional operation and one end of the convolutional layer, and has a probability that there is a center coordinate of the boundary cell and the lesion in the boundary cell. Calculate Includes at least one pulley connected layer, and pre-learns by determining weights of connection strengths between the convolutional layer and neurons inside the pulley connected layer based on the training data and the correct answer data about the predetermined boundary cell. And a residual block including a path connecting adjacent convolution layers and a residual path crossing at least one of the adjacent convolution layers to be connected to another convolution layer, wherein the residual block receives an input of the residual block. Reference is made to the input of another adjacent residual block.

As is clear from the above description, the standard image information service method and service system for diagnosing severity of inflammatory skin disease of the present invention can provide standardized image data for diagnosis of inflammatory skin disease such as atopic skin disease. It supports physicians' visits and has the effect of ensuring that patients have accurate and reliable treatment.

In addition, according to the present invention, it is possible to preoccupy the market of atopic dermatitis diagnostic equipment, in which more than 1 million patients spend 1 trillion won or more annually, and furthermore, to secure a technological advantage in the global market, thereby improving national competitiveness in the medical field. It is a very useful invention that can contribute greatly.

1 is a flowchart illustrating a standard image information service method for diagnosing severity of inflammatory skin disease according to the present invention.
Figure 2 is a flow chart showing a process of processing the lesion image taken in the standardized image information service process for the diagnosis of inflammatory skin disease severity according to the present invention.
Figure 3 is a block diagram schematically showing the configuration of the standard image information service system for diagnosing the severity of inflammatory skin disease according to the present invention.
Figure 4 is a perspective view showing an embodiment of the lesion imaging device unit constituting a standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention.
5 is a front view showing an embodiment of a lesion photographing apparatus unit constituting a standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention;
Figure 6 is a side view showing an embodiment of a lesion photographing device unit constituting a standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention.
Figure 7 is a side view showing a folded state of the imaging unit support arm constituting an embodiment of the lesion imaging apparatus unit constituting a standard image information service system for diagnosing inflammatory skin disease severity according to the present invention.
Figure 8 is a block diagram showing the configuration of a control unit constituting an embodiment of the lesion imaging apparatus unit constituting a standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention.
Figure 9 is a block diagram schematically showing a block diagram of the configuration of the image processing unit constituting the standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention.
10 is a diagram illustrating a data processing process in an image processing processor constituting a standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention.
11 is a view showing the structure of the convolutional neural network used by the calculation unit of the image processing unit constituting the standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention;
12 is a view showing a grid cell and a boundary cell generated by the image processing unit using a convolutional neural network, which constitutes a standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention.
Figure 13 is an illustration of the image taken by lesion, severity of atopic dermatitis,
14 is an exemplary view of the image taken by lesion, severity of atopic dermatitis,
15 is a view showing the structure of a convolutional neural network used by the diagnostic information generation unit of the image processing unit constituting the standard image information service system for diagnosing inflammatory skin disease severity according to the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention.

First of all, in adding reference numerals to the components of the drawings, it should be noted that the same reference numerals have the same reference numerals as much as possible even if displayed on different drawings. In addition, in describing the present invention, when it is determined that the detailed description of the related well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

1 is a flowchart showing a standard image information service method for diagnosing inflammatory skin disease severity according to the present invention, and FIG. 2 is a lesion photographed image in a standard image information service process for diagnosing inflammatory skin disease severity according to the present invention. This is a flow chart showing the process of processing to standardized information.

As shown in Figures 1 and 2, the standard image information service method for diagnosing the severity of inflammatory skin disease according to the present invention, the photographed image acquisition step of capturing the lesion lesion of the patient to obtain a photographed image (S100); A processing step (S200) of receiving the captured image acquired in the captured image obtaining step (S100) and processing the processed image into standardized information for diagnosing a lesion; And an information transmission step (S300) of transmitting the standardized information processed in the processing step (S200) to each registered hospital.

The photographing image acquiring step S100 may employ a camera capable of macro photographing lesions of the affected part. Preferably, the photographing image acquisition step S100 may be acquired through a lesion photographing apparatus unit described later. This will be described in detail below.

Next, the processing step (S200) is a feature information calculation step of calculating the feature information for determining a region of interest in the photographed image by receiving a photographed image of the diagnosis target lesion in the photographing step (S100) (S210) ); A region of interest detection step (S220) of detecting the region of interest in the photographed image by using the calculated feature information; A target lesion from the photographed image (input image) by using the first neural network that uses the detected ROI as an input and outputs at least one numerical value as an indicator for diagnosing the type or severity of the lesion. Diagnostic information generating step (S230) of generating the diagnostic information; And a storage step (S230) of storing the information generated in the diagnosis information generation step (S230) in a server.

In the feature information calculating step (S210), the ROI includes at least a part of an image corresponding to the lesion in the photographed image of the lesion, and the feature information includes a center coordinate of the ROI in the image. Is done.

In detail, the feature information calculating step (S210) may resize the image to a predetermined size, divide the resized image to generate a grid cell having a predetermined size, and center the inside of the generated grid cell. And generate a plurality of boundary cells including at least a portion of an image of the target lesion photographed to represent the existence of the lesion, and calculate the feature information using the generated grid cells and boundary cells.

In addition, the feature information calculating step (S210) uses the second neural network as an input of the photographed image and outputs a center coordinate of the boundary cell or a probability that the lesion is present in the boundary cell. By calculating the center coordinates of each of the generated boundary cells and the probability that the lesion is present in the boundary cells, it is possible to calculate the feature information using the probability and the boundary cells calculated from the center coordinates.

Here, the second neural network is connected to at least one convolutional layer for extracting a convolutional feature through a convolutional operation and one end of the convolutional layer to calculate a center coordinate of the boundary cell and a probability that the lesion exists in the boundary cell. It includes at least one pulley connected layer, and is pre-learned by determining the weight of the connection strength between the convolutional layer and the neurons inside the pulley connected layer based on the training data and the correct answer data for the predetermined boundary cell It consists of

The second neural network may further include a residual block provided with a path connecting adjacent convolution layers and a residual path crossing at least one or more adjacent convolution layers and connected to another convolution layer, wherein the residual block includes the residual block. The input of the dual block can be referred to as the input of another adjacent residual block.

Subsequently, in the detection of the ROI (S220), the lesion is photographed in consideration of whether the probability of the lesion being present in the boundary cell among the boundary cells including the image of the lesion being overlapped is greater than or equal to a preset threshold. A portion of the boundary cell including the overlapped image may be removed, and the region of interest may be detected using the remaining boundary cell.

Next, in the generating of the diagnostic information (S230), the plurality of captured images from which the ROI is detected are classified by one or more lesions according to preset criteria, and the severity of the lesions included in the classified captured images is labeled. And setting some of the plurality of photographed images labeled with the severity as learning data and the other as verification data, learning the first neural network based on the learning data and verification data, and learning the first The neural network is used to generate information for diagnosing the lesion in the photographed image.

Here, the diagnosed information may include a lesion, severity, and an image.

In operation S230, the first neural network may be connected to at least one convolutional layer extracting a convolutional feature through a convolution operation and a pulley connected layer connected to one end of the convolutional layer to classify the type or extent of the lesion. It includes, and is learned in advance by determining the weight on the connection strength between the convolutional layer and the neurons inside the pulley connected layer, the arrangement structure and the number of arrangement of the convolutional layer and the pulley connected layer is to be diagnosed It is made of differently provided according to the type of lesion.

Next, the information transmission step (S300) of transmitting the standardized information processed in the processing step (S200) is each of the demand destination (hospital) registered to be associated with the storage server from the storage server for storing the generated diagnostic information The diagnostics information is transmitted to wires and wireless communication to the demand server.

Meanwhile, a standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention will be described with reference to FIG. 3. 3 is a block diagram schematically showing the configuration of a standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention.

Standard image information service system for diagnosing the severity of inflammatory skin disease according to the present invention, as shown in Figure 3, the lesion imaging device unit (A) for capturing the lesion lesion of the patient to obtain a photographed image; An image processing unit (B) for receiving the captured image acquired by the lesion imaging unit (A) and processing the image into standardized information for diagnosing the lesion; A storage server unit (C) for storing the standardized information processed by the processing unit (B) and transmitting the standardized information to the demand destination server unit (D) below; And a demand destination server unit (D) registered to be associated with the storage server unit (C) and provided to each hospital as a demand destination to receive standardized information from the storage server unit (C).

The lesion imaging apparatus unit A will be described with reference to FIGS. 4 to 7. Figure 4 is a perspective view showing an embodiment of the lesion imaging device portion constituting a standard image information service system for diagnosing inflammatory skin disease severity according to the present invention, Figure 5 is a standard image for diagnosing inflammatory skin disease severity according to the present invention FIG. 6 is a front view showing an embodiment of the lesion imaging apparatus unit constituting the information service system, and FIG. 6 is a side view illustrating an embodiment of the lesion imaging apparatus unit constituting the standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention. 7 is a side view showing a folded state of the imaging unit support arm constituting an embodiment of the lesion imaging apparatus unit constituting a standard image information service system for diagnosing inflammatory skin disease severity according to the present invention, Figure 8 Standardized image for diagnosing severity of inflammatory skin disease according to the invention Beam Services is a block diagram showing a configuration of a control unit which constitutes one embodiment lesions up device unit in this system.

The lesion photographing unit A includes a base plate 100 having a predetermined size, as shown in FIGS. A main frame 200 installed to stand on an upper surface of the base plate 100; A photographing unit support arm 300 installed at right angles to the main frame 200 in a lateral direction; Support arm driving means for operating the photographing unit support arm 300 to be movable up and down in the main frame 200; A plurality of photographing units 400 installed on the photographing unit support arms 300; A display unit 500 provided on one side of the front surface of the main frame 100; And a controller 600 provided in the main frame 100 to control operations of the support arm driving means, the photographing unit 400, and the display unit 500.

The base plate 100 is not particularly limited as long as it is a shape or a shape capable of supporting and fixing the main frame 200.

Preferably, the lower surface of the base plate 100 may be configured with a rolling wheel to secure the mobility of the image capturing apparatus for obtaining the standard image data, when moving the image capturing apparatus for obtaining the standard image data for fixing at the installation position It may further include a fixing member which can be fixed by releasing or withdrawing the folded state at the installation position in the folded state or the retracted state.

Subsequently, the main frame 200 may be installed in the base plate 100 in a predetermined length, and may be configured as a housing-type frame in which other components are accommodated or slidably coupled.

Next, the photographing unit support arm 300 is coupled to the support arm driving means to be described later and the moving frame 310 is provided to extend to both sides with respect to the main frame 200, and both ends of the moving frame 310 The photographing unit installation frame 320 is coupled to each other, and the photographing unit installation frame 320 includes a folding means configured to be foldably coupled to the moving frame (310).

The folding means is not particularly limited as long as the photographing unit mounting frame 320 is configured to be folded and unfolded by manual operation with respect to the moving frame 310 or to be automatically folded and unfolded by driving of a motor or the like.

For example, the folding means is rotatably coupled to the recording unit mounting frame 320 to the end of the moving frame 310, the axis is forward and reverse rotation and transmits the control signal of the control unit 600 It may be configured to be connected to a drive motor that receives rotational drive. Accordingly, the controller 600 may be automatically operated based on a manager command input through a control panel constituting the controller 600 or a predetermined control logic (for example, at the start and end of photographing).

In addition, the installation frame 320 of the photographing unit support arm 300 is further provided with a recording unit moving means for moving the photographing unit 400 along its longitudinal direction.

Specifically, the photographing unit moving means is a movement guide slot 321 formed in the longitudinal direction of the installation frame 320, the movement block is moved along the movement guide slot 331 and the shooting unit is installed, and the movement And mounting frame driving means for driving the block.

The installation frame driving means is provided in the main frame 200 or the photographing unit support arm 300, the drive motor for driving the rotation by receiving the control signal of the control unit 600, and linearly rotates the driving force of the drive motor And a driving force transmitting member for converting the moving driving force to the moving block. The driving force transmitting member may be adopted as a rack gear driving method or a pulley interlock driving method.

The control of the driving motor may be automatically performed based on a manager command or a predetermined control logic input through a control panel constituting the controller 600.

Of course, it is possible to employ a configuration that can omit the installation frame driving means and manually move the moving block and then fix it, but it is preferably configured to be automatically moved for accurate and reliable shooting in the left and right photographing unit. Do.

Next, the support arm driving means for operating the recording unit support arm 300 to move up and down in the main frame 200, the coupling block is configured in the moving frame 310, and the main frame 200 And a linear motor coupled to the coupling block and configured to linearly move the moving frame. Here, since the configuration of the linear motor can adopt a known one, a detailed description thereof will be omitted for clarity and simplification of the present invention.

In addition, the plurality of photographing units 400 installed in the photographing unit support arm 300 may employ a camera capable of macro photographing.

The photographing unit 400 may be configured to automatically operate based on a manager (operator) command or a predetermined control logic input through a control panel constituting the controller 600.

Subsequently, the display unit 500 may guide the patient to a posture to be taken for shooting and a shooting situation such as a shooting situation, for example, by a manager's voice during shooting, but the shape of the human body in the display unit 500. Through what position to take through, the recording unit 400 can be made to provide information about what is currently shooting at what position.

Next, the control unit 600 controls the support arm driving means to support the support arm driving control unit 610 for driving the imaging unit support arm 300 up and down, and to control the shooting of the photographing unit 400. The display controller 620 includes a photographing controller 620 and a display controller 630 for displaying and guiding a photographing environment (posture and region to be photographed, etc.) photographed according to a control logic of a preset setting condition.

The support arm driving control unit 610 and the photographing control unit 620 are controlled by a manager's control in a spaced apart manager control panel, and the display unit control unit 630 displays a manager's selection operation as a display environment according to a preset program. Is made.

Here, the control unit 600 may be formed integrally with the image processing unit (B) to be described later. In addition, the following image processing unit (B) may be formed integrally with the storage server unit (C).

Next, the image processing unit B will be described with reference to FIGS. 9 to 11. 9 is a block diagram schematically showing the configuration of an image processing unit constituting a standard image information service system for diagnosing inflammatory skin disease severity according to the present invention, Figure 10 is a diagnosis of inflammatory skin disease severity according to the present invention FIG. 11 is a diagram illustrating a data processing process in an image processing unit configuring a standard video information service system for a system. 11 is a view showing the structure of a convolutional neural network used by the calculation unit of the image processing unit constituting the standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention.

The image processing unit (B) receives a captured image of a lesion to be diagnosed by the lesion imaging unit (A), and calculates feature information for determining feature information for determining a region of interest in the image (710). ), The ROI detection unit 720 that detects the ROI in the image using the calculated feature information, and the detected ROI as input, and at least as an index for diagnosing the type or severity of the lesion. And a diagnostic information generator 730 for generating information for diagnosing the target lesion from the input image by using a first neural network that outputs one lesion value.

In the feature information calculating unit 710, the region of interest includes at least a portion of an image corresponding to the lesion in the photographed image of the lesion, and the feature information includes a center coordinate of the region of interest in the image. do.

The feature information calculator 710 may include a preprocessor 711 for resizing the image to a predetermined size and dividing the resized image to generate a grid cell having a predetermined size. The feature information is calculated.

In detail, the preprocessor 711 may resize the size of the photographed image of the lesion to the size necessary for the second neural network, in order to calculate feature information for extracting the ROI including the lesion. In addition, the preprocessor may generate a predetermined grid cell by dividing the resized image, and may generate a total of 49 grid cells, such as 7 * 7.

In addition, the feature information calculator 710 has a center in the generated grid cell and includes a plurality of boundary cells indicating the probability that the lesion is present, including at least a part of an image of the target lesion. The feature information may be calculated.

In addition, the feature information calculator 710 receives the captured image as an input, and uses a second neural network that outputs a center coordinate of the boundary cell or a probability that the lesion exists in the boundary cell. A calculation unit 712 for generating the boundary cell dependent on the grid cell, and calculating a center coordinate of each of the generated boundary cells and a probability that the lesion exists in the boundary cell, wherein the probability and the The feature information may be calculated by using a boundary cell whose center coordinate is calculated.

Here, the first neural network includes at least one convolutional layer for extracting a convolutional feature through a convolution operation and a pulley connected layer connected to one end of the convolutional layer to classify the type or extent of the lesion, wherein the convolutional layer And pre-learned by determining weights of connection strengths between neurons within the pulley connected layer, and the arrangement structure and the number of placement of the convolutional layer and the pulley connected layer are different depending on the type of the lesion to be diagnosed. It consists of being prepared.

In addition, the second neural network is connected to at least one convolutional layer for extracting a convolutional feature through a convolutional operation and one end of the convolutional layer to calculate a center coordinate of the boundary cell and a probability of existence of the lesion in the boundary cell. It includes at least one pulley connected layer, and is pre-learned by determining the weight of the connection strength between the convolutional layer and the neurons inside the pulley connected layer based on the training data and the correct answer data for the predetermined boundary cell It consists of

The second neural network may further include a residual block provided with a path connecting adjacent convolution layers and a residual path crossing at least one or more adjacent convolution layers and connected to another convolution layer, wherein the residual block includes the residual block. The input of the dual block can be referred to as the input of another adjacent residual block.

The first and second neural networks may use an image recognition algorithm based on neural networks to calculate feature information for setting a region of interest in the photographed image of the target lesion. For example, the image recognition algorithm used by the feature information calculator 710 may be implemented in a neural network to which a YOLO (You Only Look Once) structure is applied. Unlike the neural network to which the YOLO (You Only Look Once) neural network is applied to the conventional Fast R-CNN structure, the YOLO neural network has the advantage of simultaneously classifying the boundary cell position and the boundary cell class at the final output of the neural network.

For example, the second neural network includes a neural network to which the YOLO structure is applied, and further includes a residual block provided with a residual connection, thereby solving the problem of loss of slope, thereby applying the conventional YOLO structure. Improved image recognition performance can be achieved over algorithms using neural networks.

Specifically, the second neural network used by the calculator 712 is connected to at least one convolutional layer 643a for extracting a convolutional feature through a convolutional operation, and one end of the convolutional layer, and the center coordinates of the boundary cell and the It may include at least one pulley connected layer (643b) for calculating the probability that the lesion is present in a boundary cell.

In addition, the second neural network used by the calculator 712 may further include a pooling layer 643c that is alternately arranged with the convolutional layer 643a in addition to the convolutional layer 643a and the pulley connected layer 643b. have.

The second neural network used by the calculation unit 712 is a composite multiply neural network to which the YOLO structure is applied, and includes a residual block provided with a residual connection to at least a part of the convolutional layer 643a included in the neural network. Can be achieved. That is, the second neural network used by the calculator may be implemented using an Atopic Region Detection (ARD) algorithm in which a residual connection is applied in addition to the YOLO algorithm.

At least a portion of the convolutional layer 643a of the second neural network is a path connecting the adjacent convolutional layer 643a and a residual path connected to another convolutional layer 643a by skipping at least one adjacent convolutional layer 643a. It includes a residual block is provided, the input of the residual block may be referred to as the input of another adjacent residual block.

In addition, the second neural network used by the calculator 712 may determine a weight of the connection strength between the convolutional layer and the neurons inside the pulley connected layer based on the training data and the correct answer data about the predetermined boundary cell. Can be learned in advance.

Next, a process in which the specific information calculating unit 710 of the image processing unit calculates the feature information using the second neural network will be described. FIG. 12 is a diagram illustrating grid cells and boundary cells generated by an image processing unit constituting a standard image information service system for diagnosing severity of inflammatory skin disease according to the present invention.

The preprocessor 711 of the feature information calculator 710 receives an image of a diagnosis target lesion, and the preprocessor 720 resizes the input photographed image and generates grid cells having a predetermined interval.

The calculation unit 712 of the feature information calculator 710 has a center coordinate inside the grid cell using a second neural network, and any number of boundary cells S1, S2, S3, and S4 that are dependent on the grid cell. , And at the same time, classify the class of the image included in the boundary cell.

The boundary cells S1, S2, S3, S4, and S5 generated by the calculation unit 712 using the second neural network may be specified by the center coordinates of the boundary cells or the relative size of the boundary cells in the entire resized image. have. In addition, the boundary cells S1, S2, S3, S4, and S5 generated by the calculation unit 712 using the second neural network, respectively, have information on the probability that the lesion image is present inside the boundary cell or the lesion image. May have probability information on what type of lesion is present.

For example, the preprocessing unit 711 of the feature information calculating unit 710 generates grid cells having a width of 7 * 7 in a resized image, and boundary cells S1, S2, and S3 that are dependent on each grid cell. If the number of S4 and S5 is 2, a total of boundary cells generated by the calculator 712 may be generated in total of 7 * 7 * 2.

Next, the ROI detection unit 720 of the image processing unit B determines whether the probability of the lesion present in the boundary cell among the boundary cells including the captured image of the lesion is greater than or equal to a preset threshold. In consideration of this, a part of the boundary cell including the overlapped image of the lesion is removed, and the feature information is calculated using the remaining boundary cell.

In detail, the region of interest detector 720 detects the region of interest in the image using the calculated feature information. The ROI is an area including at least a part of an image corresponding to the lesion among the photographed images of the lesion, and boundary cells S1, S2, and S3 generated by the calculator 712 of the feature information detector 710. , S4 and S5).

For example, the ROI detector 720 may include boundary cells S1, S2, S3, S4, and S5 of which the probability that a lesion exists in a boundary cell generated by the calculator of the feature information calculator 710 is different. The boundary cells S1 and S2 having the highest probability of having a lesion therein may be detected as the ROI. For this purpose, among the boundary cells S1 and S2 in which any one lesion existing in the photographed image overlaps the image, the lesion is considered in consideration of whether or not the probability that the lesion exists in the boundary cell is greater than or equal to a preset threshold. A part of the boundary cell S2 including the captured image may be removed.

For example, the ROI detection unit 720 may determine that the probability that the lesion exists in the boundary cell is not greater than or equal to a preset threshold value among boundary cells S2 and S3 including the lesion image at the lower right of FIG. 12. It can remove (here, remove boundary cell S3). Accordingly, the ROI detector 720 may detect one boundary cell S1 including the lesion image shown in the lower right of FIG. 12 as the ROI.

In addition, the ROI detector 720 may use only one boundary cell by using a Nom-maximal Suppression Non-Maximum Suppression (NMS) algorithm to remove a boundary cell having a probability that a lesion exists in the cell below the set threshold. I can leave it. That is, the ROI detection unit 720 may detect any lesion image on the image of the lesion by using the NMS algorithm when the boundary cells of the plurality of boundary cells generated by the calculation unit exist with overlapping lesions on the captured image. Only one boundary cell can be left per. The ROI detector 720 may detect one boundary cell remaining per lesion image as the ROI.

Next, the diagnostic information generator 730 classifies the plurality of photographed images from which the ROI is detected by one or more lesions according to preset criteria, and labels the severity of the lesions included in the classified photographed images. And the lesion is diagnosed using the first neural network learned based on the labeled image. Figure 13 is an illustration of the image taken by lesion, severity of atopic dermatitis, Figure 14 is an illustration of the image taken by lesion, severity of atopic dermatitis, Figure 15 is for the diagnosis of inflammatory skin disease severity according to the present invention FIG. 9 is a diagram illustrating a convolutional neural network used by the diagnostic information generation unit of the image processing unit constituting the standard image information service system.

The diagnostic information generator 730 sets some of the plurality of photographed images labeled with the severity as learning data and others as verification data, and the convolutional layer and the pulley based on the learning data and verification data. The first neural network may be trained by determining weights of connection strengths between neurons within a connected layer, and the lesion may be diagnosed in the photographed image by using the learned first neural network.

Here, the diagnosis information generator 730 learning the first neural network is based on the weight of the first neural network, and the actual value of the first neural network derived based on the output of the first neural network and the verification data according to the weight. The first neural network is trained in consideration of the running rate and the number of learning repetitions to minimize the loss function defining the difference in output.

Subsequently, the diagnostic information generator 730 combines the first neural network learned differently according to the type of the lesion in consideration of the type of the lesion to be diagnosed, and considers the type of the lesion. The neural network is used to generate information for diagnosing the lesion.

Specifically, the lesions associated with atopic dermatitis diseases include erythema, papulation, scraping, excoriation and lichenification.

The diagnostic information generating unit 730 of the image processing unit B automatically detects a region of interest in which the lesion is mainly included in the photographed image including the lesion of atopic dermatitis, and types of lesions in the detected region of interest and the type of lesion. The severity can be objectively diagnosed, and the severity of each lesion can be classified according to the purpose and criteria of diagnosis.

The diagnostic information generator 730 of the image processing unit B may receive the detected ROI, and may diagnose the type and severity of the lesion included in the ROI input using the pre-learned first neural network. .

The first neural network and the second neural network may be provided as neural networks to which an image recognition algorithm based on a synthetic multiplicative neural network is applied.

Although the diagnostic information generator 730 of the image processing unit B may generate a diagnostic model using the first neural network trained by dividing the lesions, the diagnosis information generator 730 may diagnose the diagnosis using the trained first neural network without dividing each lesion. A model can be created and the lesion can be diagnosed using the generated diagnostic model. When the diagnostic information generator 730 of the image processing unit B generates the diagnostic model using the first neural network trained by the lesions, the diagnostic information generator 730 is generated by using the first neural network trained by the lesions. A new combination diagnostic model can be generated by combining at least one diagnostic model.

In addition, the diagnostic information generating unit 730 of the image processing unit B classifies the plurality of captured images from which the ROI is detected by one or more lesions according to a preset criterion, and includes the lesions included in the classified captured images. Label the severity of.

For example, the diagnostic information generator 730 may include a plurality of photographed images in which atopic dermatitis lesions are photographed to generate training data for learning a first neural network and verification data for verifying the learned first neural network. Detects the ROI that mainly contains the lesion, classifies the plurality of captured images detected by the ROI into three types by lesion type, and indicates the lesion that appears in the captured image where the ROIs classified according to the lesion type are detected The severity of the lesion is labeled as '0', severity '1', severity '2', and severity '3', and at least some of the labeled photographed images are set as learning data and the rest as correct data, and the learning data And the first neural network based on the correct answer data.

The diagnostic information generating unit 730 of the image processing unit B may use 70% of the labeled photographed images as training data and 30% as verification data. For example, in the present invention, the diagnostic information generator 730 uses 1400 learning data for learning erythema and scratches, and uses 308 learning data for lichenification. Done. In addition, the neural network can be trained in advance by utilizing Tensorflow, which can easily create an artificial neural network model in various hardware environments for smooth lesion-specific model learning.

Subsequently, the diagnostic information generator 730 connects the neuron within the convolutional layer and the pulley connected layer based on the training data set as at least a part of the image of the diagnosis target lesion and the verification data set as the remaining part. The first neural network is trained by determining weights for strengths.

In the present invention, the first neural network includes at least one convolutional layer for extracting a convolutional feature through a convolution operation, and a pulley connected layer connected to one end of the convolutional layer to classify the type or extent of the lesion, wherein the convolutional layer And the arrangement structure and the number of arrangements of the pulley connected layers may be provided differently according to the type of the lesion to be diagnosed.

For example, the 'Gradient Descent' method may be used to adjust the weight of the connection strength between the convolutional layer in the neural network and the neuron in the fully connected layer.

[Equation 1]

는 가중치,

는 가중치를 입력으로 하는 손실 함수(Loss Function),

는 손실 함수의 기울기(gradient),

는 러닝 레이트(learning rate)를 의미한다 상기 수학식 1은 학습부(340)가 경사 하강법(gradient descent)을 이용하여 손실 함수를 최소화하는 가중치를 결정하는 과정을 나타낸다. 예를 들어, 진단정보 생성부(730)는 현재 가중치에서 손실 함수의 기울기의 음수에 비례하는 만큼 이동하는 것을 반복하여 손실 함수 를 최소화 하는 가중치를 결정할 수 있다.here,

Is the weight,

Is a loss function with weights as input,

Is the gradient of the loss function,

Equation 1 represents a process in which the learner 340 determines a weight for minimizing a loss function by using a gradient descent. For example, the diagnostic information generator 730 may determine the weight to minimize the loss function by repeatedly moving the current weight as much as the negative number of the slope of the loss function.

In addition, the diagnostic information generation unit 730 takes a weight input and defines a difference between the output of the first neural network according to the weight and the actual output of the first neural network derived based on the verification data. The first neural network may be trained in consideration of the running rate and the number of learning repetitions to minimize the learning rate. The running rate of the present invention may be experimentally determined to minimize the loss function, and may have a value in the range of 001 to 0001. In addition to the running rate, the loss function may have another function output according to the number of learning iterations. The learning unit 340 may determine a weight in consideration of the number of learning iterations in order to minimize the loss function. That is, the diagnostic information generator 730 of the present invention may learn the first neural network in consideration of the running rate and the number of learning repetitions to minimize the loss function.

Subsequently, the diagnostic information generator 730 may learn a neural network based on the training data and the verification data provided for each lesion, and generate a diagnostic model for each lesion. For example, the erythema diagnosis model may be trained to determine only the severity of the erythema based on the training data and diagnostic data related to the erythema, and only the severity of the thymus is determined based on the training data and the diagnostic data related to lichenification. You can train the diagnostic model. In the present invention, the diagnostic information generator 730 may not only learn a diagnostic model for diagnosing the severity of each lesion, but also learn a diagnostic model for diagnosing both the type and the severity of the lesion.

In addition, the diagnosis information generating unit 730 combines the types of lesions to be diagnosed to be differently learned according to the types of the lesions. For example, a combination diagnosis model may be generated by combining a diagnosis model according to the first neural network learned for each lesion, and the lesion diagnosis apparatus 10 may diagnose the lesion using the combined diagnosis model.

On the other hand, the storage server unit (C) and the customer destination server unit (D) is configured to include a transmission and reception module capable of transmitting and receiving mutual information through wired and wireless communication.

According to the imaging device for acquiring standard image data for diagnosing inflammatory skin disease severity as described above, it is possible to provide standardized image data for diagnosis of inflammatory skin disease such as atopic skin disease. There is an advantage in supporting vision and enabling patients to involve accurate and rapid treatment.

In addition, according to the present invention, it is possible to preoccupy the market of atopic dermatitis diagnostic apparatus, in which more than 1 million patients spend 1 trillion won or more annually, and furthermore, by securing technological advantages in the global market, improving national competitiveness in the medical field. There is an advantage to this.

The above description is merely illustrative of the technical spirit of the present invention, and those skilled in the art to which the present invention pertains various modifications, changes, and substitutions without departing from the essential characteristics of the present invention. will be. Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical spirit of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by the embodiments and the accompanying drawings. . The scope of protection of the present invention should be interpreted by the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: base plate, 200: main frame
300: shooting unit support arm, 310: moving frame
320: shooting unit installation frame, 321: moving guide slot
400: shooting unit, 500: display unit
600: control unit, 610: support arm drive control unit
620: shooting control unit, 630: display control unit
710: specific information calculating unit, 711: preprocessing unit
712: calculator, 720: ROI detector
730: diagnostic information generation unit, A: lesion imaging unit
B: image processing unit, C: storage server unit
D: demand server unit, S100: lesion image acquisition step
S200: processing step with standardized information, S210: calculating specific information
S220: detection of region of interest, S230: generation of diagnosis information of target lesion
S240: diagnostic information storage step, S300: transmission step

Claims (12)

A photographed image obtaining step of obtaining a photographed image by photographing a lesion lesion of a patient;
A processing step for processing the received captured image into standardized information for diagnosing a lesion; And,
Information transmission method for transmitting the processed standardized information to each registered hospital; Standard image information service method for diagnosing the severity of inflammatory skin disease comprising a.
The method of claim 1,
The photographed image acquisition step,
A base plate, a main frame erected on an upper surface of the base plate, a photographing unit support arm installed at right angles to the main frame, and a photographing unit support arm driven on the main frame to be lifted and lowered. A support unit configured to support the arm, a plurality of photographing units installed on the photographing unit support arm, a display unit provided at one side of the main frame, and configured to display a predetermined screen under the control of the following controller; Standard image information service method for diagnosing severity of inflammatory skin disease, characterized by acquiring a captured image by a lesion photographing apparatus including a control arm configured to control operations of the support arm driving means, the photographing unit, and the display unit. .
The method of claim 2,
The support arm driving means includes a coupling block provided in the moving frame, and a linear motor provided in the main frame and coupled to the coupling block to linearly move the moving frame.
The photographing unit support arm includes a moving frame interlocked with the support arm driving means and extended to both sides with respect to the main frame, a photographing unit installation frame coupled to both ends of the moving frame, and the photographing unit installation frame. A folding means configured to be foldably coupled to the moving frame, a moving guide slot formed in the longitudinal direction of the photographing unit installation frame, a moving block moved along the moving guide slot, and the shooting unit installed; A driving motor provided in the main frame or the photographing unit support arm and rotating to receive the control signal of the controller, and a driving force transmitting member converting the rotation driving force of the driving motor into a linear moving driving force and transmitting the linear driving force to the moving block. Including,
The folding means is rotatably coupled to the end of the moving frame and the photographing unit installation frame, the shaft is configured to be connected to a drive motor that is rotated forward and backward and rotated by receiving the control signal of the control unit Standard image information service method for diagnosing severity of inflammatory skin disease.
The method of claim 1,
The processing step is,
A feature information calculating step of receiving feature images of the lesion to be diagnosed in the photographing step and calculating feature information for determining a region of interest in the captured image;
A region of interest detection step of detecting the region of interest in the photographed image by using the calculated feature information;
Information of diagnosing a target lesion from the photographed image using a first neural network using the detected ROI as an input and outputting at least one lesion value as an indicator for diagnosing the type or severity of the lesion. Generating diagnostic information; And,
And a storage step of storing the information generated in the diagnosis information generation step in a server.
The method of claim 3,
The calculating of the feature information may include resizing the image to a predefined size, dividing the resized image to generate a grid cell having a predetermined size, and having a center inside the generated grid cell, wherein the target lesion is present. Generating a plurality of boundary cells representing the probability of the lesion being included, including at least a portion of the captured image, and receiving the captured image as an input, wherein the center coordinates of the boundary cell or the lesion in the boundary cell Calculating a center coordinate of each of the generated boundary cells and a probability that the lesion is present in the boundary cell by using a second neural network outputting a probability of existence; And a boundary using a boundary cell or the probability and the center coordinates calculated. The feature information is calculated using a recell,
In the detecting of the ROI, the image of the lesion may be overlapped by considering whether the probability of the lesion being present in the boundary cell among the boundary cells including the image of the lesion overlapping is greater than or equal to a preset threshold. Removing a part of a boundary cell including the cell, and detecting the region of interest using the remaining boundary cell.
The generating of the diagnostic information may include classifying the plurality of photographed images from which the ROI is detected by one or more lesions according to preset criteria, labeling the severity of the lesions included in the classified photographed images, and labeling the severity. Some of the plurality of captured images are set as training data and the other part as verification data, and the first neural network is trained based on the learning data and the verification data, and the photographing is performed using the learned first neural network. Standard image information service method for diagnosing severity of inflammatory skin disease, characterized in that to generate information for diagnosing the lesion in the image.
The method of claim 5,
The first neural network includes at least one convolutional layer for extracting a convolutional feature through a convolution operation, and a pulley connected layer connected to one end of the convolutional layer to classify the type or extent of the lesion, wherein the convolutional layer and The weight of the connection strength between neurons in the pulley connected layer is determined in advance, and the arrangement structure and the number of arrangements of the convolutional layer and the pulley connected layer are different from each other according to the type of the lesion to be diagnosed. Is made possible,
The second neural network is connected to at least one convolutional layer for extracting a convolutional feature through a convolution operation and at least one end of the convolutional layer to calculate a center coordinate of the boundary cell and a probability that the lesion exists in the boundary cell. A single pulley connected layer, and pre-learning by determining weights of connection strengths between the convolutional layer and the neurons inside the pulley connected layer based on the training data and the correct answer data about the predetermined boundary cell, And a residual block provided with a path connecting adjacent convolutional layers and a residual path crossing at least one of the adjacent convolutional layers to be connected to another convolutional layer, wherein the residual block is adjacent to an input of the residual block. Standard image information service method for diagnosing severity of inflammatory skin disease, characterized in that it is referred to as an input of another residual block.
A lesion photographing apparatus unit which acquires a photographed image by photographing a lesion lesion of a patient;
An image processing processor configured to receive the captured image acquired by the lesion imaging apparatus and process the processed image into standardized information for diagnosing the lesion;
A storage server unit for storing standardized information processed by the processing unit and transmitting the processed standardized information to a demand destination server unit; And,
Standard image information service system for diagnosing severity of inflammatory skin disease, comprising: a demand destination server unit registered to be associated with the storage server unit and provided in each hospital as a demand destination to receive standardized information from the storage server unit. .
The method of claim 7, wherein
The lesion imaging device unit,
Base plate;
A main frame installed to stand on an upper surface of the base plate;
A photographing unit support arm installed to be perpendicular to the main frame in a lateral direction;
Support arm driving means configured to drive the photographing unit support arm in a liftable manner on the main frame;
A plurality of photographing units installed on the photographing unit support arms;
A display unit provided at one side of the main frame to display a predetermined screen under control of a controller; And,
And a control unit provided in the main frame to control operations of the support arm driving unit, the photographing unit, and the display unit.
The method of claim 8,
The support arm driving means includes a coupling block provided in the moving frame, and a linear motor provided in the main frame and coupled to the coupling block to linearly move the moving frame.
The photographing unit support arm may include a moving frame interlocked with the support arm driving means and extended to both sides with respect to the main frame, a photographing unit installation frame coupled to both ends of the moving frame, and the photographing unit installation frame. A folding means configured to be foldably coupled to the moving frame, a moving guide slot formed in the longitudinal direction of the photographing unit installation frame, a moving block moved along the moving guide slot, and the shooting unit installed therein; And a driving motor provided in the main frame or the photographing unit support arm and configured to receive a control signal from the controller, and to rotate the driving motor, and convert the rotation driving force of the driving motor into a linear movement driving force and transmit the driving force to the moving block. Including members,
The folding means is rotatably coupled to the end of the moving frame and the photographing unit installation frame, the shaft is configured to be connected to a drive motor that is rotated forward and backward and rotated by receiving the control signal of the controller,
The control unit controls the support arm driving means for photographing according to a support arm driving control unit for driving the photographing unit support arm up and down, a photographing control unit for controlling photographing of the photographing unit, and control logic of a preset setting condition. Standard image information service system for diagnosing severity of inflammatory skin disease, characterized in that it comprises a display control unit for displaying and guiding the shooting environment.
The method of claim 7, wherein
The image processing unit may include a feature information calculation unit configured to receive a captured image of a diagnosis target lesion by the lesion photographing device unit, and to calculate feature information for determining a region of interest in the image; A first neural network using the ROI detector for detecting the ROI in the image and the detected ROI, and outputting at least one lesion value as an indicator for diagnosing the type or severity of the lesion. A diagnostic information generator for generating information for diagnosing the target lesion from the input image using a neural network;
The feature information calculator may be configured to: resize the image to a predetermined size, divide the resized image, and generate a grid cell having a predetermined size, and the photographed image as an input, and the center of the boundary cell. The boundary cell is dependent on the grid cell by using a second neural network outputting a coordinate or a probability that the lesion is present in the boundary cell, and the center coordinates of the generated boundary cells and the center coordinates of the boundary cells are generated. A calculation unit for calculating a probability that the lesion is present in a boundary cell,
And the feature information calculating unit calculates the feature information using the preprocessed image and the boundary cell from which the probability and the center coordinates are calculated.
The method of claim 10,
The region of interest includes at least a portion of an image corresponding to the lesion among the photographed images of the lesion, wherein the feature information includes a center coordinate of the region of interest in the image,
The feature information calculating unit may calculate the feature information by using a plurality of boundary cells having a center inside the generated grid cell and including at least a portion of an image in which the target lesion is photographed to indicate a probability of the lesion being present. Standard image information service system for diagnosing severity of inflammatory skin disease.
The method according to claim 10 or 11, wherein
The first neural network includes at least one convolutional layer for extracting a convolutional feature through a convolution operation, and a pulley connected layer connected to one end of the convolutional layer to classify the type or extent of the lesion, wherein the convolutional layer and The weight of the connection strength between neurons in the pulley connected layer is determined in advance, and the arrangement structure and the number of arrangements of the convolutional layer and the pulley connected layer are different from each other according to the type of the lesion to be diagnosed. Is made possible,
The second neural network is connected to at least one convolutional layer for extracting a convolutional feature through a convolution operation and at least one end of the convolutional layer to calculate a center coordinate of the boundary cell and a probability that the lesion exists in the boundary cell. A single pulley connected layer, and pre-learning by determining weights of connection strengths between the convolutional layer and the neurons inside the pulley connected layer based on the training data and the correct answer data about the predetermined boundary cell, And a residual block provided with a path connecting adjacent convolutional layers and a residual path crossing at least one of the adjacent convolutional layers to be connected to another convolutional layer, wherein the residual block is adjacent to an input of the residual block. Standard image information service system for diagnosing severity of inflammatory skin disease, characterized in that it is referred to as an input of another residual block.

Images