JP2023177589A - Image processing apparatus and image processing method - Google Patents

Abstract

To provide an image processing apparatus and an image processing method for presenting photographing guidance suitable for a method for evaluating a skin disease and guidance to cope with the skin disease based on image data acquired through photographing.SOLUTION: In an image processing apparatus 101, an operation unit functions as estimation means that estimates the type of a skin disease of a patient from image data generated by photographing the patient by an imaging apparatus, determination means that determines a method for evaluating the skin disease according to the estimated type, evaluation means that evaluates the severity of the skin disease from the image data by using the determined evaluation method, and guide means that outputs at least one of photographing guidance according to the determined evaluation method and coping guidance according to the evaluated severity.SELECTED DRAWING: Figure 2

Description

The present invention relates to a technique for providing information that assists in photographing by processing an image of an affected area acquired by photographing.

As an imaging device that supports diagnosis using image processing technology, Patent Document 1 discloses an imaging device that includes a process for capturing information necessary for disease analysis when capturing an image of a patient's affected area. has been done.

JP 2019-213873 Publication

However, the imaging apparatus of Patent Document 1 does not provide a photographing method for acquiring necessary and sufficient images for diagnosis or the like, nor does it provide support for photographing. For this reason, it is not possible to obtain images necessary for diagnosis according to the type of disease, etc., making it difficult to perform accurate diagnosis or to determine an appropriate response to the diagnosis result.

The present invention provides an image processing device and an image processing method that can present imaging guidance and treatment guidance for skin diseases that are appropriate for a skin disease evaluation method based on image data acquired through imaging.

An image processing device according to one aspect of the present invention includes an estimation means for estimating the type of skin disease of a patient from image data generated by photographing the patient, and a method for evaluating the skin disease according to the estimated type. A determination means for determining, an evaluation means for evaluating the severity of skin disease using the determined evaluation method from image data, imaging guidance according to the determined evaluation method, and treatment guidance according to the evaluated severity. and a guide means for outputting at least one of them. Note that an imaging device including the above image processing device and an imaging element that images a patient also constitutes another aspect of the present invention.

Further, an image processing method as another aspect of the present invention includes the steps of estimating the type of skin disease of a patient from image data generated by photographing the patient, and evaluating the skin disease according to the estimated type. A step of determining the method, a step of evaluating the severity of the skin disease using the determined evaluation method from the image data, and a step of providing imaging guidance according to the determined evaluation method and treatment guidance according to the evaluated severity. and outputting at least one of them. Note that a program that causes a computer to execute processing according to the above image processing method also constitutes another aspect of the present invention.

According to the present invention, it is possible to present imaging guidance and treatment guidance for skin diseases that are appropriate for a skin disease evaluation method based on image data acquired through imaging.

FIG. 1 is a diagram showing the configuration of an image processing system in an embodiment of the present invention. FIG. 1 is a diagram showing an image processing device in an example. FIG. 1 is a diagram showing an imaging device in an example. The figure which shows the relationship between the disease and its evaluation method in an Example. 5 is a flowchart showing processing executed by the diagnostic system in the embodiment. The figure which shows the example of the photography guidance in an Example. Flowchart showing processing in an example. FIG. 3 is a diagram showing external specifications of an imaging device in an example. The figure which shows the example of a part estimation result in an Example. FIG. 7 is a diagram showing another external specification of the imaging device in the example. FIG. 7 is a diagram showing still another external specification of the imaging device in the example.

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows the configuration of an image processing system including an image processing apparatus 101 according to an embodiment. The imaging device 102 photographs a subject (affected part of a patient) to generate visible light image data, and transmits the image data to the image processing device 101 via the network 103. The image processing device 101 is constituted by a personal computer, a server computer, or the like, and performs image processing, which will be described later, on received image data. The imaging device 102 is connected to the network 103 as soon as the power is turned on, and communication with the image processing device 101 is enabled via the network 103. This communicable state continues until the power of the imaging device 102 is turned off.

The network 103 performs data transmission, various controls, etc. wirelessly or by wire. The network 103 may use various communication protocols such as Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Picture Transfer Protocol (PTP), Universal Serial Bus (USB), Local Area Network (LAN), etc. can.

The imaging device 102 may be a normal monocular camera or a compound eye camera such as a stereo camera. By transmitting the three-dimensional image data generated by the compound eye camera to the image processing device 101, it becomes possible to improve the accuracy when estimating the area, shape, etc. of a disease in the image processing device 101.

As image processing (image analysis), the image processing device 101 estimates a skin disease from the image data of the affected area, and sends information such as imaging conditions for more suitable imaging from the estimation results to the imaging device 102. or send it. The photographing conditions include the composition of the subject and photographing parameters (focus position, zoom position, ISO sensitivity, exposure, Tv value, Av value, white balance, strobe light emission, color adjustment processing, edge enhancement processing, etc.). In addition, the image processing device 101 performs image processing such as determining whether image data necessary for determining the severity of a disease, etc. has been collected, and analyzing the image data to determine the degree of severity of the patient. FIG. 2 shows the configuration of the image processing apparatus 101. The image processing device 101 is connected to the imaging device 102 shown in FIG. 1 and another imaging device 104 via a network 103. The image processing device 101 also includes an input/output display device 201 configured with a liquid crystal display panel, etc., and an operation input device 202 for performing various operation inputs such as a keyboard, a mouse, a touch panel, an input operation controller, and a gesture input device. is connected. The input/output display device 201 and the operation input device 202 mainly constitute a user interface. By using these user interfaces, settings of the image processing apparatus 101 can be changed and programs can be updated. The user interface does not need to be connected to the image processing apparatus 101 at all times, and may be separated from the image processing apparatus 101 if the input/output display device 201 and the operation input device 202 are unnecessary. Note that a user interface function may be provided in the imaging device 102 so that operation input can be performed from the imaging device 102 to the image processing device 101.

Although not shown in FIG. 2, the image processing system may be provided with a lighting device that illuminates the subject. The lighting device has a light source such as a halogen lamp or a light emitting diode.

It is possible to communicate with the imaging devices 102 and 104, the input/output display device 201, and the operation input device 202 via an interface (I/F) 203 arranged on the internal bus of the image processing device 101. Each interface 203 is composed of a network interface, a serial communication interface, and the like.

The image processing device 101 includes an arithmetic unit 204 as a processing means configured by a CPU configured as a general-purpose microprocessor, an image processing processor, or the like. The calculation unit 204 is connected to the storage unit 205 via an internal bus (data bus, address bus, other control lines, etc.).

The storage unit 205 is composed of nonvolatile memory devices such as Read only memory (ROM), Random Access Memory (RAM), and Electrically Erasable Programmable Read-Only Memory (EEPROM), file areas on external storage devices, virtual storage areas, etc. Ru. The external storage device is composed of a semiconductor element, a hard disk, etc., and is connected to the interface 203. A processing data storage area 206 included in the storage unit 205 is configured by a RAM area, a file area of an external storage device, a virtual storage area, etc., and is used to temporarily store image data to be processed and set parameters for image processing. to remember. Furthermore, the storage unit 205 stores an image processing program 207 for executing image processing. The image processing program 207 changes image processing settings and executes image processing in response to various operation inputs through the operation input device 202 and the like. Further, the changed settings can be saved in the processed data storage area 206 or deleted from the processed data storage area 206.

The image processing program 207 includes an image processing unit 208 that executes image processing, and an image processing setting unit 210 that determines settings and operations of the image processing unit 208 in response to various operation inputs through the operation input device 202 and the like.

The image processing unit 208 is a main body of a program for executing image processing implemented in the image processing library 209. The image processing library 209 is provided in the storage unit 205 as a statically or dynamically linked library. The image processing implemented in the image processing library 209 includes inference processing using a machine learning model, and analysis processing and inference processing using information such as brightness, color, hue, and gradient of image data. There may be. Furthermore, inference processing may be performed using feature matching based on shape feature information such as the gradation distribution and brightness gradient of image data. For example, pixels with a brightness gradient above a certain level may be extracted as edges, and rule-based inference processing may be performed based on the position, center of gravity, inclination, etc. of the edges.

Furthermore, the image processing program 207 includes functional units (input/output routines) corresponding to various functions: an external device control unit 211, a storage data generation unit 212, an instruction reception unit 213, a temporary storage unit 214, a display screen generation unit 215, a storage It includes a data output section 216 and an operation reception section 217.

The external device control unit 211 sets or changes imaging conditions for the imaging devices 102 and 104. The storage data generation unit 212 generates temporary or permanent storage data from the image data generated by the imaging devices 102 and 104. The instruction accepting unit 213 and the operation accepting unit 217 accept instructions and operations through the operation input device 202 and the like. The temporary storage 214 includes a RAM area, a cache area of the calculation unit 204, and the like. Display screen generation 215 generates display image data to be displayed on the monitors of the input/output display device 201 and the imaging devices 102 and 104. These functional units are implemented in the storage unit 205 in the form of application (utility) programs, subroutines configured as statically or dynamically linked libraries, and the like.

By executing the image processing program 207, the image processing apparatus 101 can control the imaging apparatuses 102 and 104 and perform image processing using the calculation unit 204. The calculation unit 204 performs image processing by calling the image processing unit 208 of the image processing program 207, the image processing library 209, and each functional unit in response to instructions and operations from the imaging device 102 and the operation input device 202. The image processing result is then output to the input/output display device 201, the imaging devices 102 and 104, or the data storage area 206. The calculation unit 204 may also store (log) the results of image processing in an external storage device.

Note that in this embodiment, the image processing device 101 is configured separately from the imaging device (102, 104), but the image processing device may be built into the imaging device. The image processing device 101 may also be provided as a remote server device using an Internet protocol suite that interconnects multiple computer networks.

FIG. 3 shows the configuration of a digital still camera as the imaging devices 102 and 104. A digital still camera performs photographing processing by executing a program. An imaging unit 300 captures an optical image (subject image) using a solid-state imaging device such as a CCD sensor or a CMOS sensor to generate image data. The CPU 301 controls the overall operation of the digital still camera. The ROM 302 stores programs that execute initial processing, basic input/output processing, and photographing processing that are executed when the power is turned on.

RAM 303 functions as the main memory of CPU 301. A program stored in the ROM 302 is loaded into the RAM 303 . Further, the RAM 303 provides a work area when the CPU 301 executes various processes.

The display device 304 is a rear monitor configured with a liquid crystal display panel or the like, and displays image data generated by the imaging unit 300 and various information regarding photography. The input device 305 has operation members such as buttons and dials for various operations by the photographer, and outputs signals corresponding to the operations to the CPU 301. The operating members include a shooting button for instructing shooting, a dial for making various settings related to shooting, a touch panel provided on the display device 304, and the like.

The media drive 306 stores image data in a removable storage medium and reads stored image data. Interface 307 is connected to network 103 via communication line 309. The CPU 301 can communicate with the image processing apparatus 101 through the interface 307. A system bus (address bus, data bus, and control bus) 308 transmits and receives signals and data between the above-mentioned components.

The CPU 301 temporarily stores image data generated by the imaging unit 300 and metadata related to the image data (shooting time, shooting position, image format, shooting parameters, etc.) in the RAM 303. The image processing unit 311 performs image conversion processing to convert the image data stored in the RAM 303 into image data that matches human visual characteristics and image data that includes three-dimensional data. Note that if the load on the image processing unit 311 for this image conversion process is high, the raw image data before the image conversion process is downloaded to the image processing apparatus 101, and the image processing apparatus 101 performs the above image conversion process. Good too. The file generation unit 312 generates files of raw image data, moving image data, and still image data before image conversion processing.

FIG. 4 shows examples of skin diseases of patients and typical evaluation methods and treatment methods for each skin disease in this example. In this figure, burns 407, atopic dermatitis 408, and psoriasis 409 are illustrated as types of skin diseases 401. The evaluation method 402 exemplifies a method for evaluating the typical severity of each skin disease. Area estimation method 403 exemplifies a typical method for estimating the area of each skin disease. The evaluation index 404 exemplifies typical indexes for evaluating the severity of each skin disease. The area measurement site 405 is a typical body (human body) site where the area of each skin disease is measured. The treatment method 406 shows typical treatment methods recommended depending on the degree of severity of each skin disease.

The flowchart in FIG. 5 shows the flow of basic image processing (image processing method) executed by the image processing apparatus 101 (arithmetic unit 204) in FIG. 1 according to the image processing program 207. Here, a case will be described in which the imaging device 102 is caused to take an image. The calculation unit 204 functions as an estimation means, a determination means, an evaluation means, and a guide means.

In step S501, when the imaging device 102 performs imaging and generates image data, the image processing device 101 downloads the image data.

Next, in step S502, the image processing apparatus 101 performs image processing (image analysis) on the downloaded image data to detect a skin disease and estimate the type of skin disease. In image analysis, we use machine learning models to detect and estimate the type of skin diseases, and use information such as brightness, color, hue, and gradient of image data to detect and estimate the type of skin disease. Detection/type estimation is performed using feature matching using shape feature information such as. For example, pixels with a brightness gradient above a certain level are extracted as edges, and rule-based detection/type estimation is performed based on the position, center of gravity, inclination, etc. of the edges.

Next, in step S503, the image processing apparatus 101 determines whether or not the type of skin disease could be estimated in step S502, and if the type of skin disease could be estimated, the process proceeds to step S504. If the estimation cannot be made, the image processing apparatus 101 proceeds to step S505, outputs re-imaging guidance, and then returns to step S501. Details of the reshoot guidance will be described later.

In step S504, the image processing device 101 determines the evaluation method 402 according to the type of the estimated skin disease 401 (407-408).

Next, in step S506, the image processing apparatus 101 detects the body of the patient 810 from the image data. Here, the body is detected by image analysis similar to the image analysis for detecting and estimating the type of skin disease described above.

Next, in step S507, the image processing apparatus 101 determines whether the whole body has been photographed based on the detected shape and size of the body, and if the whole body has been photographed, the process advances to step S508. Furthermore, if only one imaging device is used and the whole body cannot be photographed or only a local part of the body can be photographed, the process proceeds to step S509 to output photographing guidance, and additional information is added in step S510. Once photographing is completed, the process returns to step S506. Note that even if only half the body, such as the front or back, is photographed, if the skin disease exists only on the photographed half of the body, the process may proceed to step S508 in accordance with instructions from the photographer.

In step S508, the image processing apparatus 101 estimates the body part in each image data generated by the previous imaging. Estimation of the site is also performed by image analysis similar to the image analysis for detecting/estimating the type of skin disease described above.

Note that the patient's age may be estimated by detecting a face within the image data. If the age can be estimated, the degree of severity of the skin disease can be evaluated in consideration of the age, which improves the accuracy of the estimation result of the severity of the skin disease. In addition, if live view image data (video data), which is image data for display, can be obtained while the imaging in step S501 or step S509 is repeated, the live view image data is also used as interpolation information to estimate the body part. You may do so. This improves the accuracy of part estimation.

Next, in step S511, the image processing apparatus 101 performs image analysis to measure an index indicating the area and severity of the skin disease. In the image analysis performed here, measurements are made in relation to the area and severity of the skin disease area. Specifically, in addition to performing image analysis similar to the image analysis for detecting and estimating the type of skin disease described above, the method uses the area on the image data, the focal length of the imaging device, the area obtained from 3D data, etc. Measure the actual area. In addition, if there is a risk that the compositions of multiple image data obtained from multiple shots may overlap, it is possible to combine these multiple image data first to avoid being affected by the overlap of compositions. good. Furthermore, division or the like may be performed after estimating the part from the image data and detecting the image area of the overlapping part.

Next, in step S512, the image processing apparatus 101 determines the success or failure of the image analysis performed in step S511, and if the image analysis is successful, in step S513, the recommended countermeasure 406 is carried out according to the result of the image analysis. Output. Then, this process ends. On the other hand, if the image analysis fails, the image analysis result is fed back and imaging guidance is output in step S514, and when additional imaging is performed in step S515, the process returns to step S506.

FIG. 6 shows an example of types 601 of shooting guidance and specific contents 602 of shooting guidance. Further, a handling guidance 607 corresponding to the recommended handling method and the contents of the specific handling guidance are also shown.

There are different types of imaging guidance corresponding to each of a plurality of stages in a series of imaging. That is, there is imaging guidance for the initial imaging 603 in step S501 in FIG. 5, re-imaging guidance in step S505, imaging guidance (circled 1) in step S509, and imaging guidance (circled 2) in step S509. These imaging guidances mainly include guidance for instructing a subject to be imaged within the patient's body, guidance for instructing reimaging, and guidance for instructing changes in imaging parameters.

Further, the coping guidance mainly includes guidance indicating treatment methods (external medicine, oral medicine, phototherapy, injection therapy, etc.) and guidance recommending hospital visits or hospitalization (even intensive treatment).

FIG. 8 shows an image displayed on the rear monitor 808 (display device 304) of the imaging device 102 shown in FIG. 3. When the photographer operates the power button 801 while the power of the imaging device 102 is OFF, the power of the imaging device 102 is turned ON, and the CPU 301 causes the rear monitor 808 to display a live view image. When the photographer operates the shooting button 802, the CPU 301 causes the imaging unit 300 to generate still image data or video data for recording, and causes the image processing unit 311 to perform image conversion processing.

When the photographer operates any of the upper button 803, right button 904, lower button 805, and left button 806 included in the input device 305 while a menu item (not shown) is displayed on the rear monitor 808, the CPU 201 Toggle the selected menu item. Further, when the photographer operates the determination button 807, the CPU 201 determines the selected menu item, enables detailed settings for that menu item, holds the setting information in the RAM 303, and switches the settings of the imaging apparatus 102. Note that if the rear monitor 808 is provided with a touch panel, the CPU 201 performs the same operation as when the buttons 803 to 807 are operated by detecting the photographer's touch on the rear monitor 808.

The image displayed on the rear monitor 808 shows a patient (subject) 810 and regions 811 and 812 of skin diseases (for example, burns). Further, a guidance display area 809 is provided within the image. In the guidance display area 809, characters indicating the contents of the photographing guidance shown in FIG. 6 are displayed. The displayed shooting guidance does not have to be text, and may be an icon showing the whole body, the palm of the hand, or a combination of text and icons. By displaying such shooting guidance, it becomes possible to present the necessary shooting area and composition to the photographer in an easy-to-understand manner.

The flowchart in FIG. 7 shows the flow of processing when the skin disease estimation result in step S502 is a burn in the image processing shown in FIG.

When the imaging device 102 photographs the patient 810 and generates image data in step S501, the image processing device 101 downloads (acquires) the image data.

In step S502, the image processing apparatus 101 analyzes the image data to detect skin disease regions 811 and 812, and estimates that the type of skin disease is a burn.

The image processing apparatus 101, which has proceeded to step S504 after making the determination in step S503, determines to evaluate the severity of the burn injury 407 shown in FIG. 4 using % TBSA (Total Body Size Area). %TBSA is the percentage of burn area to total body surface area.

In step S506, the image processing apparatus 101 detects the body of the patient 810 from the image data. If the whole body of the patient 810 has been imaged here, the image processing apparatus 101 proceeds to step S508 and estimates the body part of the patient 810.

FIG. 9 shows an example of part estimation results. In general, when estimating body parts, the granularity of parts is analyzed in such detail that the actual skeleton can be accurately analyzed. For example, in the estimation result 901, the whole body is divided into 14 parts (the parts between the black circles). On the other hand, in other estimation results 902 and 903, sufficient analysis was not performed due to poor conditions such as image resolution, photographic composition, performance of the learning model, and shape of the patient's clothing, and the results were divided into 8 or 6 parts. It's just that. Furthermore, if the conditions deteriorate further and the reliability of part estimation becomes lower, an error is output as in the estimation result 904.

In step S508, the image processing device 101 estimates the age of the patient 810 by detecting a face from the image data as described above, and corrects the severity of the skin disease according to the age estimation result. Here, assuming that the age is estimated to be around 25 years old, the severity evaluation for adults is performed from step S701 onwards.

In step S701, the image processing apparatus 101 checks the granularity (number of divided parts) in the part estimation in step S508, and determines an evaluation method according to the granularity. For example, if the granularity is sufficient as in the estimation result 901 of FIG. 9, in step S702, the law of 9 or Lund and Browder's law shown in FIG. Decide to evaluate. The rule of nines states that the 11 parts of the head, upper limbs (left and right), lower limbs (left and right lower legs, left and right thighs), and trunk (front chest, abdomen, chest back, waist and buttocks) are each 9%, and the genitals are 1%. This is a method to calculate the burn area. Lund and Browder's law is a method for calculating the burn area in more detail by dividing the trunk into smaller pieces than the rule of nines.

Furthermore, if the particle size is not sufficient as in the estimation result 903, it is determined in step S703 that the evaluation is to be performed using the rule of five. The rule of fives is a method for evaluating the burn areas of the head, arms, legs, and chest for infants, children, and adults. For adults, the burn area on the anterior chest or both legs is evaluated by adding 5% of the total area.

Further, if the body part cannot be estimated as shown in the estimation result 904 or if the composition for photographing the whole body is difficult, it is determined in step S704 that the evaluation will be performed using the palmar method. The palmar method is a simple evaluation method that considers the palm as 1% of the whole body, and the palm 1001 is photographed as shown in FIG. Although the palmar method may reduce the accuracy of estimating the area of the affected area, it simplifies imaging and makes it possible to quickly evaluate the severity.

In step S704, the image processing apparatus 101 performs image analysis on the image of each part estimated in step S508, and estimates regions of burn depths I to III and their area ratios.

In step S512, the image processing apparatus 101 determines whether the estimation of the burn area and depth has been successful. If the estimation of the burn area/depth is not sufficiently successful, such as when some affected areas cannot be analyzed well, the photographer uses the imaging guidance in step S514 to instruct the photographer to change the imaging parameters and perform additional imaging. urge. Then, the processing from step S506 is performed again on the image data obtained by the additional photographing in step S15.

If the area/depth of the burn is successfully estimated in step S512, the image processing apparatus 101 proceeds to step S513, and provides guidance on the recommended treatment method according to the position, area, and depth of the burn, as shown in FIG. It is displayed in the display area 809.

In the image shown in FIG. 11, patient 810 and burn areas 811 and 812 as well as patient and burn information 1101 are displayed. Here, it is displayed that the skin disease is a burn and was evaluated using the rule of five, that the patient is an adult estimated to be around 25 years old, and that the severity has been evaluated as severe. The guidance display area 809 displays intensive treatment at an emergency center as a recommended treatment method.

If the displayed estimated age of the patient is significantly different from the known age, the age may be corrected using the buttons 803 to 807 or the touch panel, and the image processing device 101 may perform re-evaluation.

As explained above, the image processing apparatus 101 of this embodiment not only estimates the type of skin disease based on image data obtained by photographing a patient, but also estimates the severity of the skin disease according to the estimated type. Outputs (presents) imaging guidance for acquiring the image data necessary for evaluating the degree of injury. Therefore, even if the photographer has little knowledge of skin diseases, it is possible to acquire (collect) image data necessary for image analysis of skin diseases through efficient imaging. The image processing device 101 also performs image analysis on the collected image data, evaluates the severity of the skin disease, and presents treatment guidance indicating an appropriate treatment method. As a result, even at sites where specialized emergency personnel or doctors are not available, it is possible to determine a more appropriate response method by referring to the presented response methods.

In the above embodiment, a case has been described in which both the photographing guidance and the countermeasure guidance are output, but it is sufficient that at least one of the photographing guidance and the countermeasure guidance can be output.

Furthermore, since the image processing device 101 estimates the patient's age and the site of the skin disease from the image data, it is possible to evaluate the severity with higher accuracy using the year series and the site estimation results.

Further, the image processing apparatus 101 can combine and analyze a plurality of image data when one image data does not include the entire affected area. Therefore, the severity of the skin disease can be evaluated favorably in cases where occlusion exists or the affected area extends to the front and back of the body.

In addition, the image processing device 101 uses a plurality of methods, such as the rule of 9, the rule of 5, the palm method, and the Lund and Browder law, as a method for estimating the burn area percentage (%TBSA) when the skin disease is a burn. can be selected. Therefore, it is possible to select a more suitable severity evaluation method according to the granularity of the part, the number of image data, etc. obtained by analyzing the image data.

In addition, when the skin disease is atopic dermatitis, the image processing device 101 uses an EASI (Eczema Area and Severity Index) that scores the area and symptoms of the disease site as a disease evaluation method. Severity is assessed using the index of Therefore, it is possible to use a suitable severity evaluation method according to the granularity of the site obtained by analyzing the image data. Furthermore, the severity is evaluated according to the area of the atopic dermatitis site and the symptoms of the site, which are estimated from the image data. Therefore, the severity can be evaluated with high accuracy.

In addition, when the skin disease is psoriasis, the image processing device 101 uses BSA (Body Surface Area: ratio of the area of psoriasis to the body surface) or PASI ( Psoriasis Area Severity Index: Select psoriasis area and severity index. Therefore, a more suitable severity evaluation method can be selected depending on the granularity of the site obtained by analyzing the image data. In addition, when the evaluation method is BSA, the severity is evaluated based on the area of the hand estimated from image data and the area of psoriasis on the hand, and when the evaluation method is PASI, the severity is evaluated based on the area of psoriasis estimated from the image data. Assess severity according to area. Therefore, the severity can be evaluated with high accuracy.

Furthermore, even when there are multiple evaluation methods for a skin disease, the image processing apparatus 101 presents the evaluation method used together with the evaluation result. Therefore, it is possible to check the reliability of the presented countermeasures and determine the necessity of re-imaging.

The embodiment described above includes the following configuration.
(Configuration 1)
Estimating means for estimating the type of skin disease of the patient from image data generated by photographing the patient;
determining means for determining an evaluation method for the skin disease according to the estimated type;
an evaluation means for evaluating the severity of the skin disease using the determined evaluation method from the image data;
An image processing apparatus comprising: a guide means for outputting at least one of photographing guidance according to the determined evaluation method and coping guidance according to the evaluated severity.
(Configuration 2)
The image processing device according to configuration 1, wherein the estimating means estimates the type using a machine learning model related to the skin disease.
(Configuration 3)
3. The image processing apparatus according to configuration 1 or 2, wherein, when there is a plurality of image data, the evaluation means evaluates the severity from image data obtained by combining the plurality of image data.
(Configuration 4)
The image processing according to any one of configurations 1 to 3, wherein the evaluation means estimates the age of the patient from the image data and evaluates the severity according to the age estimation result. Device.
(Configuration 5)
According to any one of configurations 1 to 4, the evaluation means estimates a body part of the patient from the image data and evaluates the severity according to the estimation result of the part. Image processing device.
(Configuration 6)
According to configurations 1 to 5, the imaging guidance includes at least one of the following: guidance for instructing a subject to be imaged of the patient's body, guidance for instructing reimaging, and guidance for instructing change of imaging parameters. The image processing device according to any one of the above.
(Configuration 7)
7. The image processing apparatus according to any one of configurations 1 to 6, wherein the treatment guidance includes at least one of guidance indicating a treatment method and guidance recommending hospital visit or hospitalization.
(Configuration 8)
The image processing device according to any one of configurations 1 to 7, wherein the guide means also outputs the determined evaluation method and the evaluated severity level when outputting the coping guidance. .
(Configuration 9)
The determining means determines the evaluation method to be %Total Body Size Area if the type is a burn injury;
Any one of configurations 1 to 8, wherein the evaluation means evaluates the severity according to a burn area and a burn depth relative to the total surface area of the patient's body, respectively estimated from the image data. The image processing device described in .
(Configuration 10)
10. The image processing apparatus according to configuration 9, wherein the evaluation means selects a method for estimating the burn area according to the number of body parts of the patient estimated from the image data.
(Configuration 11)
The determining means determines the evaluation method to be Eczema Area and Severity Index if the type is atopic dermatitis;
Structures 1 to 8, wherein the evaluation means evaluates the severity according to the area of the atopic dermatitis site on the patient's body and the symptoms of the site, respectively estimated from the image data. The image processing device according to any one of the above.
(Configuration 12)
If the type is psoriasis, the determining means determines the evaluation method to be Body Surface Area or Psoriasis Area Severity Index according to the number of body parts of the patient estimated from the image data. The image processing device according to any one of configurations 1 to 11.
(Configuration 13)
The evaluation means is
When the evaluation method is Body Surface Area, the severity is evaluated from the area of the patient's hand estimated from the image data and the area of psoriasis on the hand,
13. The image processing device according to configuration 12, wherein when the evaluation method is Psoriasis Area Severity Index, the severity is evaluated according to the site and area of psoriasis estimated from the image data.
(Configuration 14)
An imaging device comprising the image processing device according to any one of Configurations 1 to 13 and an imaging element that images the patient.

(Other examples)
The present invention provides a system or device with a program that implements one or more functions of the embodiments described above via a network or a storage medium, and one or more processors in a computer of the system or device reads and executes the program. This can also be achieved by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The embodiments described above are merely representative examples, and various modifications and changes can be made to each embodiment when implementing the present invention.

101 Image processing device 102, 104 Imaging device 204 Arithmetic unit

Claims (16)

Estimating means for estimating the type of skin disease of the patient from image data generated by photographing the patient;
determining means for determining an evaluation method for the skin disease according to the estimated type;
an evaluation means for evaluating the severity of the skin disease using the determined evaluation method from the image data;
An image processing apparatus comprising: a guide means for outputting at least one of photographing guidance according to the determined evaluation method and coping guidance according to the evaluated severity. The image processing apparatus according to claim 1, wherein the estimating means estimates the type using a machine learning model related to the skin disease. 2. The image processing apparatus according to claim 1, wherein when there is a plurality of image data, the evaluation means evaluates the severity based on image data obtained by combining the plurality of image data. The image processing apparatus according to claim 1, wherein the evaluation means estimates the age of the patient from the image data and evaluates the severity according to the age estimation result. The image processing apparatus according to claim 1, wherein the evaluation means estimates a body part of the patient from the image data, and evaluates the severity according to the estimation result of the part. 2. The imaging guidance includes at least one of guidance for instructing a subject to be imaged of the patient's body, guidance for instructing reimaging, and guidance for instructing change of imaging parameters. image processing device. The image processing apparatus according to claim 1, wherein the treatment guidance includes at least one of guidance indicating a treatment method and guidance recommending hospital visit or hospitalization. The image processing apparatus according to claim 1, wherein the guide means also outputs the determined evaluation method and the evaluated severity level when outputting the coping guidance. The determining means determines the evaluation method to be %Total Body Size Area if the type is a burn injury;
The image processing apparatus according to claim 1, wherein the evaluation means evaluates the severity according to a burn area and a burn depth relative to the entire surface area of the patient's body, respectively estimated from the image data. . 10. The image processing apparatus according to claim 9, wherein the evaluation means selects a method for estimating the burn area according to the number of body parts of the patient estimated from the image data. The determining means determines the evaluation method to be Eczema Area and Severity Index if the type is atopic dermatitis;
2. The evaluation means evaluates the severity according to the area of the atopic dermatitis site on the patient's body and the symptoms of the site, respectively estimated from the image data. image processing device. If the type is psoriasis, the determining means determines the evaluation method to be Body Surface Area or Psoriasis Area Severity Index according to the number of body parts of the patient estimated from the image data. The image processing apparatus according to claim 1. The evaluation means is
When the evaluation method is Body Surface Area, the severity is evaluated from the area of the patient's hand estimated from the image data and the area of psoriasis on the hand,
13. The image processing apparatus according to claim 12, wherein when the evaluation method is Psoriasis Area Severity Index, the severity is evaluated according to the site and area of psoriasis estimated from the image data. An image processing device according to any one of claims 1 to 13,
An imaging device comprising: an imaging element that images the patient. estimating the type of skin disease of the patient from image data generated by photographing the patient;
determining an evaluation method for the skin disease according to the estimated type;
Evaluating the severity of the skin disease using the determined evaluation method from the image data;
An image processing method comprising the step of outputting at least one of photographing guidance according to the determined evaluation method and coping guidance according to the evaluated severity. A program that causes a computer to execute processing according to the image processing method according to claim 15.